The design of multiple experiments is commonly undertaken via suboptimal
strategies, such as batch (open-loop) design that omits feedback or greedy
(myopic) design that does not account for future effects. This paper introduces
new strategies for the optimal design of sequential experiments. First, we
rigorously formulate the general sequential optimal experimental design (sOED)
problem as a dynamic program. Batch and greedy designs are shown to result from
special cases of this formulation. We then focus on sOED for parameter
inference, adopting a Bayesian formulation with an information theoretic design
objective. To make the problem tractable, we develop new numerical approaches
for nonlinear design with continuous parameter, design, and observation spaces.
We approximate the optimal policy by using backward induction with regression
to construct and refine value function approximations in the dynamic program.
The proposed algorithm iteratively generates trajectories via exploration and
exploitation to improve approximation accuracy in frequently visited regions of
the state space. Numerical results are verified against analytical solutions in
a linear-Gaussian setting. Advantages over batch and greedy design are then
demonstrated on a nonlinear source inversion problem where we seek an optimal
policy for sequential sensing.