as energy terms are those by Black and Anandan . In their work, they propose
causality of the solution in the form of the temporal term. Roughly speaking, en-
ergy based methods incorporating temporal information can be divided into causal-
and batch processing. In the case of causal processing, a solution calculated at t is
propagated forward in time, for example to t + 1, and then is used to improve tem-
poral coherence of the solution starting from t + 1. On the other hand, in the case of
batch processing, the complete sequence of interest is processed at once: in this case,
a 3D-regularization (smoothness) term is needed. Batch type processing methods are
less suitable for real-time implementations than the causal type due to their increased
demand of processing power. More recent methods incorporating temporal terms are
those of Werlberger et al. , Weickert and Schn
orr , and by Salgado and S
. The ﬁrst, by Werlberger, can be regarded to be causal whereas the latter belong to
the batch type. As is mentioned in both  and , incorporating temporal informa-
tion raises an additional challenge of modeling the movement between several frames.
If the movement is modeled by being symmetrical both forward and backward in time
between several frames, this imposes restrictions on the movement: it is expected to be
of a constant velocity (acceleration zero). As can be expected, and is shown in ,
models accepting only constant velocities do ﬁne as long as this assumption is not bro-
ken and actually perform worse when it does not hold. In our causal model, movement
is modeled by having both the velocity and acceleration components, as in , and
therefore, the model does not suffer from this shortcoming.
Our work differs from the above mentioned ones in that we use geometrical knowl-
edge of the scene for constraining the disparity solution and incorporate both the spatial-
and temporal constraints simultaneously for the optical-ﬂow. We also give an example
of a hypothesis forming-validation loop: a hypothesis of a plane is made based on
the data after which, iteratively, the hypothesis is veriﬁed against the data and used to
constrain the solution.
2 Variational Stereo and Optical-ﬂow
Since variational methods incorporate a regularization term, they are, in general, robust
and relatively well performing. An additional beneﬁt is that they have a solid mathe-
matical base, with existing efﬁcient solvers , they are well understood with clearly
deﬁnable terms and therefore, extending these methods is straightforward .
We start by introducing the used notation and then, continue to the energy functionals
with the added spatial and temporal constraint terms. Finally, each of the terms is
described in more detail.
Table 1 has been included for convenience, since three different types of error func-
tions are used in the model. The use of each error function will be justiﬁed later on in
the text. In the data terms, I
refers to a k:th channel of left or right image, deﬁned
by subindex L or R, at time t. By channel here we mean channel of a vector valued
image, such as RGB. Without k written explicitly, all channels are referred. I
refers to a warped version of the image . In the error function case, subindices
refer to functionality or, in other words, how the error function in question is used in
the model. Thus, D, R, CS, and CT refer to data, regularization, constraint-spatial, and
constraint-temporal, respectively. The functionals for stereo and optical-ﬂow are given
in (1) and (2), respectively.