See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/317129116

Generalized Gromov method for stochastic particle flow filters

Conference Paper · April 2017

DOI: 10.1117/12.2248723

CITATIONS READS

4 133

3 authors, including:

Fred Daum
Raytheon Company
113 PUBLICATIONS   2,076 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Fred Daum on 25 May 2017.

The user has requested enhancement of the downloaded file.

 Generalized Gromov method for stochastic
particle flow filters

Fred Daum, Jim Huang & Arjang Noushin

Raytheon
Woburn MA USA
daum@raytheon.com

Abstract—We describe a new algorithm for stochastic particle flow filters using Gromov’s method. We derive a simple
exact formula for Q in certain special cases. The purpose of using stochastic particle flow is two fold: improve estimation
accuracy of the state vector and improve the accuracy of uncertainty quantification. Q is the covariance matrix of the
diffusion for particle flow corresponding to Bayes’ rule.

Keywords- particle filter, nonlinear filter, particle flow, transport problem, extended Kalman filter, Monge-Kantorovich
optimal transport

I. INTRODUCTION
Particle filters generally suffer from the curse of dimensionality owing to particle degeneracy, which is caused by
Bayes’ rule, as explained in [7]. The root cause of particle degeneracy is that the particles are in the wrong place to
compute Bayes’ rule efficiently. Particle flow is an algorithm to compute Bayes’ rule using the flow of particles
rather than as the multiplication of two functions. The purpose of the particle flow algorithm is to mitigate the
problem of particle degeneracy for particle filters, by moving the particles to the correct region in state space to
represent the conditional probability density efficiently. We derive a law of motion for the random set of particles,
analogous to physics. If the initial random set of particles corresponds to the prior density, then the final set of
particles resulting from the flow corresponds to the posteriori density. We can prove this under mild regularity
conditions on the densities (e.g., smooth nowhere vanishing densities on a convex subset of a d-dimensional smooth
manifold); see [29] for details. A refreshingly honest survey of standard particle filters is in [10], the best
introduction to which is [15], with more details on nonlinear filters in [4] and [11]. In this paper we derive a simple
exact formula for the covariance matrix (Q) of the diffusion for our flow, and we explain that this simple concrete
formula is an example of a very general theorem of Gromov [78]. Numerical experiments which show the utility of
this theory are given in our companion paper [68].

II. PARTICLE FLOW FOR BAYES’ RULE

The purpose of particle flow is to fix the problem called “particle degeneracy” as explained in [7]. Particle
degeneracy is caused by Bayes’ rule, and it completely destroys the performance of standard particle filters for high
dimensional state vectors. A “particle” is a point in d-dimensional state space, and we represent the numerical values
of the probability densities at each particle in the nonlinear filter; this is the standard approach in essentially all
particle filters [10]. We denote the value of each particle in d-dimensional space by the symbol x(λ), and we consider
x to be a function of λ. This is a novel viewpoint that is not used in any standard particle filters.
We start by defining a flow of the logarithm of the conditional probability density function with respect to λ:
log p( x, λ ) = log g ( x) + λ log h( x) − log K (λ ) (1)
in which p(x, λ) is the conditional probability density of x as a function of λ, h(x) is the likelihood, g(x) is the prior
density of x and K(λ) is the normalization of the conditional probability density. The flow of the probability density
function defined by (1) corresponds to Bayes’ rule for a single discrete time measurement; in particular, p(x) = g(x) at
λ = 0, and p(x) = g(x)h(x)/K at λ = 1 (see [5] for details). The parameter λ varies continuously from 0 to 1 along the
flow of particles (from the prior to the posteriori), and it is like time, but it is distinct from time. We suppose that the
flow of particles for Bayes’ rule obeys the following Ito stochastic differential equation:

Signal Processing, Sensor/Information Fusion, and Target Recognition XXVI, edited by Ivan Kadar, Proc.
of SPIE Vol. 10200, 102000I · © 2017 SPIE · CCC code: 0277-786X/17/$18 · doi: 10.1117/12.2248723

Proc. of SPIE Vol. 10200 102000I-1

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 dx = f ( x, λ )dλ + dw (2)
Moreover, given the flow of the conditional probability density p(x, λ) defined by (1), we can compute the function
f(x, λ) by using the Fokker-Planck equation:

∂p 1  ∂p 
= − div( fp ) + div Q ( x)  (3)
∂λ 2  ∂x 
in which Q is the covariance matrix of the diffusion w = w(λ), and div(.) is the divergence of (.). The diffusion w(λ)
should not be confused with so-called “process noise” of the dynamical model of x(t). Process noise is a random
process that evolves in time, whereas w(λ) evolves with λ. Moreover, Q is not the covariance matrix of the process
noise, because w is not process noise. If we assume that the flow of particles is deterministic, then Q = 0 and the PDE
simplifies significantly. We have derived many deterministic flows as shown in figure 1 of [38] with details given in
the references listed here. Our deterministic flows are similar to optimal transport algorithms, but our particle flows
are much simpler and faster to compute, because we avoid solving a variational problem, as explained in [21].

As explained in [27], the best particle flows avoid explicitly normalizing the conditional probability density; we have
learned this from many numerical experiments as well as theory. It is easy to derive a PDE for particle flow that
avoids normalization of the conditional density. In particular, plug (1) into (3) and take the gradient, and we get the
following PDE for the flow f:

 ∂p 
2 ∂ div (Q ) / 2 p 
∂ log h ∂ log p ∂div( f ) ∂ log p ∂f
+  
∂x
=−fT 2
− − (4)
∂x ∂x ∂x ∂x ∂x ∂x
This PDE does not depend on the normalization K(λ), because we killed it by taking the gradient with respect to x.
That is, multiply p(x, λ) or h(x) by any non-zero constant factor, and the PDE does not change at all. One can
simplify this equation by picking Q = 0 or else picking Q to have values that make nice things happen, in which case
we can actually solve (4) exactly as a formula. But in this paper we will compute a simple formula for Q to improve
estimation accuracy and also improve uncertainty quantification (UQ). There are many different ways to pick the
matrix Q, as explained in [39].
We would like to avoid computing the conditional probability density p itself, owing to severe numerical problems for
particles in the tails of the distribution for high dimensional state vectors, and it is easy to write the last term in (4)
using the log of p rather than p itself with the help of the following well known identity:

 ∂p 
div  T
 ∂x  = div ∂ log p  +  ∂ log p  ∂ log p  (5)
    
p  ∂x   ∂x  ∂x 

III. SIMPLE EXACT FORMULA FOR Q

Recall that our geodesic particle flow derived in [20] and [25] has the form of equation (2) with:
−1 T
 ∂ 2 log p   ∂ log h 
f = − 2    (6)
 ∂x   ∂x 
and the covariance matrix of the random process w(λ) is given by Q. That is,

dwdwT = Qdλ (7)

Proc. of SPIE Vol. 10200 102000I-2

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 This flow was derived in [20] and [25] by assuming that there exists a Q such that the last three terms in the PDE (4)
sum to zero. The computation of Q is highly underdetermined for d ≥ 2, because we have d equations in n unknowns,
in which
n = d (d + 1) / 2 (8)
But if we assume that the prior density (g) and the likelihood (h) are both multivariate Gaussian densities, and if we
assume that Q is a symmetric positive semi-definite matrix independent of x, then we can derive a simple exact
formula for Q as follows:

[
Q = P −1 + λH T R −1H ] −1
[
H T R −1 H P −1 + λH T R −1 H ]
−1
(9)
This simple exact formula was recently derived jointly by Rob Fuentes and Eric Blake [44]. In equation (9) R
denotes the measurement noise covariance matrix, P is the error covariance matrix of x prior to a given measurement,
and H is the measurement sensitivity matrix for a linear measurement model: z = Hx + v, in which v is Gaussian zero
mean measurement noise. If the measurements are nonlinear in x, then we can compute H using the standard
linearization approach as in extended Kalman filters. Numerical experiments show that this is an excellent
approximation for the examples that we have studied so far [68]. In many practical applications, one can pick a
coordinate system in which the measurements are exactly linear in x, in which case our theory works without any
approximations. We have used this simple trick in many real world applications of EKFs with radars over the last
five decades; for example, see [4] and [67]. One can also derive a simple exact formula for Q, which is essentially the
same as (9), assuming Gaussian prior and Gaussian likelihood, without the additional assumption of geodesic flow.
However, the geodesic flow is generally better than the Gaussian flow for nonlinear non-Gaussian applications. The
Gaussian flow was derived in [5] assuming Q = 0, but it is easy to generalize this exact particle flow for non-zero Q
given by equation (9).
We can avoid computing large matrix inverses by using Woodbury’s matrix inversion formula as follows:
−1
1 
(P −1
+ λH T R −1 H )
−1
= P − PH T  R + HPH T  HP (10)
λ 
and hence (9) becomes:

 T 1 T 
−1
 T −1  T 1 T 
−1
 (11)
Q =  P − PH  R + HPH  HP  H R H  P − PH  R + HPH  HP 
 λ    λ  
To avoid dividing by zero when λ = 0 (which always happens at the start of the flow), we can use the equivalent form:

(12)
[
Q = P − λPH T R + λHPH T ( )
−1
] [
HP H T R −1 H P − λPH T R + λHPH T ( )
−1
HP ]
We need to compute the square root of the matrix Q in order to generate the random variables dw for use in the actual
stochastic flow (2). Such a real valued matrix square root always exists because Q is symmetric and positive semi-
definite (in theory). We have used the MATLAB matrix square root primitives to do this, but very small numerical
errors in computing Q can easily result in a non-symmetric matrix for Q (or maybe a matrix that is not positive semi-
definite). In such cases MATLAB will complain that the square root of Q does not exist, or it will give us a complex
valued square root. To avoid such numerical troubles, it is very easy to symmetrize Q immediately before computing
the matrix square root:

Q + QT
Q→ (13)
2

Proc. of SPIE Vol. 10200 102000I-3

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 This is a good idea for any covariance matrix in general for EKFs or UKFs or particle filters. Furthermore, one might
force Q to be positive definite by using Tychonov regularization; that is, add a small positive multiple of the identity
matrix to Q. But we have found that Tychonov regularization is unnecessary for our applications so far using the
appropriate matrix square root with MATLAB, as long as we symmetrize Q first. But other less robust software tools
than MATLAB or more stressing applications may need this extra help. Tychonov regularization of Q is fast and
easy, with no down side in practice. In theory, however, Q is positive semi-definite, but it is generally not positive
definite (see discussion of this theoretical point in the next paragraph). One could also use an SVD and take the scalar
square roots of the positive singular values of Q rather than computing the matrix square root directly.

There are two important properties of our simple exact formula for Q in equation (9). First, Q is zero for
uninformative measurements. For example, let R be very large corresponding to useless data; inspection of (9) shows
that Q is essentially zero in this case. This is a crucial feature of any good Q for particle flow. Obviously the
particles should not move if the measurement is not informative! Hence, it is clearly a bad idea to ask for Q that is
always positive definite. More specifically, it is a bad idea to ask for a Q that is a positive multiple of the identity
matrix, say Q = αI, and compute a fixed α using Monte Carlo simulations analogous to tuning the process noise
covariance matrix for a Kalman filter. Nevertheless, several popular MCMC algorithms violate this obvious law of
good particle behavior. Second, inspection of equation (9) shows that Q is generally not full rank, because H is rarely
full rank in any practical applications. At first this seemed like an undesirable feature, because we generally prefer
positive definite covariance matrices for diffusion tensors. For example, we like positive definite Q for Kalman
filters, because this tends to stabilize the filter, because this makes the plant model obviously and robustly completely
controllable. Stability of the Kalman filter is guaranteed by a completely controllable plant model and a completely
observable measurement model, according to Kalman’s beautiful theorem [43]. More generally, for nonlinear filters,
a positive definite Q avoids a degenerate Fokker-Planck equation, although one can live with hypo-elliptic PDEs
(which correspond to controllable plant models). Upon further reflection, we see that Q cannot always be full rank,
because certain directions in state space are not informed by a given measurement, and hence we do not want the
particles to move in those directions. In summary, rather than being undesirable, it is crucial that Q is not always full
rank. The Tychonov regularization of Q mentioned in the last paragraph results in a very small motion of the
particles for non-informative measurements, but this has no practical effect on filter state vector estimation accuracy
or filter uncertainty quantification accuracy.

IV. GROMOV’S METHOD

The simple exact formula for Q shown in equation (9) is a concrete example of the application of a very general
theorem of Gromov. In particular, on pages 148 to 149 of Gromov’s book [40], we find a rather surprising theorem
about the existence of solutions to linear underdetermined PDEs. In particular, Gromov shows that there exists a very
nice solution to this system of PDEs if and only if it is sufficiently underdetermined. A “very nice solution” in this
context means one that does not require integration, but rather only uses differentiation and algebra. A “sufficiently
underdetermined” system of PDEs means that the number of unknowns is at least the number of equations plus the
dimension of x. This is a very strange theorem for several reasons. First, solving PDEs without any integration
seems too good to be true. Second, when was the last time that you heard a theorem that requires the number of
unknowns to be sufficiently large? Also, there are several caveats: (1) Gromov’s solution to the PDE system is not
the most general solution, and it could be very far from the most general solution; and (2) the solution is formal and
local.
We were aware of Gromov’s theorem for over seven years, as discussed in [8], and it seemed that Gromov’s method
could be useful for us, because we were trying to solve a highly underdetermined linear first order PDE. But the
trouble was that our PDE was not sufficiently underdetermined to satisfy the hypotheses of Gromov’s theorem. In
particular, with zero diffusion, we had a single scalar-valued PDE with a d-dimensional unknown function. This is
highly underdetermined, but not underdetermined enough. We needed more unknowns! But our recent attempts to
compute Q solved this problem. Q is a whole matrix of unknowns, and posing the problem as jointly finding f and Q
has plenty of unknowns compared with the number of equations. In summary, two open problems solved each other:
(1) how to compute f and Q jointly, and (2) how to apply Gromov’s theorem to particle flow. In summary, Gromov’s
theorem does not apply to deterministic particle flow, but it always applies to stochastic particle flow. This is yet
another benefit of stochastic vs. deterministic methods.

Proc. of SPIE Vol. 10200 102000I-4

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 There is another reason that Gromov’s theorem cannot apply to deterministic flows. Recall that the PDE for our
deterministic flows is:
div( pf ) = η (14)
in which η is defined in [27]. As explained in [8], it is well known that a necessary and sufficient condition for the
existence of a solution to this PDE is that the volume integral of η is zero. That is, a solution of (14) exists if and only
if the following condition is satisfied:

∫η ( x, λ )dx = 0
Ω
(15)

This existence theorem is valid even for the weakest notion of solution, including formal and local solutions. But
Gromov’s theorem does not mention any such condition for the existence of a solution to the underdetermined system
of PDEs. Hence, Gromov’s theorem does not apply to (14), because it is not sufficiently underdetermined. This was
bothering us for many years. Moreover, the addition of more unknowns avoids the stringent requirement of equation
(15) by invoking Gromov’s theorem. That is, the extra degrees of freedom using non-zero Q allows us to balance the
RHS and LHS of our PDE and thereby avoid the requirement in (15). Furthermore, we conjecture that transport
problems using non-zero diffusion will be less sensitive to errors in computing the normalization of the conditional
probability density, because normalization of the density is equivalent to condition (15). This connection is explained
in [27].
The mathematical literature on optimal transport almost always assumes zero diffusion (Q = 0). In particular, Villani
mentions the possibility of random transport in passing, but there are no theorems or algorithms for such stochastic
transport in [45] or [46], and there is very little mention of stochastic transport in [47]. Moreover, there are very few
papers on stochastic optimal transport, with the conspicuous exception of [48], as far as we know. But it seems
intuitively that such stochastic transport would be useful for the reasons given above in connection with Gromov’s
theorem avoiding the stringent and unforgiving requirement of (15).
Gromov is one of the very few mathematicians who study underdetermined PDEs seriously (see the foreword to [40]
for elaboration on this point). Underdetermined linear PDEs are extremely simple, and hence they are considered
beneath contempt by most research mathematicians; for example, Trèves says that: “underdetermined systems [of
PDEs] are much easier to study than overdetermined ones (and in most cases, than determined ones also); they are
also much less interesting” on page 9 in [49]. Naturally, Gromov and Lanczos and Hilbert and Schmidt and we do
not agree with the last part of Trèves’ assertion. (The generalized inverse of linear differential operators was invented
by Hilbert & Schmidt several decades before Moore & Penrose reinvented the generalized inverse for matrices; see
Nashed [52] for a good historical survey). Moreover, an underdetermined PDE is “ill-posed” according to the
standard textbook definitions. The solution to this ill-posed problem in optimal transport theory is to ask for the
unique solution of the transport PDE that minimizes some convex functional of the flow of particles (e.g., Wasserstein
distance). But this solution comes at the extremely high computational cost of solving a variational problem. We
prefer to avoid solving such a variational problem.
To dispel the mystery about Gromov’s method, let us consider a nice simple example and show the exact solution to
the PDE. The simplest non-trivial example is the following underdetermined PDE:
∂q1 ∂q2
+ + q3 = η ( x) (16)
∂x1 ∂x2
in which η(x) is a given smooth function of x, and we have 3 unknowns and one scalar-valued equation. We can
write this in the more concise form: Lq = η, in which L is a linear differential operator and q is the vector of
unknowns. Let us try to find a solution of the form q = Mη, in which M is a linear differential operator, the existence
of which is promised by Gromov’s theorem. Plugging our solution into the PDE, we get LMη = η, for arbitrary
smooth functions η(x), which is the same as LM = 1. Taking the formal adjoint we get: M*L* = 1. This is the
equation that we need to solve for M. By inspection, we get the following exact simple solution:
T
 ∂η ∂η 
q = Mη =  − , ,η  (17)
 ∂x2 ∂x1 

Proc. of SPIE Vol. 10200 102000I-5

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 Notice that this is an exact solution to our PDE (16) that requires no integration, but rather only differentiation and
algebra, as promised by Gromov’s theorem. To check that this is indeed an exact solution, we plug this q into our
original PDE (16), and we get:

∂ 2η ∂ 2η
− + +η = η (18)
∂x1∂x2 ∂x2∂x1
For functions η(x) that are at least twice continuously differentiable with respect to x, the mixed partials are equal,
and hence (18) is an identity for all smooth η(x). We can easily generalize this simple example to arbitrarily high
dimensions of x, as well as much more general linear operators L. But thinking about this simple example gives the
basic intuitive understanding of what is actually going on with Gromov’s method. Notice that this method would not
work unless the PDE was sufficiently underdetermined, as per Gromov’s theorem. Two unknowns and one equation
is not enough, but rather we need at least three unknowns and one equation for x with dimension d = 2.
We can generalize this example as follows. Consider the single scalar-valued linear underdetermined PDE:

div(r ) + bT s = η (19)
in which q = (r, s), where r is d dimensional, and s has arbitrarily high dimension. One possible family of exact
solutions to (19) is:
T T
 ∂η   ∂θ 
r = A  + B   + β (x) (20)
 ∂x   ∂x 

bη  bbT 
s= 2
+ I − 2  y (21)
b  b 
in which A and B are arbitrary skew-symmetric matrices that are independent of x, and both θ and η are arbitrary
functions of x that are twice continuously differentiable, and β is an arbitrary smooth function of x such that the ith
component of β is not a function of the ith element of x, and y is a completely arbitrary vector to match the dimension
of s. It is easy to check that this is an exact solution to (19) using the fact that the Hessians of η and θ are symmetric
matrices, because η and θ are smooth functions of x. Moreover, we can generalize this example to arbitrarily high
number of equations. This simple example makes it clear why Gromov’s theorem assumes that the number of
unknowns is at least equal to the number of equations plus the dimension of x. Apparently our solution in (20) and
(21) is more general than Gromov’s solution q = Mη.
Suppose that we have an exact solution (or an excellent approximation) of the following PDE:
r ) = η~
div(~ (22)
We can use this information to generalize our solution of (15) even further. In particular, add (22) to (19):

r ) + bT s = η + η~
div(r + ~ (23)
Now we can compute a family of solutions for (23) analogous to our solution given in (20) and (21) as follows:
T T
 ∂η   ∂θ 
r = A  + B   + β (x) - ~r (24)
 ∂x   ∂x 

b(η + η~ )  bbT 
s= 2
+ I − 2  y (25)
b  b 
in which A, B, y, β, and θ have the same meanings as given above. This family of solutions is more general than that
given in (20) and (21). We can generalize this still further by taking linear combinations of solutions of PDEs like
(22), which results in:

Proc. of SPIE Vol. 10200 102000I-6

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 T T
 ∂η   ∂θ 
r = A  + B   + β (x) - ∑ w k ~rk (26)
 ∂x   ∂x  Λ

b(η + ∑ wkη~k ) 
bbT 
s= Λ
2
+  I − 2
y (27)
b  b 

in which Λ denotes the set of solutions of the PDE (22) that we wish to exploit, and wk is an arbitrary real number
that weights the kth solution of (22).
It is easy to manufacture a plethora of solutions of (22) by starting with ~r and computing the divergence to find out
what the corresponding η~ is. This is the reverse of the usual process of “solving” a PDE, and it is obviously much
easier. Solutions of (22) can be computed off-line and stored in memory to be used in real time. For d ≥ 2 the PDE in
(22) is underdetermined, and hence we have many solutions of (22) to choose from. The vector y can be chosen to
stabilize the particle flow (e.g., as shown in [16]), or we could choose y to minimize the sensitivity of the flow to
errors in the normalization of the conditional density (e.g., as shown in [26]). One can also imagine other methods to
choose y.
These solutions of our PDE generalize Gromov’s solution in several ways. First, we have not assumed as much
smoothness as Gromov assumed; in particular, we assume that the functions are twice continuously differentiable;
∞
that is, we work in C², whereas Gromov works in C . Second, Gromov assumes that the coefficients of the PDE are
constant (wrt x and λ), whereas we do not make such an assumption. Third, our solutions are apparently more general
than Gromov’s solutions, as explained above (e.g. equations (26) and (27)). It is often said that Gromov works in a
very general and abstract setting, which is certainly true, but nevertheless we have generalized Gromov’s solution of
linear constant coefficient underdetermined PDEs in the three specific ways explained above.
How is it possible that such a simple exact solution of PDEs eluded the grasp of Goursat [8] and Courant & Hilbert
[8] and Forsyth [8]? It is because these mathematicians did not consider underdetermined PDEs worth serious study,
because they are “ill-posed” and “non-physical.” Moreover, even if such mathematicians had studied
underdetermined PDEs, who would imagine that they needed the PDE to be “sufficiently underdetermined” in
Gromov’s sense? Furthermore, who would have imagined that underdetermined PDEs would be useful for anything?
In contrast with the simple examples given above in (16) to (27), Gromov’s book is famously difficult to read. In fact
the book by Spring [50] was intended to help mathematicians understand [40] by filling in all the details in the proofs,
and also by stating the theorems more completely and more precisely. But Spring’s book is of little help to normal
engineers, because it does not discuss the relevant theorem on page 149 in Gromov’s book at all, and it is inscrutable
for normal engineers who do not have jet bundles and fiber bundles for breakfast. One might instead consult Lanczos
[51] for a much more readable treatment of underdetermined PDEs, with simple explicit examples, although it
obviously does not cover Gromov’s work. One must be careful about the meaning of certain mathematical terms in
[40]; for example, when Gromov says “inverse” of a linear operator, he really means “right inverse” or sometimes
“left inverse.” Also, the theorem in Gromov’s book is shrouded in rather general and abstract structures, which
normal engineers do not have for breakfast. The more recent exposition on page 40 of Gromov’s paper [41] is
somewhat easier to understand than Gromov’s book [40]. In addition, Yang’s note [42] gives several examples that
make Gromov’s theory easier to grasp by normal engineers. Another important reason to study Yang’s paper is to
understand the generalization of Gromov’s method using exterior differential forms [42]. The generalization of
Gromov’s method suggested by Yang [42] is roughly analogous to our simple examples in equations (19) to (27),
except that we used linear algebra and elementary calculus rather than Cartan’s exterior calculus; also, Yang’s
examples give solutions to ODEs, whereas we solve PDEs. The solutions to the PDE (19) given here in (20) to (27)
were invented using Polya’s method, as explained on page 357 of Seiler’s superb book [53].
Both Hilbert and Cartan attempted to find solutions to underdetermined differential equations without using any
integration but only algebra and differentiation, similar to Gromov’s theorem, but without success. These
unsuccessful attempts are surveyed on page 339 of Seiler’s book [53]. In a certain sense Gromov’s theorem is
intuitively obvious once it is pointed out. In particular, as explained before, the plethora of unknowns play the role of
extra degrees of freedom, which we can use to balance the RHS and LHS of our PDE and avoid integration wrt x or λ;
the intuitive idea of Gromov’s theorem is that simple. Of course, explaining why we need exactly so many unknowns

Proc. of SPIE Vol. 10200 102000I-7

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 is not intuitive, except in the context of our simplest non-trivial example (16) to (18) given here. Also, the proof of
Gromov’s theorem is not deep; it only takes about one half page of tedious but straightforward linear algebra with
linear differential operators (e.g., see [76] for a nice clear elementary proof of Gromov’s theorem). Alternatively, one
could prove this by taking the Fourier transform of the PDE, resulting in a linear algebraic system of equations, which
can be analyzed using the well known fact that the range space of a linear operator A is equal to the orthogonal
complement of the null space of the adjoint of A. Gromov did not bother to provide a proof of his theorem, and he
did not bother to state this result as a theorem in [40] or elsewhere. Nevertheless, Gromov’s brilliant insight eluded
other famous researchers, including Hilbert and Cartan.

FUTURE RESEARCH

For future research, we plan to ruthlessly exploit Gromov’s method (and generalizations thereof) to derive new
particle flow filter algorithms. First, we could compute Q and f jointly by solving the PDE (4) without assuming that
the last three terms in (4) sum to zero. There are many variations on this theme. Second, we could compute f and Q
jointly by solving (4) assuming that the conditional probability densities are from the multivariate exponential family
[4], rather than from the Gaussian family of probability densities. We can compute the exact solution of the Fokker-
Planck equation for the multivariate exponential family using separation of variables (see [56] and [77]). Using this
theory we can explicitly compute f and Q by solving a system of underdetermined linear algebraic equations, using
the Moore-Penrose inverse and the projection into the null space of the relevant linear operator, very similar to (27).
Hence, we consider this a very promising direction of research. Third, we could compute f and Q jointly by
assuming that the measurement is an arbitrary smooth nonlinear function of x with additive Gaussian noise and a
multivariate Gaussian prior probability density of x. This is obviously a special case of the second idea, but allows us
to obtain more explicit formulas.
Fourth, we could compute f and Q jointly by assuming that the last three terms in our PDE (4) sum to zero, and solve
for Q using the value of f given in (6) along with the constraint that Q is a symmetric positive semi-definite matrix.
For example, we could factorize Q using the Cholesky square root, which would enforce the obvious constraints for a
covariance matrix. This would result in a quadratic algebraic system of equations for SQRT(Q), which can be solved
explicitly as a formula using ideas similar to [19], at least in certain special cases. Recall that in [19] we solved a
vector Riccati equation explicitly as a formula for the particle flow. If Q is assumed to be invariant with respect to x,
then the PDE (4) is a linear algebraic system in Q, and one could also consider solving this linear algebraic system
directly without the explicit constraint on Q as a covariance matrix; this approach in fact works in certain cases,
including the Gaussian example given in this paper. In particular, in order to derive (9) we did not explicitly enforce
any constraint on Q as a covariance matrix. One can also use a homotopy method to compute SQRT(Q) starting with
the initial value of SQRT(Q) given by (9); this homotopy would morph both the PDE and the solution jointly. Other
initial values of SQRT(Q) are also possible (e.g., the identity matrix), but such methods do not exploit our knowledge
of exact solutions of our PDE for related problems. One can imagine many other methods to compute Q or SQRT(Q)
for general nonlinear filter problems with smooth nowhere vanishing conditional probability densities p, including
Newton’s method or quasi-Newton methods or hybrids of homotopy and Newton’s method. The control theory
literature on solving algebraic matrix Riccati equations is vast, and it can be used fruitfully here. For example, we
know how to solve the algebraic matrix Riccati equation for a unique positive semi-definite solution by integrating
the corresponding Riccati matrix differential equation to steady state, assuming a positive definite initial condition
and positive semi-definite matrix coefficients, which correspond to natural controllability conditions on the relevant
linear dynamical system; that is, no explicit constraints to obtain positive definite solutions are needed; this occurs
naturally under the right conditions. This control theory method is analogous to a homotopy algorithm described
earlier, in that we are integrating a matrix differential equation with respect to a scalar. We emphasize that these
methods do not make any assumptions about the dynamics or measurements or conditional density or likelihood other
than smoothness and nowhere vanishing conditional densities. In particular, we do not assume Gaussian priors or
densities from the multivariate exponential family or linear measurements or linear dynamics for the purpose of
computing Q or f. That is, this approach is much more general than several of the earlier methods suggested.
Fifth, it would be nice to have a geometric (i.e., coordinate free) version of our solutions to (4) and (15), perhaps
using exterior differential systems, as suggested by Yang [42]; however, whether this mathematical flourish is
actually useful to obtain new and better solutions in practice remains to be seen.
Sixth, we would like to compute the most general family of solutions to our PDE, rather than a series of special
families of solutions. Maybe completion of the PDE system to involution (in the style of Kähler) would be useful to

Proc. of SPIE Vol. 10200 102000I-8

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 find the most general solution of our PDE, as explained in Seiler’s superb book [53]. Seiler believes that “involution
is the central principle in the theory of under-determined systems [of PDEs]” (page xi in [53]). A gentle introduction
to involution for solving PDEs is in [54]. In addition to Yang’s ODE examples [42], one reason to expect that
Cartan’s exterior calculus would be useful here is shown by our first non-trivial example. In particular, our solution
to (16) works because of the equality of mixed second partial derivatives as shown in (18); and this is essentially the
same as d² = 0, which is built into Cartan’s calculus by construction, where d denotes the exterior derivative (rather
than dimension). A second reason to expect that Cartan-Kähler theory would be helpful here is that it studies
“solutions to a system of arbitrarily many differential equations with arbitrarily many unknowns” (page 1 in [57]).
On the other hand, Cartan-Kähler theory is not necessarily the royal road to solving underdetermined PDEs; see
Hermann’s survey comparing Lie theory vs. Cartan theory [55] for a sober and authoritative view of this situation.
But Hermann did not have the benefit of Gromov’s work on sufficiently underdetermined linear PDEs. Moreover, we
learned many decades ago that Lie theory is not the royal road to solving PDEs for nonlinear filtering, contrary to
Kalman’s vision (e.g., see [4], [65] and [66]). However, exploiting gauge transformations has been successful for
solving stochastic PDEs for nonlinear filtering [56]. Designing the gauge transformation that is actually useful to
solve the PDE was the key trick in [56].
Seventh, one can derive an ODE to solve our PDE (4) using another homotopy, as explained in [8]. This homotopy
deforms both the PDE and the solution of the PDE from some initial known value (e.g., Gaussian flow for linear
problems) to the desired target PDE and its corresponding solution. Such a homotopy results in a solution of the PDE
that is given as the solution of an ODE, by construction, and the solution is unique subject to the usual Lipschitz
conditions.
Eighth, we could assume that the last three terms of the PDE (4) do not sum to zero, but rather sum to something else
nice (e.g., a quadratic in x) of our choice. Moreover, we could force this sum to be something nice by suitable choice
of the problem itself. It is somewhat miraculous that the simple choice of zero for this sum worked out so well in [20]
and [25].
Ninth, there is no need to rule out integration of our PDE completely. In particular, we are perfectly happy with a
solution to our PDE that requires integration with respect to a scalar, as in solving an ODE for the separation of
variables solution of a PDE [56]. Moreover, we would also be happy with a sequence of decoupled scalar integrals,
as in Gibbs flow [32]. That is, integrate with respect to one component of x at a time, which is roughly analogous to
sampling one component of x at a time in Gibbs sampling for MCMC methods. Gibbs flow can be represented as a
triangular Jacobian of the flow (in a special coordinate system), and it corresponds to a solvable Lie algebra of the
flow in coordinate free language. Subject to certain regularity conditions, roughly speaking we know that solvable
estimation Lie algebras (with fixed finite dimension) induce exact fixed finite dimensional nonlinear filters (e.g., see
[63] and [64]).
Tenth, there is no practical requirement to have a fixed finite dimensional filter, but rather a finite dimension that
grows (at a reasonable rate) with time is also manageable computationally. This corresponds to a new separation of
variables solution of our PDE at each discrete step in time for each discrete time measurement. This more general
notion of a solution to our PDE was suggested in [37] and [38]. Intuitively, Gromov’s condition of a “sufficiently
underdetermined system of PDEs” would be useful for finding these more general solutions to our PDE, but we
expect that a modification of Gromov’s hypothesis is needed.
Eleventh, it is conceivable that some other law of motion for particle flow, other than the Ito stochastic differential
equation (2) might result in lower computational complexity or better accuracy for the filter. For example, one can
imagine using a fractional stochastic differential equation, as in [69], or perhaps a stochastic difference equation, or a
stochastic integral-differential equation, or a spectral random law of motion, or a law that is defined implicitly, or a
random finite state law, or chaos, or combinations of the above, etc.
Twelfth, we should certainly explore the benefits of using quasi-Monte Carlo (QMC) samples rather than boring old
Monte Carlo samples for our stochastic particle flow filter. Using QMC for particle filters is an obvious idea, and
many researchers (including us [72]) have tried to use QMC with particle filters over the last decade but without any
success (e.g., see table 1 in [72]); however, the brilliant new paper by Gerber and Chopin makes QMC work well for
particle filters by using a clever new ingredient: the Hilbert space filling curve [70]. Moreover, the impressive
performance of QMC particle filters reported in [70] for low dimensions (d = 1 and d = 2) is not significant for d = 6
problems, and it vanishes completely by the time we get to d = 10 problems. Nobody really understands why QMC
stops working for high dimensional particle filter problems in practice [71]. Maybe this is caused by the dimensional
dependence in the theoretical bounds on accuracy; for example, the Koksma-Hlawka inequality gives an upper bound
on QMC error of the following form:

Proc. of SPIE Vol. 10200 102000I-9

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 c log d N
e ≤ (28)
N
This theoretical bound for QMC contrasts with the usual Monte Carlo errors, which vary as

c(d )
e ≤ (29)
N
The so-called “constants” in these two bounds hide a multitude of details about the problem, including: smoothness of
the functions, coupling between components of the state vector, errors in the proposal densities, dimensional
dependence of sampling errors, etc.; for details of such fraud see [73] and [38]. Moreover, it is well known that QMC
works much better for low dimensional problems in practice, which has motivated the search for methods of
dimensional reduction or decoupling; for example, it is standard to apply a PCA to a data set before attempting QMC
sampling. Also, more recent research has improved the Koksma-Hlawka inequality by explicitly modeling the
smoothness of the functions as well as the coupling between components of the state vector. Finally, it is well known
that the Koksma-Hlawka inequality is far from tight, whereas the modern QMC bounds are much tighter, because
they actually include the key ingredients which the K-H bounds sweeps under the rug. For example, a more recent
bound on L² approximation error has the form:

e ≤ cN − k [log N ]( d −1)( k +1/ 2) (30)

in which k measures smoothness of the functions that are to be approximated; in particular, the L² norm of the kth
mixed partial derivatives of the functions are bounded by unity. A beautifully lucid and refreshingly honest recent
survey of such theoretical bounds is given by Novak [74], and an authoritative tutorial on QMC methods is in [75].
This bound (30) suggests that QMC methods should start to degrade as the dimension grows, and also it predicts that
QMC beats boring old Monte Carlo methods for sufficiently smooth functions. There are QMC algorithms that
actually follow these theoretical bounds in numerical experiments, at least up to reasonable values of k, d and N.
But we conjecture that the root cause of this problem (i.e., QMC particle filters in [70] losing steam at high
dimensions) is our old nemesis particle degeneracy, which ruins all particle filters in high dimensions, and which
would be mitigated by stochastic particle flow. Furthermore, QMC can be used in two distinct ways for stochastic
particle flow filters: (1) prediction of the conditional probability density with time, and (2) Bayes’ rule using
stochastic particle flow in λ. Hence, intense research on the combination of QMC (using the Hilbert space filling
curve as in [70]) with stochastic particle flow should be pursued immediately (if not sooner). Finally, one can
combine the dozen ideas listed above with the dozens of other ideas described in [37] and [38].
Concerning idea number six, there is reason to believe that the geometrical method (i.e., coordinate-free) is exactly
the wrong approach to solving PDEs. This notion is contrary to the usual mathematical propaganda for the last 80
years. After reading a typical paper on abstract Lie algebras written in the last eighty years, one would never guess
that Lie algebras had any practical application. But strange as it may seem, Sophus Lie invented Lie algebras over
one hundred forty years ago in order to understand a very practical problem: when PDEs can be solved exactly. This
is at the heart of the nonlinear filtering problem, because if we can solve the Fokker-Planck exactly in terms of
ODEs, then engineers can implement the nonlinear filter in real time. Unfortunately, despite a century of
mathematical research, attempts to understand separation of variables for PDEs in terms of Lie algebras have been
largely unsuccessful. One reason for this failure is that separation of variables requires finding a preferred coordinate
system, whereas Lie algebras are coordinate free. Lie theory has been somewhat successful for ODEs but not really
useful for PDEs. But of course, we do not need any help solving ODEs in real time today, given our fast cheap
computers. More details and references on separation of variables and Lie theory are in [4]. The same basic story
applies to both Cartan and Kähler. Cartan’s original motivation was to solve PDEs exactly using exterior
differential systems; this is crystal clear in Cartan’s early papers [61] and [62], but it is also evident in his later book
[60] to some extent. Similarly, Kähler’s goal was to solve PDEs exactly using his new improved version of Cartan’s
theory; this is obvious from Kähler’s beautifully clear book [57]. The introduction to Kähler’s book explains that it
is “a commentary on the profound but inaccessible treatises of the great French mathematicians.” Unfortunately,
modern mathematicians have essentially given up trying to solve PDEs exactly, and instead they publish research on
the 99th refinement of abstract theorems on existence, uniqueness and regularity of solutions to PDEs. The reason

Proc. of SPIE Vol. 10200 102000I-10

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 for this is obvious: it is much easier to prove such theorems than actually solve PDEs. The task of actually solving
PDEs has been delegated to physicists. This explains the outstandingly good advice that I got from Dan Zwillinger
about solving my PDEs: “think like a physicist.” There are, however, a few conspicuous exceptions; for example,
Professor Robert Bryant is a pure mathematician at Duke University, but he is also an expert at actually solving
PDEs using exterior differential systems (EDS) combined with many years of accumulated ad hoc tricks and
creativity and hard work (e.g., see [58] and [59] and also Chapter 6 in [54] which is based on Bryant’s lectures).
Another notable exception is Professor Werner Seiler, who has written a beautiful book [53] on actually solving
PDEs using EDS and other mathematical tools, but it turns out that his doctoral advisor was a physicist, whose
doctoral advisor was also a physicist who was trained by Dirac and Heisenberg. There is a message here.

ACKNOWLEDGEMENTS

The simple exact beautiful formula for Q (equation 9) was derived jointly by Dr. Rob Fuentes and Dr. Eric Blake,
and their work is gratefully acknowledged. We thank Professor Simon Maskell for telling us that even for examples
in which the state vector estimate of our particle flows (with Q = 0) is excellent, the particle distribution generally
underestimates the variance of the correct conditional density; this was a strong motivation for our research reported
here. Theoretical and numerical confirmation of Simon’s observation, along with a resolution of this issue, are
given in [68]. We thank Flávio de Melo for his nice clear proof of Gromov’s theorem, as well as excellent questions
and suggestions about how to actually apply Gromov’s method [76].

REFERENCES

[1] Uwe Hanebeck, Kai Briechle & Andreas Rauh, “Progressive Bayes: a new framework for nonlinear state estimation,” Proceedings of SPIE
Conference, Orlando Florida, April 2003.
[2] Simon Godsill & Tim Clapp, “Improvement strategies for Monte Carlo particle filters,” pages 139-158 in “Sequential Monte Carlo methods
in practice,” edited by Arnaud Doucet, Nando de Freitas & Neil Gordon, Springer-Verlag, 2001.
[3] Christian Musso, Nadia Oudjane & Francois Le Gland, “Improving regularised particle filters,” pages 247-271 in “Sequential Monte Carlo
methods in practice,” edited by Arnaud Doucet, Nando de Freitas & Neil Gordon, Springer-Verlag, 2001.
[4] Fred Daum, “Nonlinear filters: beyond the Kalman filter,” IEEE AES Systems magazine, special tutorial issue, pages 57 to 69, August
2005.
[5] Fred Daum, Jim Huang and Arjang Noushin, “Exact particle flow for nonlinear filters,” proceedings of SPIE Conference, Orlando Florida,
April 2010.
[6] Fred Daum & Jim Huang, “Numerical experiments for nonlinear filters with exact particle flow induced by log-homotopy,” proceedings of
SPIE conference, Orlando Florida, April 2010.
[7] Fred Daum & Jim Huang, “Particle degeneracy: root cause and solution,” Proceedings of SPIE Conference, Orlando Florida, April 2011.
[8] Fred Daum & Jim Huang, “Exact particle flow for nonlinear filters: seventeen dubious solutions to a first order linear underdetermined
PDE,” Proceedings of IEEE Asilomar conference, November 2010.
[9] Fred Daum, Jim Huang & Arjang Noushin, “Coulomb’s law particle flow for nonlinear filters,” Proceedings of SPIE Conference, San
Diego CA, August 2011.
[10] Arnaud Doucet & A. M. Johansen, “A tutorial on particle filtering and smoothing: fifteen years later,” Chapter 24 in “The Oxford handbook
of nonlinear filtering,” edited by Dan Crisan & Boris Rozovskiǐ, Oxford University Press, 2011.
[11] “The Oxford handbook of nonlinear filtering,” edited by Dan Crisan & Boris Rozovskiĭ, Oxford University Press, 2011.
[12] Fred Daum, Jim Huang & Arjang Noushin, “Numerical experiments for Coulomb’s law particle flow for nonlinear filters,” Proceedings of
SPIE Conference, San Diego CA, August 2011.
[13] Fred Daum & Jim Huang, “Hollywood log-homotopy: movies of particle flow for nonlinear filters,” Proceedings of SPIE conference,
Orlando Florida, April 2011.
[14] Fred Daum & Jim Huang, “Nonlinear filters with log-homotopy,” Proceedings of SPIE Conference on signal & data processing, San Diego
California, September 2007.
[15] Branko Ristic, Sanjeev Arulampalam & Neil Gordon, “Beyond the Kalman filter,” Artech House publishing, 2004.
[16] Fred Daum & Jim Huang, “Generalized particle flow for nonlinear filters,” Proceedings of SPIE Conference, Orlando Florida, April 2010.
[17] Fred Daum, “A new nonlinear filtering formula for discrete time measurements,” IEEE CDC Proceedings, Fort Lauderdale Florida,
December 1985.
[18] Fred Daum, “A new nonlinear filtering formula for non-Gaussian discrete time measurements,” IEEE CDC Proceedings, Athens Greece,
December 1986.
[19] Fred Daum and Jim Huang, “Zero curvature particle flow for nonlinear filters,” Proceedings of SPIE Conference, Baltimore, April 2013.

Proc. of SPIE Vol. 10200 102000I-11

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 [20] Fred Daum and Jim Huang, “Particle flow with non-zero diffusion for nonlinear filters,” Proceedings of SPIE Conference, Baltimore, April
2013.
[21] Fred Daum & Jim Huang, “Particle flow and Monge-Kantorovich transport,” Proceedings of FUSION Conference, Singapore, July 2012.
[22] Fred Daum & Jim Huang, “Small curvature particle flow for nonlinear filters,” Proceedings of SPIE Conference, Baltimore, May 2012.
[23] Fred Daum and Jim Huang, “Fourier transform particle flow for nonlinear filters,” Proceedings of SPIE Conference, Baltimore, April 2013.
[24] Fred Daum and Jim Huang, “Particle flow inspired by Knothe-Rosenblatt transport for nonlinear filters,” Proceedings of SPIE Conference,
Baltimore, April 2013.
[25] Fred Daum and Jim Huang, “Particle flow with non-zero diffusion for nonlinear filters, Bayesian decisions and transport,” Proceedings of
SPIE Conference on Signal & Data Processing, San Diego, August 2013.
[26] Fred Daum and Jim Huang, “renormalization group flow in k-space for nonlinear filters, Bayesian decisions and transport,” Proceedings of
International FUSION Conference, Washington DC, July 2015.
[27] Fred Daum and Jim Huang, “how to avoid normalization of particle flow for nonlinear filters, Bayesian decisions and transport,”
Proceedings of Conference on signal processing, Baltimore, May 2014.
[28] Fred Daum and Jim Huang, “seven dubious methods to mitigate stiffness in particle flow with non-zero diffusion for nonlinear filters,
Bayesian decisions and transport,” Proceedings of Conference on signal processing, Baltimore, May 2014.
[29] Fred Daum and Jim Huang, “proof that particle flow corresponds to Bayes’ rule: necessary and sufficient conditions for existence,”
Proceedings of SPIE conference on signal processing, edited by Ivan Kadar, Baltimore, April 2015.
[30] Sebastian Reich and Colin Cotter, “probabilistic forecasting and Bayesian data assimilation,” Cambridge University Press, 2015.
[31] Neil Gordon, David Salmond & A. F. M. Smith, “novel approach to nonlinear/non-Gaussian Bayesian state estimation,” IEE Proceedings,
volume 140, number 2, pages 107-113, 1993.
[32] Jeremy Heng, Arnaud Doucet and Yvo Pokern, “Gibbs flow for approximate transport with applications to Bayesian computation,”
arXiv:1509.08787v1, 30 September 2015.
[33] Flávio Eler de Melo, Simon Maskell, Matteo Fasiolo and Fred Daum, “stochastic particle flow for nonlinear high-dimensional filtering
problems,” arXiv:1511.01448, 4 November 2015.
[34] Gareth Roberts and Jeff Rosenthal, “examples of adaptive MCMC,” available on-line January 2008.
[35] Muhammad Altamash Khan and Martin Ulmke, “improvements in the implementation of log-homotopy based particle flow filters,”
Proceedings of 18th international conference on information fusion, Washington DC, July 2015.
[36] Jasper Snoek, Hugo Larochelle and Ryan Adams, “practical Bayesian optimization of machine learning algorithms,” NIPS 2012.
[37] Fred Daum and Jim Huang, “a baker’s dozen of new particle flows for nonlinear filters, Bayesian decisions and transport,” Proceedings of
SPIE Conference on signal processing, Baltimore, April 2015.
[38] Fred Daum and Jim Huang, “a plethora of open problems in particle flow research for nonlinear filters, Bayesian decisions and learning and
transport,” Proceedings of SPIE Conference on signal processing, edited by Ivan Kadar, Baltimore, April 2016.
[39] Fred Daum and Jim Huang, “seven dubious methods to compute optimal Q for Bayesian stochastic particle flow,” Proceedings of
International FUSION Conference, Heidelberg, July 2016.
[40] Mikhael Gromov, “partial differential relations,” Springer-Verlag, 1986.
[41] Mikhael Gromov, “geometric, algebraic and analytic descendants of Nash isometric embedding theorems,” freely available on-line, 9
October 2015.
[42] Deane Yang, “Gromov’s approach to solving underdetermined systems of PDEs,” freely available on-line, 6 March 2016.
[43] Rudolf E. Kalman, “new methods in Wiener filtering theory,” RIAS technical report 61-1 (February 1961); also published in J. Bogdanoff
and F. Kozin (editors) Proceedings of first symposium on engineering applications of random function theory and probability, John Wiley
& Sons Inc., pages 270-388, 1963.
[44] Rob Fuentes and Eric Blake, unpublished notes, July 2016.
[45] Cédric Villani, “topics in optimal transportation,” AMS Press, 2003.
[46] Cédric Villani, “optimal transport: old and new,” Springer-Verlag, 2009.
[47] Yann Ollivier, Hervé Pajot and Cédric Villani (editors), “optimal transportation: theory and applications,” Cambridge University Press,
2014.
[48] Yongxin Chen, Tryphon Georgiou and Michele Pavon, “optimal mass transport over bridges,” pages 77 to 84 in Proceedings of second
international conference on geometric science of information, Palaiseau France, edited by Frank Nielsen and Frédéric Barbaresco, 28 to 30
October 2015.
[49] François Trèves, “basic linear partial differential equations,” Dover Books Inc., 2006.
[50] David Spring, “convex integration theory,” Birkhäuser, 1998.
[51] Cornelius Lanczos, “linear differential operators,” (originally published 1961) Dover Books, 1997.
[52] M. Zuhair Nashed (editor), “generalized inverses and applications,” Academic Press, 1976.
[53] Werner Seiler, “Involution,” Springer-Verlag, 2010.
[54] Thomas Ivey and Joseph M. Landsberg, “Cartan for beginners: differential geometry via moving franes and exterior differential systems,”
AMS Press, second edition, 2017.
[55] Robert Hermann, “Elie Cartan’s geometric theory of partial differential equations,” pages 265 to 317, Advances in mathematics, Elsevier,
1965.
[56] Fred Daum, “solution of the Zakai equation by separation of variables,” IEEE Transactions on automatic control, October 1987.

Proc. of SPIE Vol. 10200 102000I-12

Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/ on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

 [57] Erich Kähler, “introduction to the theory of systems of differential equations,” (translated into English by D. H. Delphenich & freely
available on-line), originally published in German by B. G. Teubner Verlag, Leipzig und Berlin, 1934.
[58] Robert Bryant, S. S. Chern, Robert Gardner, H. L. Goldschmidt and Phil Griffiths, “exterior differential systems,” Springer-Verlag, 1991.
[59] Robert Bryant, “nine lectures on exterior differential systems,” freely available on-line, March 2013.
[60] Elie Cartan, “exterior differential systems and its applications [sic],” translated into English by Mehdi Nadjafikhah, freely available on-line;
originally published in French by Hermann, Paris, 1945.
[61] Elie Cartan, “on the integration of systems of total differential equations,” (translated into English by D. H. Delphenich & freely available
on-line), originally published in French as “L’intégration des systèmes d’équations aux différentielles totales,” pages 241-311, Ann. sci. de l’É.
N. S. (3) 18 1901.
[62] Elie Cartan, “on certain differential expressions and the Pfaff problem,” (translated into English by D. H. Delphenich & freely available on-
line), originally published in French as “Sur certaines expressions différentielles et le problème de Pfaff,” pages 239-332, Ann. sci. de l’E. N. S.
(3) 16, 1899.
[63] Steve Marcus, “algebraic and geometric methods in nonlinear filtering,” SIAM Journal of control and optimization, 1984.
[64] Michiel Hazewinkel, “Lie algebraic method in filtering and identification,” in the book “stochastic processes in physics and engineering,”
edited by Sergio Albeverio, Reidel Publishing, 1988.
[65] Michel Cohen de Lara, “finite-dimensional filters. part I: the Wei-Norman technique,” SIAM Journal of control and optimization, 1997.
[66] Mireille Chaleyat-Maurel and D. Michel, “Des résultats de non existence de filtre de dimension finie,” Stochastics, 1984.
[67] Raman Mehra, “a comparison of several nonlinear filters for reentry vehicle tracking,” IEEE Transactions on Automatic Control, August
1971.
[68] Fred Daum, Jim Huang and Arjang Noushin, “numerical experiments for Gromov’s stochastic particle flow,” Proceedings of SPIE
Conference on signal processing,” edited by Ivan Kadar, Anaheim California, 11 April 2017.
[69] Sabir Umarov, Fred Daum and Kenric Nelson, “Fractional generalizations of filtering problems and their associated fractional Zakai
equations,” Fractional calculus and applied analysis, pages 745 to 764, September 2014.
[70] Mathieu Gerber and Nicolas Chopin, “Sequential quasi Monte Carlo,” Journal of Royal Statistical Society, series B, pages 509 to 579, with
rejoinders and rebuttal, 2015. This brilliant paper (alas, without the rejoinders and rebuttal) is freely available on the internet at
arXiv:1402.4039v5, but you can find this more quickly by simply doing a Google search for the exact title. The full published paper with
rejoinders and rebuttal is also freely available somewhere on the internet with a little more effort.
[71] Nick Chopin and Fred Daum, private communication at MCQMC Conference, Stanford University, Palo Alto California, August 2016.
[72] Fred Daum and Jim Huang, “quasi-Monte Carlo hybrid particle filters,” Proceedings of SPIE Conference on signal processing, edited by
Oliver Drummond, Orlando Florida, April 2004.
[73] Fred Daum and Jim Huang, “the curse of dimensionality and particle filters,” Proceedings of IEEE AES Conference at Big Ski Montana,
2003.
[74] Erich Novak, “some results on the complexity of numerical integration,” pages 161 to 183 in “Monte Carlo and quasi-Monte Carlo
methods,” edited by Ron Cools and Dirk Nuyens, Springer-Verlag, 2016.
[75] Joseph Dick, Frances Kuo and Ian Sloan, “high-dimensional integration: the quasi-Monte Carlo way,” pages 133 to 288 in Acta Numerica,
Cambridge University Press, 2013.
[76] Flávio de Melo, “answers to Fred’s questions,” unpublished notes, 8 March 2017.
[77] Fred Daum, “new exact nonlinear filters,” chapter 8 in “Bayesian analysis of times series and dynamical systems,” edited by James C. Spall,
New York: Marcel Dekker Inc., 1988.
[78] Fred Daum, Jim Huang and Arjang Noushin, “Gromov’s method for Bayesian stochastic particle flow: a simple exact formula for Q,”
Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent systems (MFI), Baden-Baden Germany, 19
to 21 September 2016.

Proc. of SPIE Vol. 10200 102000I-13

DownloadedViewFrom:
publicationhttp://spiedigitallibrary.org/pdfaccess.ashx?url=/data/conferences/spiep/92568/
stats on 05/25/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx