Lp-Nested Symmetric Distributions
Fabian Sinz, Matthias Bethge; 11(111):3409−3451, 2010.
Abstract
In this paper, we introduce a new family of probability densities called Lp-nested symmetric distributions. The common property, shared by all members of the new class, is the same functional form ρ(x) = ~ρ(f(x)), where f is a nested cascade of Lp-norms ||x||p = (∑ |xi|p)1/p. Lp-nested symmetric distributions thereby are a special case of ν-spherical distributions for which f is only required to be positively homogeneous of degree one. While both, ν-spherical and Lp-nested symmetric distributions, contain many widely used families of probability models such as the Gaussian, spherically and elliptically symmetric distributions, Lp-spherically symmetric distributions, and certain types of independent component analysis (ICA) and independent subspace analysis (ISA) models, ν-spherical distributions are usually computationally intractable. Here we demonstrate that Lp-nested symmetric distributions are still computationally feasible by deriving an analytic expression for its normalization constant, gradients for maximum likelihood estimation, analytic expressions for certain types of marginals, as well as an exact and efficient sampling algorithm. We discuss the tight links of Lp-nested symmetric distributions to well known machine learning methods such as ICA, ISA and mixed norm regularizers, and introduce the nested radial factorization algorithm (NRF), which is a form of non-linear ICA that transforms any linearly mixed, non-factorial Lp-nested symmetric source into statistically independent signals. As a corollary, we also introduce the uniform distribution on the Lp-nested unit sphere.
[abs]
[pdf][bib]| © JMLR 2010. (edit, beta) |