Normalizing Flow Estimator ================================================== The Normalizing Flow Estimator (NFE) combines a conventional neural network (in our implementation specified as :math:estimator) with a multi-stage Normalizing Flow [REZENDE2015]_ for modeling conditional probability distributions :math:p(y|x). Given a network and a flow, the distribution :math:y can be specified by having the network output the parameters of the flow given an input :math:x [TRIPPE2018]_. If the normalizing flow is expressive enough, arbitrary conditional distributions can be approximated. The flows work by transforming a base distribution (in our case a normal distribution) into successively more complex distributions by applying bijectors. Example of a normal distribution being transformed by two planar flows: .. image:: normalizing_flows/planar_flow.png Using the change of variable formula, the resulting probability distribution :math:p_1 for a single flow :math:f applied to the base distribution :math:p_0 becomes: .. math:: p_0(\mathbf{z_0}) = \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})(\mathbf{z_0}) \mathbf{z_1} = f(\mathbf{z_0}) p_1(\mathbf{z_1}) = p_0(f^{-1}(\mathbf{z_1})) \cdot |\det \dfrac{d f^{-1}(\mathbf{z_1})}{d \mathbf{z_1}}| Using normalizing flows for density estimation requires that the inverse and the Jacobian determinant of the flow can be calculated quickly. Given input :math:x, the neural network outputs the parameters :math:\theta of the flows. The weights and biases :math:w of the neural network are learned by minimizing the negative logarithm of the likelihood (maximum likelihood) over :math:N data points for a normalizing flow consisting of :math:K flows. .. math:: E(w) = - \sum_{n=1}^N \bigg\{\log p_0(\mathbf{z_{0,n}}) + \sum_{k=1}^{K} \log|\det\dfrac{d f_k^{-1}(\mathbf{z_{k,n}}, \theta_k(\mathbf{w}, \mathbf{x_n}))}{d \mathbf{z_{k,n}}}|\bigg\} \mathbf{z_{0,n}} = f_1^{-1}(f_2^{-1}(\dots f_K^{-1}(\mathbf{z_{K,n}}))), \mathbf{z_{K,n}} = \mathbf{y_n} Available flows: .. toctree:: :maxdepth: 2 :glob: ./normalizing_flows/* .. automodule:: cde.density_estimator .. autoclass:: NormalizingFlowEstimator :members: :inherited-members: .. [REZENDE2015] Rezende, Mohamed (2015). Variational Inference with Normalizing Flows (http://arxiv.org/abs/1505.05770) .. [TRIPPE2018] Trippe, Turner (2018). Conditional Density Estimation with Bayesian Normalising Flows (http://arxiv.org/abs/1802.04908)