Planar Flow

The planar flow introduced in [REZENDE2015]. This flow was originally designed for variational inference and sampling. Therefore it doesn’t automatically fit our use-case of density estimation. Since we especially need the inverse \(f^{-1}(x)\) of the flow to be easily computable, we invert it’s direction, defining it as a mapping from the transformed distribution \(p_1(x)\) to the base distribution \(p_0(x)\). Hence the flow is called InvertedPlanarFlow in our implementation and the forward method is not implemented.

\[ \begin{align}\begin{aligned}\mathbf{u},\mathbf{w} \in \mathbb{R}^d, b \in \mathbb{R}\\f^{-1}(x) = \mathbf{x} + \mathbf{u}\cdot\tanh(\mathbf{w}^T \mathbf{x} + b)\end{aligned}\end{align} \]

To make sure \(f(x)\) exists, \(\mathbf{w}^T\mathbf{u} \geq-1\) needs to hold. Our implementation automatically constrains \(\mathbf{u}\) before assignment using

\[ \begin{align}\begin{aligned}\mathbf{û} = \mathbf{u} + (m (\mathbf{w}^T\mathbf{u}) - (\mathbf{w}^T \mathbf{u)})\dfrac{\mathbf{w}}{\|\mathbf{w}\|^2}\\m(\mathbf{x}) = -1 + softplus(\mathbf{x})\end{aligned}\end{align} \]

Dimension of parameter space: \(d + d+ 1\)

Determinant of the Jacobian of \(f^{-1}(x)\):

\[\det(\mathbf{J}^{-1}) = |\mathbf{I} + \mathbf{u}^T \cdot \tanh'(\mathbf{w}^T \mathbf{x} + b) \cdot \mathbf{w}|\]

Hence the Inverse Log Det Jacobian for this flow is:

\[\log(\det(\mathbf{J}^{-1})) = \log(|\mathbf{I} + \mathbf{u}^T \cdot \tanh'(\mathbf{w}^T \mathbf{x} + b) \cdot \mathbf{w}|)\]

class cde.density_estimator.normalizing_flows.InvertedPlanarFlow(params, n_dims, name='Inverted_Planar_Flow')

Implements a bijector x = y + u * tanh(w_t * y + b)

Parameters

• params – Tensor shape (?, 2*n_dims+1). This will be split into the parameters u (?, n_dims), w (?, n_dims), b (?, 1). Furthermore u will be constrained to assure the invertability of the flow

• n_dims – The dimension of the distribution that will be transformed

• name – The name to give this particular flow

dtype

dtype of `Tensor`s transformable by this distribution.

event_ndims

Returns then number of event dimensions this bijector operates on.

forward(x)

We don’t require sampling and it would be slow, therefore it is not implemented

Raises

NotImplementedError –

forward_event_shape(input_shape)

Shape of a single sample from a single batch as a TensorShape.

Same meaning as forward_event_shape_tensor. May be only partially defined.

Parameters

input_shape – TensorShape indicating event-portion shape passed into forward function.

Returns

TensorShape indicating event-portion shape

after applying forward. Possibly unknown.

Return type

forward_event_shape_tensor

forward_event_shape_tensor(input_shape, name='forward_event_shape_tensor')

Shape of a single sample from a single batch as an int32 1D Tensor.

Parameters

• input_shape – Tensor, int32 vector indicating event-portion shape passed into forward function.

• name – name to give to the op

Returns

Tensor, int32 vector indicating

event-portion shape after applying forward.

Return type

forward_event_shape_tensor

forward_log_det_jacobian(x, name='forward_log_det_jacobian')

Returns both the forward_log_det_jacobian.

Parameters

• x – Tensor. The input to the “forward” Jacobian evaluation.

• name – The name to give this op.

Returns

Tensor, if this bijector is injective.

If not injective this is not implemented.

Raises

• TypeError – if self.dtype is specified and y.dtype is not self.dtype.

• NotImplementedError – if neither _forward_log_det_jacobian nor {_inverse, _inverse_log_det_jacobian} are implemented, or this is a non-injective bijector.

static get_param_size(n_dims)

Parameters

n_dims – The dimension of the distribution to be transformed by the flow

Returns

(int) The dimension of the parameter space for this flow, n_dims + n_dims + 1

graph_parents

Returns this Bijector’s graph_parents as a Python list.

inverse(y, name='inverse')

Returns the inverse Bijector evaluation, i.e., X = g^{-1}(Y).

Parameters

• y – Tensor. The input to the “inverse” evaluation.

• name – The name to give this op.

Returns

Tensor, if this bijector is injective.

If not injective, returns the k-tuple containing the unique k points (x1, …, xk) such that g(xi) = y.

Raises

• TypeError – if self.dtype is specified and y.dtype is not self.dtype.

• NotImplementedError – if _inverse is not implemented.

inverse_event_shape(output_shape)

Shape of a single sample from a single batch as a TensorShape.

Same meaning as inverse_event_shape_tensor. May be only partially defined.

Parameters

output_shape – TensorShape indicating event-portion shape passed into inverse function.

Returns

TensorShape indicating event-portion shape

after applying inverse. Possibly unknown.

Return type

inverse_event_shape_tensor

inverse_event_shape_tensor(output_shape, name='inverse_event_shape_tensor')

Shape of a single sample from a single batch as an int32 1D Tensor.

Parameters

• output_shape – Tensor, int32 vector indicating event-portion shape passed into inverse function.

• name – name to give to the op

Returns

Tensor, int32 vector indicating

event-portion shape after applying inverse.

Return type

inverse_event_shape_tensor

inverse_log_det_jacobian(y, name='inverse_log_det_jacobian')

Returns the (log o det o Jacobian o inverse)(y).

Mathematically, returns: log(det(dX/dY))(Y). (Recall that: X=g^{-1}(Y).)

Note that forward_log_det_jacobian is the negative of this function, evaluated at g^{-1}(y).

Parameters

• y – Tensor. The input to the “inverse” Jacobian evaluation.

• name – The name to give this op.

Returns

Tensor, if this bijector is injective.

If not injective, returns the tuple of local log det Jacobians, log(det(Dg_i^{-1}(y))), where g_i is the restriction of g to the ith partition Di.

Raises

• TypeError – if self.dtype is specified and y.dtype is not self.dtype.

• NotImplementedError – if _inverse_log_det_jacobian is not implemented.

is_constant_jacobian

Returns true iff the Jacobian is not a function of x.

Note: Jacobian is either constant for both forward and inverse or neither.

Returns

Python bool.

Return type

is_constant_jacobian

name

Returns the string name of this Bijector.

validate_args

Returns True if Tensor arguments will be validated.

REZENDE2015

Rezende, Mohamed (2015). Variational Inference with Normalizing Flows (http://arxiv.org/abs/1505.05770)