Reference

EnergySamplers.AbstractSampler — Method
(sampler::AbstractSampler)(
    model,
    rule::AbstractSamplingRule;
    niter::Int = 100,
    clip_grads::Union{Nothing,AbstractFloat} = 1e-2,
    n_samples::Union{Nothing,Int} = nothing,
    kwargs...,
)

Sampling method for AbstractSampler. This method generates samples from the model's learned distribution.

Arguments

  • sampler::AbstractSampler: The sampler to use.
  • model: The model to sample from.
  • rule::AbstractSamplingRule: The sampling rule to use.
  • niter::Int=100: The number of iterations to perform.
  • clip_grads::Union{Nothing,AbstractFloat}=nothing: The value to clip the gradients. This is useful to prevent exploding gradients when training joint energy models. If nothing, no clipping is performed.
  • n_samples::Union{Nothing,Int}=nothing: The number of samples to generate.
  • kwargs...: Additional keyword arguments.

Returns

  • input_samples: The samples generated by the sampler.
source
EnergySamplers.ConditionalSampler — Method
ConditionalSampler(
    𝒟x::Distribution, 𝒟y::Distribution;
    input_size::Dims, batch_size::Int,
    max_len::Int=10000, prob_buffer::AbstractFloat=0.95
)

Outer constructor for ConditionalSampler.

source
EnergySamplers.JointSampler — Type
JointSampler <: AbstractSampler

Generates unconditional samples by drawing directly from joint distribution: $x \sim p(x, y).$

source
EnergySamplers.JointSampler — Method
JointSampler(
    𝒟x::Distribution, 𝒟y::Distribution, input_size::Dims, batch_size::Int;
    max_len::Int=10000, prob_buffer::AbstractFloat=0.95
)

Outer constructor for JointSampler.

source
EnergySamplers.SGLD — Type
SGLD(a::Real=1.0, b::Real=1.0, γ::Real=0.5)

Stochastic Gradient Langevin Dynamics (SGLD) optimizer.

Examples

opt = SGLD()
opt = SGLD(2.0, 100.0, 0.9)
source
EnergySamplers.UnconditionalSampler — Method
UnconditionalSampler(
    𝒟x::Distribution,
    𝒟y::Union{Distribution,Nothing};
    input_size::Dims,
    batch_size::Int = 1,
    max_len::Int = 10000,
    prob_buffer::AbstractFloat = 0.95,
)

Outer constructor for UnonditionalSampler.

source
EnergySamplers.PMC — Method
PMC(
    sampler::AbstractSampler,
    model,
    rule::AbstractSamplingRule;
    ntransitions::Int = 100,
    niter::Int = 100,
    kwargs...,
)

Runs a Persistent Markov Chain (PMC) using the sampler and model. Persistent Markov Chains are used, for example, for Persistent Contrastive Convergence (Tieleman (2008)), a variant of the Contrastive Divergence (CD) algorithm. The main difference is that PCD uses a persistent chain to estimate the negative phase of the gradient. This is done by keeping the state of the Markov chain between iterations.

In our context, the sampler is the persistent chain and the model is a supervised model. The sampler generates samples from the model's learned distribution.

Note

This function does not perform any training. It only generates samples from the model. In other words, there is no Contrastive Divergence. For training Joint Energy Models, see JointEnergyModels.jl.

Arguments

  • sampler::AbstractSampler: The sampler to use.
  • model: The model to sample from.
  • rule::AbstractSamplingRule: The sampling rule to use.
  • ntransitions::Int=100: The number of transitions to perform.
  • niter::Int=100: The number of iterations to perform.
  • kwargs...: Additional keyword arguments.

Returns

  • sampler.buffer: The buffer containing the samples generated by the sampler.
source
EnergySamplers._energy — Method
_energy(f, x, y::Int; agg=mean)

Computes the energy for conditional samples $x \sim p_{\theta}(x|y)$: $E(x)=- f_{\theta}(x)[y]$. Here f is the model, x is the input and y is the index of the target label. DOC_Grathwohl

source
EnergySamplers._energy — Method
_energy(f, x; agg=mean)

Computes the energy for unconditional samples $x \sim p_{\theta}(x)$: $E(x)=-\text{LogSumExp}_y f_{\theta}(x)[y]$. DOC_Grathwohl

source
EnergySamplers.energy_differential — Method
energy_differential(f, xgen, xsampled, y::Int; agg=mean)

Computes the energy differential between a conditional sample $x_{\text{gen}} \sim p_{\theta}(x|y)$ and an observed sample $x_{\text{sample}} \sim p(x|y)$ as $E(x_{\text{sample}}|y) - E(x_{\text{gen}}|y)$ with $E(x|y) = -f_{\theta}(x)[y]$. Here f is the model, xgen are the generated samples, xsampled are the observed training samples and y is the index of the target label. DOC_Grathwohl

source
EnergySamplers.energy_penalty — Method
energy_penalty(f, xgen, xsampled, y::Int; agg=mean)

Computes the a Ridge penalty for the overall energies of the conditional samples $x_{\text{gen}} \sim p_{\theta}(x|y)$ and an observed sample $x_{\text{sample}} \sim p(x|y)$. Here f is the model, xgen are the generated samples, xsampled are the observed training samples and y is the index of the target label. DOC_Grathwohl

source
EnergySamplers.mcmc_samples — Method
mcmc_samples(
    sampler::ConditionalSampler,
    model,
    rule::Optimisers.AbstractRule,
    input_samples::AbstractArray;
    niter::Int,
    y::Union{Nothing,Int} = nothing,
)

Sampling method for ConditionalSampler.

source
EnergySamplers.mcmc_samples — Method
mcmc_samples(
    sampler::JointSampler,
    model,
    rule::Optimisers.AbstractRule,
    input_samples::AbstractArray;
    niter::Int,
    y::Union{Nothing,Int} = nothing,
)

Sampling method for JointSampler.

source
EnergySamplers.mcmc_samples — Method
mcmc_samples(
    sampler::UnconditionalSampler,
    model,
    rule::Optimisers.AbstractRule,
    input_samples::AbstractArray;
    niter::Int,
    y::Union{Nothing,Int} = nothing,
)

Sampling method for UnconditionalSampler.

source