Reference
EnergySamplers.AbstractSamplerEnergySamplers.AbstractSamplerEnergySamplers.AbstractSamplingRuleEnergySamplers.ConditionalSamplerEnergySamplers.ConditionalSamplerEnergySamplers.ImproperSGLDEnergySamplers.JointSamplerEnergySamplers.JointSamplerEnergySamplers.SGLDEnergySamplers.UnconditionalSamplerEnergySamplers.UnconditionalSamplerEnergySamplers.PMCEnergySamplers._energyEnergySamplers._energyEnergySamplers.energyEnergySamplers.energyEnergySamplers.energyEnergySamplers.energy_differentialEnergySamplers.energy_penaltyEnergySamplers.get_logitsEnergySamplers.mcmc_samplesEnergySamplers.mcmc_samplesEnergySamplers.mcmc_samples
EnergySamplers.AbstractSampler — Type
Base type for samplers.
EnergySamplers.AbstractSampler — Method
(sampler::AbstractSampler)(
model,
rule::AbstractSamplingRule;
niter::Int = 100,
clip_grads::Union{Nothing,AbstractFloat} = 1e-2,
n_samples::Union{Nothing,Int} = nothing,
kwargs...,
)Sampling method for AbstractSampler. This method generates samples from the model's learned distribution.
Arguments
sampler::AbstractSampler: The sampler to use.model: The model to sample from.rule::AbstractSamplingRule: The sampling rule to use.niter::Int=100: The number of iterations to perform.clip_grads::Union{Nothing,AbstractFloat}=nothing: The value to clip the gradients. This is useful to prevent exploding gradients when training joint energy models. Ifnothing, no clipping is performed.n_samples::Union{Nothing,Int}=nothing: The number of samples to generate.kwargs...: Additional keyword arguments.
Returns
input_samples: The samples generated by the sampler.
EnergySamplers.AbstractSamplingRule — Type
Base type for sampling rules.
EnergySamplers.ConditionalSampler — Type
ConditionalSampler <: AbstractSamplerGenerates conditional samples: $x \sim p(x|y).$
EnergySamplers.ConditionalSampler — Method
ConditionalSampler(
𝒟x::Distribution, 𝒟y::Distribution;
input_size::Dims, batch_size::Int,
max_len::Int=10000, prob_buffer::AbstractFloat=0.95
)Outer constructor for ConditionalSampler.
EnergySamplers.JointSampler — Type
JointSampler <: AbstractSamplerGenerates unconditional samples by drawing directly from joint distribution: $x \sim p(x, y).$
EnergySamplers.JointSampler — Method
JointSampler(
𝒟x::Distribution, 𝒟y::Distribution, input_size::Dims, batch_size::Int;
max_len::Int=10000, prob_buffer::AbstractFloat=0.95
)Outer constructor for JointSampler.
EnergySamplers.SGLD — Type
SGLD(a::Real=1.0, b::Real=1.0, γ::Real=0.5)Stochastic Gradient Langevin Dynamics (SGLD) optimizer.
Examples
opt = SGLD()
opt = SGLD(2.0, 100.0, 0.9)EnergySamplers.UnconditionalSampler — Type
UnonditionalSampler <: AbstractSamplerGenerates unconditional samples: $x \sim p(x).$
EnergySamplers.UnconditionalSampler — Method
UnconditionalSampler(
𝒟x::Distribution,
𝒟y::Union{Distribution,Nothing};
input_size::Dims,
batch_size::Int = 1,
max_len::Int = 10000,
prob_buffer::AbstractFloat = 0.95,
)Outer constructor for UnonditionalSampler.
EnergySamplers.PMC — Method
PMC(
sampler::AbstractSampler,
model,
rule::AbstractSamplingRule;
ntransitions::Int = 100,
niter::Int = 100,
kwargs...,
)Runs a Persistent Markov Chain (PMC) using the sampler and model. Persistent Markov Chains are used, for example, for Persistent Contrastive Convergence (Tieleman (2008)), a variant of the Contrastive Divergence (CD) algorithm. The main difference is that PCD uses a persistent chain to estimate the negative phase of the gradient. This is done by keeping the state of the Markov chain between iterations.
In our context, the sampler is the persistent chain and the model is a supervised model. The sampler generates samples from the model's learned distribution.
Note
This function does not perform any training. It only generates samples from the model. In other words, there is no Contrastive Divergence. For training Joint Energy Models, see JointEnergyModels.jl.
Arguments
sampler::AbstractSampler: The sampler to use.model: The model to sample from.rule::AbstractSamplingRule: The sampling rule to use.ntransitions::Int=100: The number of transitions to perform.niter::Int=100: The number of iterations to perform.kwargs...: Additional keyword arguments.
Returns
sampler.buffer: The buffer containing the samples generated by the sampler.
EnergySamplers._energy — Method
_energy(f, x, y::Int; agg=mean)Computes the energy for conditional samples $x \sim p_{\theta}(x|y)$: $E(x)=- f_{\theta}(x)[y]$. Here f is the model, x is the input and y is the index of the target label. DOC_Grathwohl
EnergySamplers._energy — Method
_energy(f, x; agg=mean)Computes the energy for unconditional samples $x \sim p_{\theta}(x)$: $E(x)=-\text{LogSumExp}_y f_{\theta}(x)[y]$. DOC_Grathwohl
EnergySamplers.energy — Method
energy(sampler::ConditionalSampler, model, x, y)Energy function for ConditionalSampler.
EnergySamplers.energy — Method
energy(sampler::JointSampler, model, x, y)Energy function for JointSampler.
EnergySamplers.energy — Method
energy(sampler::UnconditionalSampler, model, x, y)Energy function for UnconditionalSampler.
EnergySamplers.energy_differential — Method
energy_differential(f, xgen, xsampled, y::Int; agg=mean)Computes the energy differential between a conditional sample $x_{\text{gen}} \sim p_{\theta}(x|y)$ and an observed sample $x_{\text{sample}} \sim p(x|y)$ as $E(x_{\text{sample}}|y) - E(x_{\text{gen}}|y)$ with $E(x|y) = -f_{\theta}(x)[y]$. Here f is the model, xgen are the generated samples, xsampled are the observed training samples and y is the index of the target label. DOC_Grathwohl
EnergySamplers.energy_penalty — Method
energy_penalty(f, xgen, xsampled, y::Int; agg=mean)Computes the a Ridge penalty for the overall energies of the conditional samples $x_{\text{gen}} \sim p_{\theta}(x|y)$ and an observed sample $x_{\text{sample}} \sim p(x|y)$. Here f is the model, xgen are the generated samples, xsampled are the observed training samples and y is the index of the target label. DOC_Grathwohl
EnergySamplers.get_logits — Method
get_logits(f::Flux.Chain, x)Retrieves the logits (linear predictions) of a Chain for the input x.
EnergySamplers.mcmc_samples — Method
mcmc_samples(
sampler::ConditionalSampler,
model,
rule::Optimisers.AbstractRule,
input_samples::AbstractArray;
niter::Int,
y::Union{Nothing,Int} = nothing,
)Sampling method for ConditionalSampler.
EnergySamplers.mcmc_samples — Method
mcmc_samples(
sampler::JointSampler,
model,
rule::Optimisers.AbstractRule,
input_samples::AbstractArray;
niter::Int,
y::Union{Nothing,Int} = nothing,
)Sampling method for JointSampler.
EnergySamplers.mcmc_samples — Method
mcmc_samples(
sampler::UnconditionalSampler,
model,
rule::Optimisers.AbstractRule,
input_samples::AbstractArray;
niter::Int,
y::Union{Nothing,Int} = nothing,
)Sampling method for UnconditionalSampler.