Skip to content

LCDM Sampling

cmb_lensing.sample_lcdm.sample_joint

sample_joint(data_set, param_init, proposal_sigmas, param_ranges, should_sample, noise_level, advanced_logging, fixed_fields=False, phi_init='MAP', hmc_num_steps=DEFAULT_HMC_NUM_STEPS, hmc_step_size=DEFAULT_HMC_STEP_SIZE, iters_per_chain=10000, num_burn_in_fix_theta=100, num_burn_in_always_accept=0, resume_chain_dict=None, seed=None, map_idx=1, sub_chain_idx=1, lmax=DEFAULT_MAX_ELL, metropolis_num_steps=1, hpc_path=None)

Gibbs-sample the joint posterior P(f, phi, theta | d) of the field, the lensing potential and the five LCDM parameters.

Each iteration draws f from its conditional, takes one HMC step on the mixed phi, then one random-walk Metropolis move per sampled parameter on the mixed logpdf, and recomputes the covariance and mixing matrices at the accepted parameters. The parameters are named as in constants.PARAM_ORDER: "theta_MC_100", "logA", "ns", "ombh2", "omch2". The spectra come from the 5D CAMB grid spline (camb_splines/camb_grid_spline.npz, downloaded on first use).

Parameters:

Name Type Description Default
data_set

The data, a DataSetT, DataSetEB or DataSetTEB (e.g. from load_sim). Its type sets the polarization mode.

required
param_init

{name: value} starting point, all five parameters.

required
proposal_sigmas

{name: width} of the Gaussian Metropolis proposal per parameter. The parameters are updated in this dict's order. Tune toward ~44 % acceptance.

required
param_ranges

{name: (low, high)}; proposals outside the range are rejected.

required
should_sample

{name: bool}; parameters set to False stay at param_init.

required
noise_level

White noise level of the data in uK-arcmin.

required
advanced_logging

{"phi_acceptance", "lcdm_acceptance", "plot_log_pdf", "plot_lcdm_sigmas": bool}. The first two print running acceptance rates, the last two overwrite progress plots in cmb_lensing/sample_lcdm_output/ (which must exist) every iteration.

required
fixed_fields

Debugging option: hold f and phi at the data set's true fields and sample theta only.

False
phi_init

Starting phi: "MAP" (the map_joint estimate, default), "RNG" (a draw from the phi step's inverse mass matrix) or anything else for zero.

'MAP'
hmc_num_steps

Leapfrog steps per phi HMC step.

DEFAULT_HMC_NUM_STEPS
hmc_step_size

Leapfrog step size. Tune toward ~65 % phi acceptance.

DEFAULT_HMC_STEP_SIZE
iters_per_chain

Number of Gibbs iterations.

10000
num_burn_in_fix_theta

Iterations before the parameters start to move.

100
num_burn_in_always_accept

Iterations for which every phi HMC step is accepted.

0
resume_chain_dict

Restart a saved chain: a dict with one entry per parameter name, "phi_map" and "sub_key" (e.g. np.load of the npz run_single_lcdm_chain.py writes). Overrides param_init and phi_init.

None
seed

Seed of the chain's random numbers. None draws one at random.

None
map_idx

Data-map index used in the history file names.

1
sub_chain_idx

Chain index used in the history file names.

1
lmax

Largest multipole of the spectra.

DEFAULT_MAX_ELL
metropolis_num_steps

Metropolis moves per parameter per iteration.

1
hpc_path

If set, every iteration appends each sampled parameter's latest value to {hpc_path}{name}_map_{map_idx}_chain_{sub_chain_idx}_history.txt.

None

Returns:

Type Description
(param_vals, phi, sub_key)

param_vals is a dict of per-parameter lists of the

chain's values (starting with param_init), phi the final lensing potential and

sub_key the final random key, which together with the last parameter values let a

chain be resumed.

Source code in cmb_lensing/sample_lcdm.py
def sample_joint(data_set, param_init, proposal_sigmas, param_ranges, should_sample, noise_level, 
                 advanced_logging, fixed_fields = False, phi_init = "MAP",
                 hmc_num_steps = DEFAULT_HMC_NUM_STEPS, hmc_step_size = DEFAULT_HMC_STEP_SIZE,
                 iters_per_chain = 10_000, num_burn_in_fix_theta = 100, num_burn_in_always_accept = 0, 
                 resume_chain_dict = None, seed = None, map_idx = 1, sub_chain_idx = 1,
                 lmax = DEFAULT_MAX_ELL, metropolis_num_steps = 1, hpc_path = None):
    """Gibbs-sample the joint posterior P(f, phi, theta | d) of the field, the lensing potential
    and the five LCDM parameters.

    Each iteration draws f from its conditional, takes one HMC step on the mixed phi, then one
    random-walk Metropolis move per sampled parameter on the mixed logpdf, and recomputes the
    covariance and mixing matrices at the accepted parameters. The parameters are named as in
    `constants.PARAM_ORDER`: "theta_MC_100", "logA", "ns", "ombh2", "omch2". The spectra come
    from the 5D CAMB grid spline (`camb_splines/camb_grid_spline.npz`, downloaded on first
    use).

    Args:
        data_set: The data, a `DataSetT`, `DataSetEB` or `DataSetTEB` (e.g. from
            `load_sim`). Its type sets the polarization mode.
        param_init: {name: value} starting point, all five parameters.
        proposal_sigmas: {name: width} of the Gaussian Metropolis proposal per parameter.
            The parameters are updated in this dict's order. Tune toward ~44 % acceptance.
        param_ranges: {name: (low, high)}; proposals outside the range are rejected.
        should_sample: {name: bool}; parameters set to False stay at `param_init`.
        noise_level: White noise level of the data in uK-arcmin.
        advanced_logging: {"phi_acceptance", "lcdm_acceptance", "plot_log_pdf",
            "plot_lcdm_sigmas": bool}. The first two print running acceptance rates, the
            last two overwrite progress plots in `cmb_lensing/sample_lcdm_output/` (which
            must exist) every iteration.
        fixed_fields: Debugging option: hold f and phi at the data set's true fields and
            sample theta only.
        phi_init: Starting phi: "MAP" (the `map_joint` estimate, default), "RNG" (a draw
            from the phi step's inverse mass matrix) or anything else for zero.
        hmc_num_steps: Leapfrog steps per phi HMC step.
        hmc_step_size: Leapfrog step size. Tune toward ~65 % phi acceptance.
        iters_per_chain: Number of Gibbs iterations.
        num_burn_in_fix_theta: Iterations before the parameters start to move.
        num_burn_in_always_accept: Iterations for which every phi HMC step is accepted.
        resume_chain_dict: Restart a saved chain: a dict with one entry per parameter name,
            "phi_map" and "sub_key" (e.g. `np.load` of the npz `run_single_lcdm_chain.py`
            writes). Overrides `param_init` and `phi_init`.
        seed: Seed of the chain's random numbers. None draws one at random.
        map_idx: Data-map index used in the history file names.
        sub_chain_idx: Chain index used in the history file names.
        lmax: Largest multipole of the spectra.
        metropolis_num_steps: Metropolis moves per parameter per iteration.
        hpc_path: If set, every iteration appends each sampled parameter's latest value to
            `{hpc_path}{name}_map_{map_idx}_chain_{sub_chain_idx}_history.txt`.

    Returns:
        (param_vals, phi, sub_key): param_vals is a dict of per-parameter lists of the
        chain's values (starting with `param_init`), phi the final lensing potential and
        sub_key the final random key, which together with the last parameter values let a
        chain be resumed.
    """

    #polarization mode follows the dataset
    if isinstance(data_set, DataSetTEB):
        pol = "IP"
    elif isinstance(data_set, DataSetEB):
        pol = "P"
    elif isinstance(data_set, DataSetT):
        pol = "I"
    else:
        raise ValueError(f"sample_joint supports DataSetT (pol = 'I'), DataSetEB "
                         f"(pol = 'P') and DataSetTEB (pol = 'IP') datasets, got "
                         f"{type(data_set).__name__}")
    bb_is_zero = camb_grid_bb_is_zero()

    #if a resume_chain_dict is specified (e.g. np.load of the npz run_single_lcdm_chain.py
    #writes: one entry per parameter name, "phi_map" and "sub_key"), restart from its parameters
    if resume_chain_dict is not None:
        param_init = {}
        for name in PARAM_ORDER:
            param_init[name] = float(resume_chain_dict[name])

    #the sampled parameters, in PARAM_ORDER: the ones whose histories are logged
    theta_log_names = [name for name in PARAM_ORDER if should_sample.get(name, False)]

    #the 5D grid predictors, checked at the starting point before anything heavy runs
    predictors = get_camb_grid_predictors()
    start_tt = predictors.tt(jnp.array([[param_init[k] for k in PARAM_ORDER]]))
    if not bool(jnp.all(jnp.isfinite(start_tt))):
        raise ValueError(
            f"the starting parameters {dict((k, param_init[k]) for k in PARAM_ORDER)} "
            f"fall outside the 5D CAMB grid, so the initial covariances would be NaN. "
            f"Check that theta_MC_100 is reachable at this ombh2/omch2 - see "
            f"theta_grid in {CAMB_GRID_PATH}")

    #Set the initial parameter values to the user specified starting guesses
    param_vals = {}
    for theta, theta_val in param_init.items():
        param_vals[theta] = [theta_val]
    #Also store the current parameter values in a JAX array
    current_params = jnp.array([param_init[k] for k in PARAM_ORDER], dtype = jnp.float64)

    #Add data that will be used throughout the sampling algorithm to an args dictionary
    args = {}
    args = add_metadata_to_args(args, data_set, lmax)
    args = add_starting_matrices_to_args(args, data_set, noise_level,
                                         param_init, current_params,
                                         predictors, pol = pol)

    #change data_set covariance matrices from ground truth to initial
    #starting point in cosmological parameter space
    data_set = set_initial_ds_conditions(data_set, args)
    data_field = data_set.data

    #Use a seed to get reproduceable results if so desired
    #otherwise use machine entropy to generate a random number
    if seed is not None:
        sub_key = jax.random.PRNGKey(seed)
    elif resume_chain_dict is not None:
        sub_key = jnp.asarray(resume_chain_dict["sub_key"])
    else:
        sub_key = jax.random.PRNGKey(np.random.randint(0, 2**31))

    #Setting the starting point in phi-space to be the MAP estimate (the mode) plus a random realization
    #of the Hessian pre-conditioner empirically seems to decrease the burn-in time (DEFAULT).
    #A resumed chain starts from its saved phi whatever phi_init says
    if resume_chain_dict is not None:
        phi = data_set.phi.replace(scalar_matrix = jnp.asarray(resume_chain_dict["phi_map"]))
    elif phi_init == "MAP":
        _, phi_map = map_joint(data_set)
        phi = phi_map
    elif phi_init == "RNG":
        inv_mass_matrix = pinv(pinv(args["phi_covariance"]) + pinv(args["quadratic_estimate"]))
        rng_key, sub_key = jax.random.split(sub_key)
        phi_rng_matrix = field_from_covar_single_key(data_set.data.nside, inv_mass_matrix.scalar_matrix, rng_key)
        phi = data_set.phi.replace(scalar_matrix = jfft.rfft2(phi_rng_matrix))
    else:
        phi = 0*data_set.phi

    #We repeatedly draw a fresh temperature / polarization field, so we need a "zeroes" object
    #of the same class as the temperature / polarization field
    field_zeroes = 0*data_set.unlensed_field

    #the stacked data matrices handed to every jitted theta kernel
    data_stack = field_matrix_stack(data_field, pol)

    #lists used to track sampling acceptance rates
    phi_acceptance = []
    mixed_logpdf_values = []
    lcdm_acceptance = {}
    lcdm_acceptance["theta_MC_100"] = []
    lcdm_acceptance["ombh2"] = []
    lcdm_acceptance["omch2"] = []
    lcdm_acceptance["ns"] = []
    lcdm_acceptance["logA"] = []

    for iter in range(1, iters_per_chain + 1):

        #The "fixed_fields" flag can be used for quick debugging of the theta step 
        #assuming we have perfect knowledge of the ground truth (f, phi) pair
        if not fixed_fields:
            #1. sample the temperature field
            rng_key, sub_key = jax.random.split(sub_key)
            temp_field = gibbs_sample_f(field_zeroes, data_field, phi, args, rng_key)

            #2. mix the fields
            mixed_temp, mixed_phi = mix(temp_field, phi, args["mixing_d"], args["mixing_g"])

            #3. sample the lensing potential phi
            rng_key, sub_key = jax.random.split(sub_key)
            mixed_phi, delta_h, accept = gibbs_sample_phi(mixed_phi, mixed_temp, data_field, rng_key,
                                                          args, iter, num_burn_in_always_accept,
                                                          hmc_num_steps = hmc_num_steps, 
                                                          hmc_step_size = hmc_step_size)
            if advanced_logging["phi_acceptance"]:
                phi_acceptance.append(int(accept))
                print(f"Phi accept rate = {np.sum(np.array(phi_acceptance)) / len(phi_acceptance)}")
                print(f"delta_H = {delta_h}")
        else:
            mixed_temp, mixed_phi = mix(data_set.unlensed_field, data_set.phi, 
                                        args["mixing_d"], args["mixing_g"])

        #4. sample your cosmo parameters
        if iter >= num_burn_in_fix_theta:
            for theta, proposal_sigma in list(proposal_sigmas.items()):
                if should_sample[theta]:
                    rng_key, sub_key = jax.random.split(sub_key)
                    theta_key_idx = PARAM_INDEX[theta]
                    theta_val = gibbs_sample_theta(jnp.array(theta_key_idx), param_ranges[theta], 
                                                   jnp.array(param_vals[theta][-1]),
                                                   lcdm_acceptance, advanced_logging["lcdm_acceptance"],
                                                   field_matrix_stack(mixed_temp, pol),
                                                   mixed_phi.scalar_matrix, data_stack,
                                                   current_params, rng_key, 
                                                   args["nside"], args["pix_width"], data_field.theta_pix,
                                                   args["ell_grid"],
                                                   args["quadratic_estimate"].scalar_matrix,
                                                   op_matrix_stack(args["noise_covariance"], pol),
                                                   op_shared_matrix(args["mask"], pol), 
                                                   op_shared_matrix(args["beam"], pol),
                                                   data_field.fourier_weights,
                                                   proposal_sigma = proposal_sigma,
                                                   metropolis_num_steps = metropolis_num_steps,
                                                   pol = pol, bb_is_zero = bb_is_zero)
                    param_vals[theta].append(theta_val)
                    current_params = current_params.at[theta_key_idx].set(theta_val)

            #5. recompute mixing and covariance matrices using the newly sampled parameter values
            args = update_args_after_sample(current_params, predictors, args, pol = pol)

        #6. unmix the fields using the updated version of the G & D matrices
        if not fixed_fields:
            _, phi = unmix(mixed_temp, mixed_phi, args["mixing_d"], args["mixing_g"])

        #7. plot certain diagnostics if user specified
        if advanced_logging["plot_log_pdf"]:
            mixed_logpdf_value = mixed_logpdf(mixed_temp, mixed_phi, data_field, args["noise_covariance"], 
                                              args["phi_covariance"], args["field_covariance"], 
                                              args["mask"], args["beam"], 
                                              args["mixing_g"], args["mixing_d"])
            mixed_logpdf_values.append(mixed_logpdf_value)
            plt.figure(figsize = (16, 10))
            plt.plot(mixed_logpdf_values, marker = "o")
            plt.title("Mixed LogPDF Over Time")
            plt.xlabel("Iteration")
            plt.ylabel("Mixed LogPDF Value")
            plt.legend()
            path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
                                "cmb_lensing", "sample_lcdm_output", "logpdf_progress.png")
            plt.savefig(path)
            plt.close() 

        if advanced_logging["plot_lcdm_sigmas"]:
            plt.figure(figsize = (16, 10))
            for name in theta_log_names:
                normalized = (np.array(param_vals[name]) - GROUND_TRUTH[name]) / PARAM_SIGMA[name]
                plt.plot(normalized, label = name, marker = "o")
            plt.axhline(0, color = "black", label = "Zero Sigma")
            plt.axhline(1, color = "grey", label = "+/- 1 Sigma")
            plt.axhline(-1, color = "grey")
            plt.title("(mean - sample)/sigma")
            plt.xlabel("Iteration")
            plt.ylabel("Standard Deviations")
            plt.legend()
            path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
                                "cmb_lensing", "sample_lcdm_output", "lcdm_progress.png")
            plt.savefig(path)
            plt.close()

        #Log chains to .txt files if we are running a larger experiment on an HPC
        if hpc_path is not None:
            for theta in theta_log_names:
                theta_path = hpc_path + f"{theta}_map_{map_idx}_chain_{sub_chain_idx}_history.txt"
                with open(theta_path, "a") as file:
                    file.write(str(param_vals[theta][-1]) + "\n")

    return param_vals, phi, sub_key