Child pages
  • Hyperparameter Updating (8.0)

Contents

Context

Inference | Hyperparameter Updating

The Bayesian networks that are used in BayesiaLab are basically non parametric models. The easiest way to design parametric models consists in using equations for describing the probability distributions of the nodes that are part of the system you are modeling, i.e. what is usually called the parameters, and represent the parameters that are used in the equations, i.e. the hyperparameters, with additional nodes in the network.

Example

Let's suppose we are interested in modeling the distribution of measures of a given component from blood samples. The measured values of the our parameter Blood Sample range from 0 to 2.5. Our experts speculate that the parameter follows a normal distribution. In a scenario where the experts know with certainty the values of the hyperparameters (the mean and standard deviation), let say 1 and 0.5 respectively, we would just define the distribution of the variable Blood Sample with the following probabilistic equation: P(?Blood Sample?) = Normal(?Blood Sample?, 1, 0.5)

However, let's suppose our experts just know the ranges of the hyperparameters Mu and Sigma, i.e. they do not know anything about the exact values, nor about their distributions. In order to represent this uncertainty, we create a node per hyperparameter, and represent their total lack of knowledge about the distribution with uniform distributions.

Then, instead of using constants in the equation that describes the distribution of the variable Blood Sample, we connect the hyperparameter nodes to Blood Sample and use the following equation: P(?Blood Sample?) = Normal(?Blood Sample?, ?Mu?, ?Sigma?)

Switching to Validation Mode shows the resulting distributions:

The networks below show various normal distributions after setting evidence on the hyperparameters:

With such parametric model, observing the parameter Blood Sample allows updating the distributions of the hyperparameters via probabilistic inference. However, the node Blood Sample can just be used for capturing one piece of evidence, i.e one sample of the distribution that is described by the model.

The traditional approach for what is usually called Bayesian Updating is to create a parameter node per observation, while hyperparameters are shared by all parameter nodes. 

Example

Let's continue with our Blood Sample example, and suppose we received the five following results: 0.7, 0.6, 0.2, 2.1, and 1.5.

It is therefore necessary to create five Blood Sample nodes for being able to compute the posterior probability distributions of the hyperparameters given these five observations.


The need to unroll the network, i.e. create as many parameter nodes as observations, is obviously not convenient, and makes exact inference quickly intractable when the number of observations increases.

The BayesiaLab's Hyperparameter Updating is a compact version where there is no need to unroll the network. It just requires:

  • The association of the hyperparameter nodes with the predefined class Hyperparameter for identifying what are the nodes to update, and 
  • The representation of the conditional dependencies between the hyperparameters by adding an arc between these nodes. Indeed, even though the hyperparameters can be independent without any observation on the parameter, they become dependent after the first observation. 


The new conditional probability distribution does not represent any dependency, i.e. we just need to select all the cells of the table and click on Normalize.


Example

The network below represents the compact version for our Blood Sample example. We have added a link between Mu and Sigma to represent the conditional dependency given Blood Sample, and associated these two nodes with the class Hyperparameter.




Renamed Feature: Hyperparameter Updating

Prior to version 8.0, this feature was available under Interactive Updating. It has been renamed Hyperparameter Updating.

New Feature: Manual Evidence-Setting

As of version 8.0, evidence scenario and/or internal data are not mandatory for using this feature. It is now possible to manually set evidence for updating the hyperparameters.

We have also added  for directly editing the predefined class Hyperparameter, without having to use the Class Editor before Hyperparameter Updating.