Description
Get more precise simulation estimates from the same number of runs by exploiting known structure of the model.
Crude Monte Carlo can need huge numbers of runs, especially to compare similar designs or estimate rare events. Variance reduction uses what is known about the model, often from a pilot run. Common random numbers drive two design variants with the same random inputs, so their difference has far less noise. Antithetic pairs use each uniform draw U and 1 - U when output responds monotonically. Control variates subtract a correlated quantity whose mean is known, cutting variance by the squared correlation. Conditional Monte Carlo replaces the output by a computable conditional expectation. Stratification samples input strata separately, ideally in proportion to probability times spread. Importance sampling draws from a distribution that favours the important region and reweights, giving enormous gains for rare events but inflating variance if badly tuned. Typical gains are up to about tenfold, with importance sampling far beyond that.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Data
- Analytical / expert
- Effort
- Days
Steps
- Run a pilot simulation to learn output variances, correlations with candidate controls, and which input regions matter.
- When comparing two designs, drive both with common random numbers so their outputs move together.
- If output is monotone in the inputs, pair each draw U with 1 - U (antithetic variates) and average each pair.
- If a correlated quantity with known mean exists, subtract a times (control minus its mean), with a = covariance over variance estimated from the sample.
- Stratify key inputs with known stratum probabilities, allocating runs in proportion to probability (never worse than crude) or to probability times stratum standard deviation (optimal).
- For rare events, sample from a proposal distribution concentrated on the important region and weight by the likelihood ratio; tune the proposal by variance minimisation or cross-entropy.
- Report efficiency against crude Monte Carlo as the variance ratio adjusted for computing time.
Sources
- Simulation and the Monte Carlo Method (Rubinstein & Kroese, 2008)
Also known as
- Variance reduction techniques
- Monte Carlo variance reduction
- Common random numbers
- Antithetic variates
- Control variates
- Conditional Monte Carlo
- Rao-Blackwellisation
- Importance sampling
- Stratified Monte Carlo
- Variance reduction for Monte Carlo estimation (common and antithetic random numbers, control variables, conditional Monte Carlo, stratified sampling, importance sampling)
Categories
- Analytics & Behaviour
- Statistics & Modelling