Pooling and testing preferences across data sources

Description

Tests whether choice data from different sources, groups or methods share the same trade-offs once differences in response noise are allowed for, then pools them if they do.

In logit-type choice models every coefficient is the product of a scale factor (inverse of noise) and the underlying part-worth. Two data sets can therefore show different coefficients purely because one is noisier, and naive comparisons confuse noise with taste. Preference regularity means the coefficient vectors are proportional. The procedure estimates separate models, plots one set of coefficients against the other (points should fall on a line through the origin), then fits a pooled model with common coefficients and a free relative scale, found by grid search or an artificial nested logit. A likelihood-ratio test compares pooled and separate fits; if pooling is rejected, differing parameters are freed and the test repeated. Applied to revealed (market) and stated (experimental) data, this 'data enrichment' combines realistic but collinear market choices with clean but hypothetical survey choices; forecasts use market constants with jointly estimated attribute weights.

Attributes

Loss type
Siloed
Effort
Weeks

Steps

  1. Collect the data sets to compare: market choices and a stated-choice task that mirrors them, or the same choice task in different segments, conditions or formats.
  2. Estimate the same model separately for each, excluding alternative-specific constants from the comparison.
  3. Plot coefficient pairs against each other and, for several groups, run PCA on the coefficient vectors to see whether they are proportional.
  4. Fit a pooled model with common coefficients and a free relative scale per data set, found by grid search or nested logit.
  5. Compute -2[LL(pooled) - sum of LL(separate)] and compare with chi-square on the difference in parameters; if rejected, free the differing parameters and retest.
  6. Forecast with market-based constants and pooled attribute weights, and validate on holdout market data.

Sources

  • Stated Choice Methods (Louviere et al., 2000)
  • Discrete Choice Methods with Simulation (Train, 2009)

Also known as

  • Combining revealed and stated preference choice data
  • Data enrichment
  • RP-SP data fusion
  • Joint RP/SP estimation
  • Testing preference regularity across groups, conditions or methods
  • Scale-adjusted parameter comparison
  • Swait-Louviere test
  • Preference regularity test

Categories

  • Analytics & Behaviour
  • Statistics & Modelling