Fieller's Confidence Interval for a Ratio of Means

Description

An exact confidence set for a ratio of two means, found by inverting a t test on Y minus a times X; unbounded when the denominator is uncertain.

Relative lift, cost per conversion and other ratio metrics need intervals for a ratio of means. The delta method gives a symmetric approximation that can be badly wrong when the denominator is noisy. Fieller's approach notes that, for a candidate ratio a, the quantity Y minus aX has mean zero, so a t statistic built from the sample means, their variances and covariance has a known distribution. The confidence set is every a for which that t lies inside the critical value, found by solving a quadratic. It is an ordinary interval only when the denominator mean is clearly non-zero; otherwise it is two half-lines or the whole line, an honest signal that the data cannot pin the ratio down. Geisser recounts how Fisher and Neyman read the same construction differently.

Attributes

Loss type
Siloed
Effort
Hours

Steps

  1. Compute the sample means of numerator and denominator, the estimated variances of each mean and their covariance (zero for independent groups).
  2. Choose the t critical value for the desired confidence level and degrees of freedom.
  3. Form the quadratic (xbar^2 - t^2 s_x^2) a^2 - 2(xbar ybar - t^2 s_xy) a + (ybar^2 - t^2 s_y^2) <= 0 and solve for its roots.
  4. If the leading coefficient is positive and the roots are real, report the interval between them.
  5. If the leading coefficient is negative, report the two half-lines outside the roots; if there are no real roots, report that the set is the whole line.
  6. Compare with the delta-method interval and explain any difference to stakeholders.

Sources

  • Modes of Parametric Statistical Inference (Geisser, 2006)

Also known as

  • Fieller's theorem
  • Fieller interval
  • Fieller-Creasy interval
  • Exact confidence interval for a ratio

Categories

  • Analytics & Behaviour
  • Statistics & Modelling