Description
An exact confidence set for a ratio of two means, found by inverting a t test on Y minus a times X; unbounded when the denominator is uncertain.
Relative lift, cost per conversion and other ratio metrics need intervals for a ratio of means. The delta method gives a symmetric approximation that can be badly wrong when the denominator is noisy. Fieller's approach notes that, for a candidate ratio a, the quantity Y minus aX has mean zero, so a t statistic built from the sample means, their variances and covariance has a known distribution. The confidence set is every a for which that t lies inside the critical value, found by solving a quadratic. It is an ordinary interval only when the denominator mean is clearly non-zero; otherwise it is two half-lines or the whole line, an honest signal that the data cannot pin the ratio down. Geisser recounts how Fisher and Neyman read the same construction differently.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Compute the sample means of numerator and denominator, the estimated variances of each mean and their covariance (zero for independent groups).
- Choose the t critical value for the desired confidence level and degrees of freedom.
- Form the quadratic (xbar^2 - t^2 s_x^2) a^2 - 2(xbar ybar - t^2 s_xy) a + (ybar^2 - t^2 s_y^2) <= 0 and solve for its roots.
- If the leading coefficient is positive and the roots are real, report the interval between them.
- If the leading coefficient is negative, report the two half-lines outside the roots; if there are no real roots, report that the set is the whole line.
- Compare with the delta-method interval and explain any difference to stakeholders.
Sources
- Modes of Parametric Statistical Inference (Geisser, 2006)
Also known as
- Fieller's theorem
- Fieller interval
- Fieller-Creasy interval
- Exact confidence interval for a ratio
Categories
- Analytics & Behaviour
- Statistics & Modelling