Engine 19.0.0rc597 · Reader 9.0.0rc656 · Development snapshot
A Wilson score interval describes sampling uncertainty around an observed proportion. Ghost Observatory uses a two-sided 95% Wilson interval for the share of checked objects or events that break a configured rule. Its alert decision uses the lower end of that interval.
This helps answer why two groups with the same deviation percentage can support different conclusions. Two deviations among ten checked objects and twenty among a hundred both give 20%, but the second result supplies more information about the underlying rate.
The formula and its inputs
Let a be the number of deviations and n the number checked. The observed share is p̂ = a / n. For a two-sided 95% interval, the standard-normal quantile z is approximately 1.959964.
d = 1 + z2 / n
A = [p̂ + z2 / (2n)] / d
B = z √[p̂(1 − p̂) / n + z2 / (4n2)] / d
Lower bound = max(0, A − B)
Upper bound = min(1, A + B)
All checked units have equal weight within their rule. There is no duration weighting. An object-level rule counts work objects; the minimum-step-time rule counts eligible events. Keep that unit consistent when interpreting the denominator.
A worked example: the same 20% share
Applying the formula to each count gives the following intervals. The example uses the documented default alert level of 10%.
| Deviations / checked | Observed share | 95% Wilson interval | Lower bound above 10%? |
|---|---|---|---|
| 2 / 10 | 20% | 5.668%–50.984% | No |
| 20 / 100 | 20% | 13.337%–28.883% | Yes |
The ten-object result leaves considerable uncertainty. With a hundred objects, the lower bound is already above the configured level. This is why the same observed percentage produces different alert flags.
Alert = lower bound > configured alert share
The alert level is a product setting, defaulting to 0.1. It is not a universal statistical standard. The interval itself uses the contract’s 95% confidence level.
Small samples, zero hits and missing values
A Wilson interval needs a positive checked count. Unlike a duration percentile, it has no additional five- or twenty-observation publication minimum. Small samples are represented by wide intervals.
Zero deviations among ten checked objects gives an interval of approximately 0% to 27.753%. Seeing no deviations in those records therefore leaves uncertainty about a larger population. If nothing was checked, the bounds and alert flag are NULL: there is no observed proportion to evaluate.
The nominal 95% level describes the interval method’s coverage over repeated samples under its binomial assumptions. Dependence between observations or a changing rate can make those assumptions inappropriate. The calculation does not establish that the records are a representative sample of future work.
Why not use the ordinary Wald interval?
The elementary Wald interval uses the observed proportion plus or minus a normal quantile times its estimated standard error. With no observed hits, that estimated standard error is zero and the interval collapses to zero. That is poor evidence about a wider population when the sample is small.
The Wilson score interval avoids that collapse: this page's zero-hit example retains a positive upper bound. It is still an approximate interval under binomial assumptions, not a way to repair biased or dependent observations. NIST compares methods for proportion confidence intervals.
Where the results are stored
Each rules row supplies hit_count and checked_count, with the Engine’s bounds in rate_low and rate_high. The configured threshold is alert_share; the resulting flag is above_alert_share. The rule belongs to one analysis scope through scope_id.
The Engine also publishes Wilson bounds in rule_breakdown.rate_low and rate_high for rows whose dimension is day. Attribute-segment rows use relative risk and its Katz interval instead.
For an object-level daily result, both deviating and nondeviating checked objects count on their day of closure in the run’s time zone. Still-open objects do not enter those day rows. This is different from counting the date on which a deviation happened. The minimum-step-time rule instead groups its eligible events by their recorded work date.
How the Reader uses the interval
Processes → Conformance displays the stored intervals beside the rule shares and uses the stored alert flag. Processes → Rule trend plots the daily bounds and counts. The Reader calculates the displayed share by dividing stored hits by stored checked units; it reads the interval bounds from the artifact.
A whole-period rule and its daily rows can have different eligible populations, particularly while objects remain open. Compare their denominators before treating a difference between the lines as a change in the process.
Related measures and sources
Use relative risk to compare one attribute group with the rest. For limits around individual durations, see statistical process control. Return to the measure reference for the other populations.
Method reference: NIST/SEMATECH, Confidence intervals for proportions, which gives the Wilson score formula and distinguishes it from other proportion intervals.