Engine 19.0.0rc597 · Reader 9.0.0rc656 · Development snapshot
The Theil index measures inequality in a distribution of non-negative values, such as incomes or event counts. When the total is positive, Theil T is zero if everyone has the same value; a larger number means greater concentration. It compares the observed shares with equal shares using information entropy.
This guide explains Theil T, its formula, interpretation and a worked calculation. Ghost Observatory applies it to the number of recorded process events attributed to each person in an analysis window. The general mathematics below is separate from the product's population and publication rules.
Use it to ask how much of the recorded activity sits with a few people, then inspect the counts behind that pattern. The calculation gives every process event the same weight. It cannot establish how difficult the work was, how long it took or how well it was done.
Theil T formula and range
Let N be the number of people, xi person i’s event count, C the total count and μ the mean count per person. The logarithm is natural, written ln.
C = ∑i=1N xi · μ = C / N
T = (1 / N) ∑i=1N (xi / μ) ln(xi / μ)
The raw index ranges from 0 to ln(N). It reaches the upper end when one person holds every event. Multiplying every person’s count by the same positive factor leaves the index unchanged.
The Engine stores this raw index. The Reader’s “Unevenness (% of maximum)” display divides it by ln(N) and multiplies by 100. That presentation puts equal counts at 0% and all events held by one person at 100%.
A worked example: 0, 1 and 3 events
Three people have four events between them, so the mean is 4/3. Applying the formula gives the following signed contributions.
| Person | Process events | Theil contribution |
|---|---|---|
| A | 0 | 0 |
| B | 1 | −0.071921 |
| C | 3 | +0.608198 |
The terms add to 0.536277. The maximum for three people is ln(3), approximately 1.098612, so the normalized value is 48.814% of maximum. The Reader rounds percentages to two significant digits and displays 49%.
A contribution is a term in that sum. A positive count below the mean contributes a negative value; a count above the mean contributes a positive one. Zero counts and counts exactly at the mean contribute zero. These terms are signed index units, so they should not be presented as percentage shares of the total.
How to interpret a Theil index
Compare distributions of the same quantity over a clearly defined population. A Theil T of 0.5 does not mean that half the people hold all the work, that half the work is unfairly allocated, or that productivity is 50%. It is an index value. The normalized percentage on this page is a separate calculation relative to the maximum for the selected population.
There is no universal cutoff for a good or bad workload distribution. Differences in assignment, working hours, activity types and recording coverage can all change event counts. Inspect those differences before interpreting concentration as a performance problem. Adding people with known zero counts also changes the population and the index.
Zero-valued observations are valid for Theil T when the total is positive: their limiting contribution is zero. Negative values are outside this formula's domain. If every value is zero, the mean is zero and the ratios are undefined; do not report that as a calculated index of zero.
Theil T, Theil L and the Gini coefficient
“Theil index” can refer to two related generalized entropy measures. Theil T is GE(1), the formula used on this page. Theil L, also called the mean logarithmic deviation, is GE(0): L = (1 / N) ∑ ln(μ / xi). For this form of L, all observations must be positive; a zero observation gives an unbounded logarithmic term. T and L need not return the same value.
The Gini coefficient is another inequality measure, commonly expressed from zero to one. Raw Theil T has a different scale, with maximum ln(N) for a fixed equally weighted population. Do not compare a raw Theil value directly with a Gini value as if they were percentages of the same quantity. Observatory's event-count measure is Theil T.
The World Bank's inequality reference distinguishes the generalized entropy measures. The US Census Bureau introduces Theil's entropy interpretation.
Within-group and between-group decomposition
Theil T can be split into inequality within groups and inequality between their means. Each observation must belong to exactly one group. If group g has Ng members, total Cg and within-group index Tg, let sg = Cg / C and qg = Ng / N.
T = ∑g sgTg + ∑g sg ln(sg / qg)
For values 1, 1, 3 and 3, grouping the two ones together and the two threes together gives zero within-group inequality. All of T ≈ 0.130812 lies between the groups. Grouping one 1 and one 3 in each group gives equal group means: the same total inequality is now entirely within groups. The partition changes its explanation, not the total index.
PySAL's guide to the Theil index provides an independent reference. This is background mathematics; the Observatory snapshot documented here does not publish this group decomposition.
Why the chance level matters
Even equal random assignment produces unequal counts when there are few events. The Engine therefore calculates an expected index for the same N people and C events, assuming that each event goes independently to an equally likely person.
X ∼ Binomial(C, 1 / N)
E[T] = ∑x=1C P(X = x) (x / μ) ln(x / μ)
With three people and four events, the expected raw index is 0.315864, or 28.751% of maximum; the Reader displays 29%. Comparing the normalized values, 48.814% observed and 28.751% expected, gives the unevenness useful context. The expectation is not a significance threshold: exceeding it does not establish that the allocation differs statistically from chance.
What the Engine publishes and the Reader shows
| Field | Meaning |
|---|---|
windows.theil_output | Raw Theil T of event counts. |
windows.theil_output_people | Number of people in that calculation, including known zero counts. |
windows.theil_output_expected | Expected raw index under equal random allocation. |
person_days.theil_output_term | Signed contribution on a person’s whole-window row, where work_date is NULL. |
In People → Theil index, the Reader draws the observed index and chance level as percentages of ln(N), then lists event counts and stored contributions. It does not recompute the index from the displayed person rows. Overview uses the same percentage presentation.
Who is included, and when the result is missing
The population includes people identified by process events, rejected process-output evidence or admitted work records. Someone with work records and no process events can therefore have a known zero count. Screen observations alone do not place someone in this population. Events with no attributable person do not become an extra person.
The result requires at least two people, a positive total and a known event count for every included person. If one count is unknown, the index, chance level and all contributions are withheld. The workload calculation also withholds results when the process records mix incompatible counting grains.
Keep the analysis window and population consistent when comparing results. The current output has one window index and individual terms; it does not publish a separate daily or team Theil index.
Related measures and sources
For recorded time, use recorded work time. For the distribution of individual durations, see median and percentiles and kernel density estimation. The measure reference groups the remaining calculations.
Method reference: Foster, Seth, Lokshin and Sajaia, A Unified Approach to Measuring Poverty and Inequality, World Bank, 2013, equation 2.28. That reference defines Theil’s first measure; the event population and random-allocation comparison above describe this Engine’s implementation.