A Grand (or FirstRate) Idea
Bill Farrell, Ph.D.
Senior Analyst
Sutter Health
Sutter Health is a family of notforprofit
hospitals and physician organizations that share
resources and expertise to advance health care
quality. Serving more than 100 communities in
Northern California, Sutter Health is a regional
leader in cardiac care, cancer treatment, orthopedics,
obstetrics, and newborn intensive care, and
is a pioneer in advanced patient safety technology.
Two common measures of quality in health care
are "medication errors per 10,000 prescriptions"
and "falls per thousand patient days."
If you're following St. Elsewhere's "New
Moms" initiative, you know they're reporting
low APGAR scores per thousand, as in "St.
Elsewhere had 9.4 low APGAR scores per thousand
deliveries in the fourth quarter." Finally,
those of you in a managed care environment will
be familiar with statements like "Hospital
utilization for commercial members at St. Eligius
Medical Group is running at 115 bed days per
thousand."
What is it with this "per thousand"
thing? It's actually a little more complicated
than you might expect, since there are two distinct
scenarios where the terminology arises. The
first is what I'll call rare events, and the
second is utilization. Let's start with rare
events. If St. Elsewhere has 8 low APGAR scores
in 853 deliveries, we could calculate the percentage
as 0.9%, but that's a little hard to interpret.
Instead we might do the following calculation:
(8
/ 853) x 1,000 = 9.4
and report that St. Elsewhere had 9.4 low APGAR
scores per thousand deliveries. Similarly, if
the hospital had 3 medication errors in 23,000
prescriptions, we might be better understood
if we reported
(3
/ 23,000) x 10,000 = 1.3
1.3 med errors per 10,000 prescriptions, as
opposed to .01% medication errors.
The 9.4 and 1.3 are known technically as rates,
one of three things that can result from dividing
two numbers. So 2 / 50 = .04, a proportion;
2 / 50 x 100 = 4, a percentage; and 2 / 50 x
1,000 = 40 per thousand, a rate.
You'll note that in the case of rates, the
numbers themselves are fictitious. In the first
example (9.4 low APGARs per thousand) the hospital
did not have 9.4 low APGARs and it did not have
1,000 deliveries. We simply calculate and report
these rates to make small numbers easier to
grasp.
And in the second example you can see that
we don't always use "thousand" as
the denominator in this kind of reporting. The
rule of thumb is that we choose a denominator
that allows us to state the numerator as a number
greater than or equal to 1. For medication errors,
the most commonly used denominator is 10,000.
Other examples (outside of health care) might
include "4.3 accidents per 100,000 miles
driven," and "7.6 satisfactory airline
meals per million passenger miles flown."
We'll turn now to the second case where "per
thousand" comes into play: managed care
utilization. To illustrate the point, let's
ask Dr. d'Arcangelis how he's doing at keeping
his pediatric patients out of the ER. He says,
"I'm doing great. Only four of my pediatric
patients went to the ER." This is not very
useful, since we don't know
a. 
how many pediatric patients
he has, 
b. 
what time period he's talking
about, or 
b. 
the total number of ER visits
for those four children. 
If we know that Dr. d'Arcangelis has 500 pediatric
patients and that he's talking about 3Q04, we
can say that 4 / 500 or .8% of his pediatric
patients visited the ER in the third quarter.
Given that .8% is a little difficult to interpret,
we might (as we did above) note that Dr. d'Arcangelis'
pediatric patients visited the ER at a rate
of 8 per thousand in the third quarter (4 /
500 x 1,000 = 8).
Suppose we now learn that those four pediatric
patients visited the ER a total of 10 times
during the third quarter. The 10 visits are
what we commonly refer to as "utilization,"
and while we COULD calculate a percentage as
above (10 / 500 = 2%), this percentage is meaningless,
since numerator and denominator are not comparable.
Typically we use member months as the denominator
in calculating utilization. Since individual
members can move in and out of a physician's
panel (or medical group) almost at will, member
months is seen as the fairest count of "opportunities"
for utilization to occur. So if someone is enrolled
with a medical group for three months, that
counts as three member months. If three different
people are enrolled in a group for one month,
that's also three member months.
Because utilization can be a difficult concept
to grasp, we'll switch to a new physician, Dr.
Gabriel, who has 1,000 patients in her panel
for each of the three months of 3Q04. We see
10 ER visits in July, 10 visits in August, and
10 in September. It seems obvious in this case
that Dr. Gabriel's utilization is going to work
out to 30 visits per thousand members for the
quarter, but getting there will be a little
tricky.
First, we divide the 10 visits in July by the
1,000 member months in July, getting .01 visits
per 1 member month. If we multiply both sides
of this by 1,000 to scale it up, we get 10 visits
per 1,000 member months. Realizing that this
is a onemonth period, a more general statement
of the result is "10 visits per thousand
member months per month." Since "month"
is now in both denominators, it cancels out,
leaving us with "10 visits per thousand
members." So with 10 visits per thousand
members in each of three months, we end up with
30 visits per thousand members for the quarter.
The actual calculation would look like this:
30
total visits / 3,000 total member months x 3
months x 1,000 = 30
Turning back to Dr. d'Arcangelis, we recall
that he had 500 pediatric patients with a total
of 10 ER visits in the quarter. His quarterly
utilization would be calculated as:
10
total visits / 1,500 total member months x 3
months x 1,000 = 20
As a reminder, when we say "20 ER visits
per thousand members" we're actually saying
"20 ER visits per thousand EQUIVALENT members,"
since we'll almost always see some enrollment
changes during a quarter.
We've calculated utilization for a quarter,
but in managed care circles this number is almost
always annualized (multiplied by 4). So in this
case an analyst would look at a quarter's worth
of Dr. d'Arcangelis' data, run the numbers,
and state that "Dr. d'Arcangelis' pediatric
ER utilization is running at 80 per thousand."
This shorthand expresses the following concept:
IF Dr. d'Arcangelis had 1,000 pediatric patients,
and IF they visited the ER for a year at the
same frequency that was observed for a quarter,
we could expect to see 80 visits.
Let's summarize: Dr. d'Arcangelis has 500 pediatric
patients. Four of them (.8%) went to the ER
in the first quarter, for a rate of 8 per thousand.
With a total of 10 visits, Dr. d'Arcangelis'
utilization is reported as 80 ER visits per
thousand equivalent children per year.
A final word on using statistical process control
to track these kinds of measures: rare events
are usually best suited to the U chart, while
utilization can only be tracked using the I
chart.
Technical Note: Discerning readers will have
noticed that there are two classes of "rare
events": one where the numerator is a subset
of the denominator and percentaging is appropriate
(low APGAR scores / all APGAR scores) and one
where numerator and denominator are not comparable
and rates are appropriate (falls / patient days).
Mathematically, P charts and U charts will give
identical results with these two classes of
data. It probably makes more sense to stick
with U charts in both cases, however, since
(a) the U chart is strictly appropriate in the
latter case (falls), and (b) the U chart gets
you thinking "per thousand" rather
than "percent" in the former case
(APGAR).
