REGULATORY HEALTH PROTECTION: ANALYTICAL CHALLENGES
By John D. Graham, Ph.D., Administrator, Office of Information and Regulatory Affairs,
Office of Management and Budget, Executive Office of the President
Remarks Prepared for International Society for Pharmacoeconomics and Outcomes Research (ISPOR)
Annual International Meeting, May 21, 2002, Arlington, Virginia
Thank you very much for the opportunity to speak today. As many of you know, I am a strong
advocate of the formal tools of regulatory analysis, such as risk analysis, decision analysis,
cost-effectiveness analysis and cost-benefit analysis. I believe these tools can help us accomplish more
public health protection at less cost than will occur without good analysis. Although these formal tools
have much insight to offer, it is fair to say that their influence to date has been limited.
In order for the potential of these tools to be realized, much work needs to be done. I believe we need
a combination of both intellectual progress in the field and institutional innovation in how analysis gets
done. Please let me explore these themes in concert.
HEALTH RISK ASSESSMENT
First, there is an enormous challenge in gauging the likelihood and severity of an emerging health threat.
Sometimes there is ample basis for public concern about an emerging hazard but there may be a very
limited historical basis for determining the magnitude of the hazard. Examples that come to mind are
bioterrorism, mad cow's disease and antibiotic resistance. In order to perform good risk assessment of
these threats, we need much better information about the most susceptible subgroups and communities
in society because it is exposures to these groups that may determine the overall public health
significance of an emerging hazard.
A closely related issue is identifying the most important sources or causes of health problems. In the
case of particulate air pollution, we often make the simple assumption that all fine particles are equally
toxic, regardless of their precise diameter or chemical composition. Yet fine particles from different
sources -- say, motor vehicles and powerplants -- vary somewhat in their typical size distribution and
chemical content. We often make "the-all-particles-are-equal" assumption because it is analytically
tractable, not because it is biologically plausible. Indeed, it is quite possible - perhaps probable - that
some particles are more toxic than others, information that could play an important role in setting
priorities and doing good benefit-cost analysis.
Which institutions in society should be responsible for this applied risk assessment research? One might
think that university-based scientists could offer analytic solutions to these challenges and they often do.
Yet these emerging health hazards, including the identification of sources and causes, often require
collaboration of scientists from multiple disciplines. They also entail a willingness to develop and validate
mathematical models that provide speculative yet useful forecasts about what is possible or probable.
Many universities do not reward faculty and students who tackle messy problems of this sort, though I
am pleased that many NSF and NIH grants support applied research. I am also aware that mission
agencies such as EPA and DOE are supporting some constructive efforts in this arena. I also believe
that it is critical for the public and private sectors to work together to support first-rate, peer-reviewed
science. Organizations such as the Health Effects Institute, the International Life Sciences Institute,
Research Triangle Institute, the Chemical Industry Institute of Toxicology, resources for the Future, and
the Electric Power Research Institute provide an excellent forum for work to be done in collaboration
with university scientists. We need to work harder to strengthen the scientific quality, credibility and
policy relevance of these applied research organizations.
HEALTH RISK EVALUATION
Second, we need better tools to compare the health benefits of policies to their economic costs. Going
beyond health risk assessment, this field might be called health risk evaluation. By evaluation, I mean the
analytic process of scoring or rating health effects in terms of overall burden on the patient and society
and quantifying the benefits of health policies in monetary or other units that capture the preferences of
The central intellectual challenge is to account for the adverse effects of both morbidity and mortality in
a single numerical index, recognizing that some bouts of sickness are more severe than others and some
deaths may be considered more tragic than others. Although answers to these questions require value
judgements, the social sciences have much to offer in providing possible answers.
In the developing world, the World Health Organization has promoted a metric called the
disability-adjusted life year, or the DALY. Diverse health problems from infectious disease to trauma
are scored in terms of the number of DALYs that a society loses. The scoring of each health condition
is based on three factors: the number of life years lost compared to the Japanese experience, weights
are applied to each healthy life year to reflect productivity at work and at home, and weights for each
unhealthy life year are applied to capture the degree of functional limitation imposed on a person.
Although my description of DALYs has been simplistic, the tool continues to be refined in various ways.
A precursor to the DALY, the quality-adjusted life year or QALY, is now the standard measure of
health policy effectiveness used by academics in North America and Europe. It is similar to the DALY
but differs in three important ways. First, the loss of life expectancy from each condition is derived from
models or data for a relevant target population rather than from the Japanese experience. Second, each
year of healthy life is weighted the same, regardless of when in the lifespan the loss occurs. This
assumption is defended on grounds of fairness and analytic simplicity. Finally, the weights for different
health states are typically derived in surveys of community residents or patients who have experienced
the condition. Early work on QALYs was very simplistic in its use of utility theory, clinical insights, and
of survey methods, but the quality of QALY research is improving.
Both the DALY and QALY methods, due to their very assumptions, place more emphasis on risks
affecting middle-age adults and young people than risks of the chronic diseases of old age. This
emphasis is not necessarily consistent with the current investment patterns in the US health care system,
where billions of dollars are devoted to health care late in the lifespan. And economic critics of DALYs
and QALYs argue that these methods depart from classical assumptions.
For example, a standard economic view is that consumer sovereignty should be respected: if
well-informed senior citizens -- who often have substantial assets and precious few remaining life years
-- have a large demand for modest health gains, so be it. And there is a growing literature in economics
aimed at quantifying the monetary benefits of reducing different diseases and health impairments. Yet
these willingness-to-pay tools face a host of ethical and technical criticisms.
At the most basic ethical level, concerns have been raised about whether a person's income level or
asset position should be permitted to influence how much a governmental study values their health
status. Others have argued that a fair allocation of public resources throughout the lifespan should be
determined in a so-called "veil of ignorance," where people's views are not influenced by their current
age or health status. Yet there are certainly some practical problems with implementing that idea!
At a technical level, substantial progress has been made in estimating willingness to pay for health
protection. Yet questions have also been raised about whether stated willingness to pay by survey
respondents is adequately sensitive to the amount of risk reduction and the context of risk reduction. In
light of these many difficulties, OMB does not currently require agencies to value health gains and losses
in monetary units and some agencies, such as OSHA and NHTSA, have refrained from this practice.
I believe that a major conference on health risk evaluation should be convened to assess the state of the
field and make recommendations on future research and analytic practice.
MODEST STEPS BY OMB
At OMB we see health policy occurring at a variety of federal agencies: FDA, EPA, USDA, CMS,
CDC, to name just a few. These agencies are now using a wide range of analytic practices that are not
always mutually consistent. Rarely are these analytic practices rooted in statute.
OMB has an important role to play in fostering a greater degree of consistency. Yet agencies sometimes
have important reasons for doing things differently and OMB's challenge is to appreciate when these
distinctions have merit. We recognize, for example, that EPA considers policies that promise important
ecological benefits that are not captured in any measure of human health benefit.
In the Bush Administration, we support development of health regulations that are based on sound
science and economics. We are certainly not allergic to regulation; indeed, we view it as a critical tool of
national policy -- as critical as government spending and taxation. My Office in particular strives to
prompt and approve good health rules while improving flawed rules and stopping harmful ones. We see
better formal analysis as a critical step toward more effective, fair and efficient health policies.
In my first year at OMB, we have taken two modest steps to promote more analytic rigor and
consistency in health policy throughout the federal government. You can learn more about these stops
on OMB's web site.
First, OMB recently established analytic guidelines for health risk assessment that cover all the federal
agencies. The guidelines, based on two rounds of public comment and interagency review, are rooted in
principles found in the Safe Drinking Water Act Amendments of 1996. A notable feature of the
guidelines is a requirement that agencies, when faced with scientific uncertainty about important health
issues, prepare central estimates of risk as well as upper and lower bounds on the true yet unknown
risks. Each federal agency engaged in health risk assessment is now preparing specific guidelines that
adopt or adapt the formulation found in the Safe Drinking Water Act. When these guidelines take effect
in October of this year, the public will be provided an opportunity to challenge any health risk
information disseminated by a federal agency that does not adhere to the OMB and agency guidelines.
Agencies will be expected to provide a prompt and objective response to these challenges.
Second, OMB - in collaboration with CEA - has initiated a process to refine the guidelines for
regulatory analysis that agencies must follow when preparing economic analyses to support rules. In this
first phase, we have asked for public comment by late May on specific analytic issues - including health
risk evaluation -- that should be addressed in OMB's re-evaluation of existing guidance. Examples of
issues we intend to address are the methods of valuing human health effects and the proper discount
rate to apply to future benefits and costs. We will prepare a proposed revision to OMB's analytic
guidance that will be released for public comment, interagency review, and external peer review. The
final OMB guidance will then be used by my Office to judge whether the regulatory analyses prepared
by agencies are adequate. Please consult OMB's web site for more details about the comment period
that extends through late May.
Looking forward, in order to enhance the roles of science and economics in health policy, we must do
something that is as obvious as it is critical: improve the technical and ethical foundations of the tools
used to inform policy makers. Organizations like Resources for the Future have played an influential role
in the past and I only see that role increasing in importance in the years ahead.
Thank you for the opportunity to speak today and I look forward to your questions and comments.