Skip Main Navigation
Office of Management and Budget
President's Budget
Management
Information &
Regulatory Affairs
Legislative Information
Agency Information

REGULATORY HEALTH PROTECTION: ANALYTICAL CHALLENGES

By John D. Graham, Ph.D., Administrator, Office of Information and Regulatory Affairs,
Office of Management and Budget, Executive Office of the President

Remarks Prepared for International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Annual International Meeting, May 21, 2002, Arlington, Virginia

Thank you very much for the opportunity to speak today. As many of you know, I am a strong advocate of the formal tools of regulatory analysis, such as risk analysis, decision analysis, cost-effectiveness analysis and cost-benefit analysis. I believe these tools can help us accomplish more public health protection at less cost than will occur without good analysis. Although these formal tools have much insight to offer, it is fair to say that their influence to date has been limited.

In order for the potential of these tools to be realized, much work needs to be done. I believe we need a combination of both intellectual progress in the field and institutional innovation in how analysis gets done. Please let me explore these themes in concert.

HEALTH RISK ASSESSMENT

First, there is an enormous challenge in gauging the likelihood and severity of an emerging health threat. Sometimes there is ample basis for public concern about an emerging hazard but there may be a very limited historical basis for determining the magnitude of the hazard. Examples that come to mind are bioterrorism, mad cow's disease and antibiotic resistance. In order to perform good risk assessment of these threats, we need much better information about the most susceptible subgroups and communities in society because it is exposures to these groups that may determine the overall public health significance of an emerging hazard.

A closely related issue is identifying the most important sources or causes of health problems. In the case of particulate air pollution, we often make the simple assumption that all fine particles are equally toxic, regardless of their precise diameter or chemical composition. Yet fine particles from different sources -- say, motor vehicles and powerplants -- vary somewhat in their typical size distribution and chemical content. We often make "the-all-particles-are-equal" assumption because it is analytically tractable, not because it is biologically plausible. Indeed, it is quite possible - perhaps probable - that some particles are more toxic than others, information that could play an important role in setting priorities and doing good benefit-cost analysis.

Which institutions in society should be responsible for this applied risk assessment research? One might think that university-based scientists could offer analytic solutions to these challenges and they often do. Yet these emerging health hazards, including the identification of sources and causes, often require collaboration of scientists from multiple disciplines. They also entail a willingness to develop and validate mathematical models that provide speculative yet useful forecasts about what is possible or probable.

Many universities do not reward faculty and students who tackle messy problems of this sort, though I am pleased that many NSF and NIH grants support applied research. I am also aware that mission agencies such as EPA and DOE are supporting some constructive efforts in this arena. I also believe that it is critical for the public and private sectors to work together to support first-rate, peer-reviewed science. Organizations such as the Health Effects Institute, the International Life Sciences Institute, Research Triangle Institute, the Chemical Industry Institute of Toxicology, resources for the Future, and the Electric Power Research Institute provide an excellent forum for work to be done in collaboration with university scientists. We need to work harder to strengthen the scientific quality, credibility and policy relevance of these applied research organizations.

HEALTH RISK EVALUATION

Second, we need better tools to compare the health benefits of policies to their economic costs. Going beyond health risk assessment, this field might be called health risk evaluation. By evaluation, I mean the analytic process of scoring or rating health effects in terms of overall burden on the patient and society and quantifying the benefits of health policies in monetary or other units that capture the preferences of the public.

The central intellectual challenge is to account for the adverse effects of both morbidity and mortality in a single numerical index, recognizing that some bouts of sickness are more severe than others and some deaths may be considered more tragic than others. Although answers to these questions require value judgements, the social sciences have much to offer in providing possible answers.

In the developing world, the World Health Organization has promoted a metric called the disability-adjusted life year, or the DALY. Diverse health problems from infectious disease to trauma are scored in terms of the number of DALYs that a society loses. The scoring of each health condition is based on three factors: the number of life years lost compared to the Japanese experience, weights are applied to each healthy life year to reflect productivity at work and at home, and weights for each unhealthy life year are applied to capture the degree of functional limitation imposed on a person. Although my description of DALYs has been simplistic, the tool continues to be refined in various ways.

A precursor to the DALY, the quality-adjusted life year or QALY, is now the standard measure of health policy effectiveness used by academics in North America and Europe. It is similar to the DALY but differs in three important ways. First, the loss of life expectancy from each condition is derived from models or data for a relevant target population rather than from the Japanese experience. Second, each year of healthy life is weighted the same, regardless of when in the lifespan the loss occurs. This assumption is defended on grounds of fairness and analytic simplicity. Finally, the weights for different health states are typically derived in surveys of community residents or patients who have experienced the condition. Early work on QALYs was very simplistic in its use of utility theory, clinical insights, and of survey methods, but the quality of QALY research is improving.

Both the DALY and QALY methods, due to their very assumptions, place more emphasis on risks affecting middle-age adults and young people than risks of the chronic diseases of old age. This emphasis is not necessarily consistent with the current investment patterns in the US health care system, where billions of dollars are devoted to health care late in the lifespan. And economic critics of DALYs and QALYs argue that these methods depart from classical assumptions.

For example, a standard economic view is that consumer sovereignty should be respected: if well-informed senior citizens -- who often have substantial assets and precious few remaining life years -- have a large demand for modest health gains, so be it. And there is a growing literature in economics aimed at quantifying the monetary benefits of reducing different diseases and health impairments. Yet these willingness-to-pay tools face a host of ethical and technical criticisms.

At the most basic ethical level, concerns have been raised about whether a person's income level or asset position should be permitted to influence how much a governmental study values their health status. Others have argued that a fair allocation of public resources throughout the lifespan should be determined in a so-called "veil of ignorance," where people's views are not influenced by their current age or health status. Yet there are certainly some practical problems with implementing that idea!

At a technical level, substantial progress has been made in estimating willingness to pay for health protection. Yet questions have also been raised about whether stated willingness to pay by survey respondents is adequately sensitive to the amount of risk reduction and the context of risk reduction. In light of these many difficulties, OMB does not currently require agencies to value health gains and losses in monetary units and some agencies, such as OSHA and NHTSA, have refrained from this practice.

I believe that a major conference on health risk evaluation should be convened to assess the state of the field and make recommendations on future research and analytic practice.

MODEST STEPS BY OMB

At OMB we see health policy occurring at a variety of federal agencies: FDA, EPA, USDA, CMS, CDC, to name just a few. These agencies are now using a wide range of analytic practices that are not always mutually consistent. Rarely are these analytic practices rooted in statute.

OMB has an important role to play in fostering a greater degree of consistency. Yet agencies sometimes have important reasons for doing things differently and OMB's challenge is to appreciate when these distinctions have merit. We recognize, for example, that EPA considers policies that promise important ecological benefits that are not captured in any measure of human health benefit.

In the Bush Administration, we support development of health regulations that are based on sound science and economics. We are certainly not allergic to regulation; indeed, we view it as a critical tool of national policy -- as critical as government spending and taxation. My Office in particular strives to prompt and approve good health rules while improving flawed rules and stopping harmful ones. We see better formal analysis as a critical step toward more effective, fair and efficient health policies.

In my first year at OMB, we have taken two modest steps to promote more analytic rigor and consistency in health policy throughout the federal government. You can learn more about these stops on OMB's web site.

First, OMB recently established analytic guidelines for health risk assessment that cover all the federal agencies. The guidelines, based on two rounds of public comment and interagency review, are rooted in principles found in the Safe Drinking Water Act Amendments of 1996. A notable feature of the guidelines is a requirement that agencies, when faced with scientific uncertainty about important health issues, prepare central estimates of risk as well as upper and lower bounds on the true yet unknown risks. Each federal agency engaged in health risk assessment is now preparing specific guidelines that adopt or adapt the formulation found in the Safe Drinking Water Act. When these guidelines take effect in October of this year, the public will be provided an opportunity to challenge any health risk information disseminated by a federal agency that does not adhere to the OMB and agency guidelines. Agencies will be expected to provide a prompt and objective response to these challenges.

Second, OMB - in collaboration with CEA - has initiated a process to refine the guidelines for regulatory analysis that agencies must follow when preparing economic analyses to support rules. In this first phase, we have asked for public comment by late May on specific analytic issues - including health risk evaluation -- that should be addressed in OMB's re-evaluation of existing guidance. Examples of issues we intend to address are the methods of valuing human health effects and the proper discount rate to apply to future benefits and costs. We will prepare a proposed revision to OMB's analytic guidance that will be released for public comment, interagency review, and external peer review. The final OMB guidance will then be used by my Office to judge whether the regulatory analyses prepared by agencies are adequate. Please consult OMB's web site for more details about the comment period that extends through late May.

Looking forward, in order to enhance the roles of science and economics in health policy, we must do something that is as obvious as it is critical: improve the technical and ethical foundations of the tools used to inform policy makers. Organizations like Resources for the Future have played an influential role in the past and I only see that role increasing in importance in the years ahead.

Thank you for the opportunity to speak today and I look forward to your questions and comments.