Information Quality: An Update on the First Year

John D. Graham, Ph.D., Administrator
Office of Information and Regulatory Affairs
Office of Management and Budget
Executive Office of the President

Speech at the American Bar Association
Section of Environment, Energy, and Resources
11^th Section Fall Meeting
Washington DC

October 8, 2003

Slickspot peppergrass, the age 60 rule, barium, the Northern Goshawk, anthraquinone, and atrazine. One would not normally expect to mention these topics in the same discussion, let alone the same sentence. However, thanks to the Information Quality Act, these subjects are now linked. The Fish and Wildlife Service received a correction request regarding its proposed listing of slickspot peppergrass (a small plant that occurs in sage-brush areas in Idaho) as an endangered species; the Federal Aviation Administration received a correction request about a rule that says pilots are prohibited from flying commercially at age 60 and beyond; the Office of Research and Development, at EPA, received a correction request about the safe level for exposure to barium (a naturally occurring metal); the Forest Service received a correction request regarding habitat management for the Northern Goshawk (a bird from the hawk family that lives in forests); the National Toxicology Program received a correction request about preliminary test results for anthraquinone (a chemical used as an intermediate in the production of dyes, other organics, birdseed, and other areas); and the Office of Pesticides, Pollution and Toxic Substances, at EPA, received a correction request regarding the potential for endocrine disruption of atrazine (a herbicide frequently used by corn growers). These are just a few examples of the breadth of topics that the Information Quality Act has touched upon.

It is my pleasure to be here today to speak with you about the Information Quality Act. The federal agencies have just completed the first year of implementing this new law, and I am happy to share with you what we have learned during this first year.

HISTORY

The story began towards the end of the previous Administration, when Congress enacted a law requiring OMB to develop procedures to improve the quality of information disseminated by federal agencies. The law was enacted as a rider to our appropriations bill.¹ Informally known as the “Information Quality Act”, the law does two main things: (1) it requires the agencies to develop pre-dissemination procedures that will ensure the quality of information disseminated by the agencies. (2) it requires agencies to develop an administrative mechanism whereby affected parties can request that agencies correct poor quality information. In the implementing guidelines, OMB applies a stricter quality standard to information that is considered “influential”. The practice of peer review plays an important role in the guidelines, particularly in establishing a presumption that peer-reviewed information is “objective.” Furthermore, if the public is dissatisfied with the initial agency response to a complaint, an appeal opportunity is provided by the Agencies.

The Bush Administration is committed to vigorous implementation of the Information Quality Law. We believe it provides an excellent opportunity to enhance both the competence and accountability of government.

PERCEPTIONS AND REALITIES

OMB has heard many concerns about the Information Quality Law and the implementation process. I would like to share with you some of those concerns, as well as the perceptions and the realities that have come to be associated with them.

Perception:

Agencies will be inundated with requests for corrections.

Reality:

The strong belief that certain agencies would be overwhelmed by the volume of complaints was one of the most common early perceptions. To many peoples’ surprise, that has not been the case. In total, the agencies have received about 30 complaints that appear to be stimulated by the Information Quality Law. There has been a large volume of complaints (almost 5000) to FEMA regarding requests for map correction changes as part of the national flood insurance program, and a large volume of requests (about 100) to the Federal Motor Carrier Safety Administration (FMCSA) regarding the incorrect reporting of individual accidents. However, these kinds of complaints were commonplace prior to the Information Quality Law. Of the approximately 30 distinctive complaints, EPA, HHS, Interior and USDA have received most of the complaints, and a few have gone to Commerce, Education, and OSTP (the White House Office of Science and Technology Policy).

Perception:

The Information Quality correction process is a tool only for industry.

Reality:

I’m pleased to report that the Information Quality Act has been used by virtually all segments of the political spectrum. Complaints have been filed by private citizens, corporations, farm groups, trade organizations, both liberal and conservative non-governmental organizations (for example: the Competitive Enterprise Institute (CEI), Wrestling Coaches Association, Sierra Club, John Muir Society, and Public Employees for Environmental Responsibility), and even other government agencies (an Air Force complaint to Fish and Wildlife). The Information Quality Law has even been used by four U.S Senators (a joint complaint by Senators Boxer, Jeffords, Lautenberg and Sarbanes to EPA).

Perception:

The Information Quality Law will result in slowing down the regulatory process at the agencies.

Reality:

We can also report that to date, neither OMB, nor our engaged stakeholders, has noticed or commented upon any slowdown of the regulatory process. Twice a year the agencies provide OMB and the public with their regulatory agendas. This is a compendium of rules that the agencies intend to take action on within the next 12 months. This acts as a management tool for the agencies and lets OMB and stakeholders know what agencies are planning. Once a rulemaking arrives at OMB, through our tracking mechanisms we know how quickly the rule moves through the review process. Additionally, on the OIRA website the public can see when a rule arrives, and when it is cleared. To our knowledge, the passing of the Information Quality Law has not affected the pace or length of rulemakings.

Nonetheless, as is seen with implementing many new administrative processes or statutes within a government agency, implementation of the Information Quality Law has been a learning experience. Agencies are finding that it takes longer than they expected to respond to correction requests. Similarly, it is also taking longer than expected for agencies to implement the appeals process. At some of the larger agencies, finding the correct specialist to respond to specific requests has not been an easy task. Furthermore, ensuring that the correct specialist has sufficient time to give priority to an information quality complaint, among all his or her other duties, has also been challenging.

Perception:

Implementation of the guidelines may chill agency disseminations.

Reality:

This was a concern that was frequently brought to OMB’s attention as the guidelines were being developed. We currently have no evidence that points to a reduced number of agency disseminations. Additionally, at least in the disseminations that OIRA has reviewed through the agencies’ rulemaking processes, we do believe that the agencies are making efforts to ensure that information disseminated through this process meets the standards set forth in the Information Quality Act.

Perception:

The appeals process, the public’s opportunity to ask for reconsideration of a complaint, will not improve anything.

Reality:

Most of the Information Quality responses to requests for correction that were denied have subsequently been appealed. The majority of the appeals are still in the process of being answered; thus, it is too early to assess the value added. However, this added step appears to have fostered corrections. The appeals process requires independent agency review of the reconsideration request, its justification, and its strength. We recently saw this process play out at HHS where, upon appeal, a complaint to the National Toxicology Program resulted in the discontinuation of the webpage dissemination of a draft abstract that contained results that were flawed (the compound tested contained a contaminant that was believed to have influenced the test results). In this situation, the appeals step was critical in order for the agency to recognize that a correction was needed.

Perception:

The Information Quality Law is aimed primarily at information in federal rulemakings.

Reality:

Most complaints that agencies have received have not been directly related to rulemakings. The correction requests have been directed towards information that is predominantly disseminated to the public as reports, notices, or as a means of sharing agency findings on webpages. These disseminations may eventually lead to regulations at federal, state and local levels, but the disseminations themselves are not rules nor are they typically contained in rulemaking notices. This is not surprising, as I have been told by the OIRA staff that many of the concerns in the previous administration, which led to the creation of the Information Quality Act, were related to questionable disseminations of agency information on their websites, not necessarily via rulemakings.

Perception:

The Information Quality Act is only about numerical data.

Reality:

If you think that the word ‘data’, as defined by Webster, includes “information organized for analysis or used as the basis for decision-making,” then there has been no misperception.² However, if you are like many people and think that data covered by the Information Quality Act must be numerical information, then you are incorrect. The Information Quality Act has been used to address complex issues and analyses that go beyond correcting errors entered into a spreadsheet. For instance, whether or not the Trumpeter Swans (native North American swans characterized by their unmistakable trumpet-like call) constitute a distinct population around the Yellowstone area, and whether or not the nickel section of the 10^th edition of the Report on Carcinogens is representative of the full body of scientific studies, are not questions that can be answered solely by looking at numerical inputs. These are just two examples of the types of complaints we have seen that deal with the information and analysis used in the decision-making process.

Perception:

Colleges and Universities are regulated by the Information Quality Law.

Reality:

OMB has heard claims that college professors and their students, if funded by the federal government, are covered by the Information Quality Law and agency guidelines. OMB believes this is a misreading of the law. The Information Quality Act covers only disseminations by federal agencies, specifically those agencies covered by the Paperwork Reduction Act. The Law does not cover colleges and universities, even when federal research funding is involved. More generally, the law covers only agency disseminations, not disseminations made by third parties (e.g., academics, stakeholders and the public). As a practical matter, it may nonetheless make sense for third parties to consider the quality of information that they disseminate or submit to the federal government. If third-party submissions are to be used and disseminated by federal agencies, it is the responsibility of the federal government, under the Information-Quality Act, to make sure that such information meets relevant information-quality standards. The agency guidelines establish performance goals and procedures to assist in the agency’s evaluation of all information for which agency dissemination is under consideration, whether that information was generated by the agency or by third parties.

OMB’s LEARNING CURVE

At OMB, we have also learned that interpreting the Information Quality Law involves some complications. For instance, we have learned, along with the agencies, that the notion of what constitutes a “dissemination” is not straightforward. Agencies have had to figure out if an oral statement by a regional employee at a public meeting, or if statements in an email to a citizen, constitutes a dissemination. Similarly, determining when an agency-commissioned study becomes subject to the Information Quality Guidelines raises complex questions. When one agency’s dissemination is used by another agency, determinations become trickier. The Department of Education recently grappled with this issue when it received a correction request that asked the department to stop disseminating one of the Secretary’s Commission Reports because of the belief that it relied upon a study that was flawed. The study in question was produced by the GAO. Deciphering the ‘correct’ or best answer to questions such as these has been challenging.

On the bright side, many of the ‘non-influential’ Information Quality correction requests have identified specific information that can be corrected, and these requests have described clear corrections. These corrections have usually been made by the agencies.

We have also learned that passing a statute on information quality is easier than improving the quality of information. Often complaints hinge on the interpretations of science or analyses. When dealing with uncertain scientific issues, it is possible to draw several reasonable inferences depending on the perspective of the reviewer. Thus more than one plausible answer or methodology may exist. We are learning that it is possible for neither the agency nor the requestor to be incorrect. Thus far, the majority of correction requests have been denied, usually on the basis that a reasonable scientist could interpret the available information in the way that the agency had. Such complaints might have been better focused if they had addressed the inadequate treatment of uncertainty rather than the accuracy of information.

LOOKING FORWARD

Despite all the misperceptions, kinks, and surprises, we feel that we are moving closer to achieving the goals of the Information Quality Act. Agencies are aware that ensuring the high quality of government information disseminations is a high priority of the Bush Administration.

By January 1, 2004, all the agencies will be sending to OMB their first annual reports covering implementation of the Information Quality Act for Fiscal Year 2003. These reports will help us to better understand how each of the agencies is handling specific correction requests. The reports should also allow us to better gauge what parts of the process are working well and what parts need improvement.

We are still in the early phases of implementation, and we expect that the process will evolve and change as we look back and learn from our successes and mistakes. The importance of information quality justifies a strong commitment, and at this point in time we plan to continue our original stance of making this a priority.

THE PROPOSED BULLETIN ON PEER REVIEW

I’d like to devote a few minutes to the Proposed Bulletin on Peer Review that was recently released for comment by OMB. The proposal was triggered by the concern that existing agency predissemination review mechanisms have not always been sufficient to ensure the reliability of regulatory information disseminated or relied upon by federal agencies. While most agencies have policies that require or encourage peer review, they do not always conduct peer review according to their own policies – even for major rulemakings.

This proposed bulletin will give operational force to the pre-dissemination review objectives of the Information Quality Guidelines. The proposed bulletin suggests stricter peer review for the most important information. For especially significant regulatory information, which includes information where there may be a possible public or private sector impact of over $100 million dollars, the proposed bulletin establishes three requirements: 1) External peer reviewers should possess the necessary expertise and independence from the agency sponsoring the review; 2) the agencys must provide the peer reviewers with sufficient information and an appropriately broad charge; and 3) the agencies must publicly respond to the peer reviewers’ written reports, and make other appropriate disclosures. The bulletin also proposes annual agency reports on peer reviews, including a summary of scientific or technical studies that might constitute or support significant regulatory information and the agency’s plan for conducting the peer review of such studies. I would be very appreciative if each of you could take the time to look at our proposed bulletin and provide OMB with your feedback. OMB will be accepting comments on the proposed bulletin until December 15^th.

CONCLUDING REMARKS

We have appreciated the many thoughtful and provocative suggestions and comments we have received throughout this first year, and we look forward to your further involvement. Thank you very much for the opportunity to speak today.

¹ The law is Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001

² Websters II New Riverside Dictionary, Houghton Mifflin Company, Boston MA, 1984.