Information
Quality: An Update on the First Year
John D. Graham, Ph.D., Administrator
Office of Information and Regulatory Affairs
Office of Management and Budget
Executive Office of the President
Speech
at the American Bar Association
Section of Environment, Energy, and Resources
11th Section Fall Meeting
Washington DC
October 8, 2003
Slickspot
peppergrass, the age 60 rule, barium, the Northern Goshawk, anthraquinone,
and atrazine. One would not normally expect to mention these topics in
the same discussion, let alone the same sentence. However, thanks to the
Information Quality Act, these subjects are now linked. The Fish and Wildlife
Service received a correction request regarding its proposed listing of
slickspot peppergrass (a small plant that occurs in sage-brush areas in
Idaho) as an endangered species; the Federal Aviation Administration received
a correction request about a rule that says pilots are prohibited from
flying commercially at age 60 and beyond; the Office of Research and Development,
at EPA, received a correction request about the safe level for exposure
to barium (a naturally occurring metal); the Forest Service received a
correction request regarding habitat management for the Northern Goshawk
(a bird from the hawk family that lives in forests); the National Toxicology
Program received a correction request about preliminary test results for
anthraquinone (a chemical used as an intermediate in the production of
dyes, other organics, birdseed, and other areas); and the Office of Pesticides,
Pollution and Toxic Substances, at EPA, received a correction request
regarding the potential for endocrine disruption of atrazine (a herbicide
frequently used by corn growers). These are just a few examples of the
breadth of topics that the Information Quality Act has touched upon.
It is my
pleasure to be here today to speak with you about the Information Quality
Act. The federal agencies have just completed the first year of implementing
this new law, and I am happy to share with you what we have learned during
this first year.
HISTORY
The story
began towards the end of the previous Administration, when Congress enacted
a law requiring OMB to develop procedures to improve the quality of information
disseminated by federal agencies. The law was enacted as a rider to our
appropriations bill.1 Informally known as
the Information Quality Act, the law does two main things:
(1) it requires the agencies to develop pre-dissemination procedures that
will ensure the quality of information disseminated by the agencies. (2)
it requires agencies to develop an administrative mechanism whereby affected
parties can request that agencies correct poor quality information. In
the implementing guidelines, OMB applies a stricter quality standard to
information that is considered influential. The practice
of peer review plays an important role in the guidelines, particularly
in establishing a presumption that peer-reviewed information is objective.
Furthermore, if the public is dissatisfied with the initial agency response
to a complaint, an appeal opportunity is provided by the Agencies.
The Bush
Administration is committed to vigorous implementation of the Information
Quality Law. We believe it provides an excellent opportunity to enhance
both the competence and accountability of government.
PERCEPTIONS
AND REALITIES
OMB has heard
many concerns about the Information Quality Law and the implementation
process. I would like to share with you some of those concerns, as well
as the perceptions and the realities that have come to be associated with
them.
Perception:
Agencies
will be inundated with requests for corrections.
Reality:
The strong
belief that certain agencies would be overwhelmed by the volume of complaints
was one of the most common early perceptions. To many peoples surprise,
that has not been the case. In total, the agencies have received about
30 complaints that appear to be stimulated by the Information Quality
Law. There has been a large volume of complaints (almost 5000) to FEMA
regarding requests for map correction changes as part of the national
flood insurance program, and a large volume of requests (about 100) to
the Federal Motor Carrier Safety Administration (FMCSA) regarding the
incorrect reporting of individual accidents. However, these kinds of complaints
were commonplace prior to the Information Quality Law. Of the approximately
30 distinctive complaints, EPA, HHS, Interior and USDA have received most
of the complaints, and a few have gone to Commerce, Education, and OSTP
(the White House Office of Science and Technology Policy).
Perception:
The
Information Quality correction process is a tool only for industry.
Reality:
Im
pleased to report that the Information Quality Act has been used by virtually
all segments of the political spectrum. Complaints have been filed by
private citizens, corporations, farm groups, trade organizations, both
liberal and conservative non-governmental organizations (for example:
the Competitive Enterprise Institute (CEI), Wrestling Coaches Association,
Sierra Club, John Muir Society, and Public Employees for Environmental
Responsibility), and even other government agencies (an Air Force complaint
to Fish and Wildlife). The Information Quality Law has even been used
by four U.S Senators (a joint complaint by Senators Boxer, Jeffords, Lautenberg
and Sarbanes to EPA).
Perception:
The
Information Quality Law will result in slowing down the regulatory process
at the agencies.
Reality:
We can also
report that to date, neither OMB, nor our engaged stakeholders, has noticed
or commented upon any slowdown of the regulatory process. Twice a year
the agencies provide OMB and the public with their regulatory agendas.
This is a compendium of rules that the agencies intend to take action
on within the next 12 months. This acts as a management tool for the agencies
and lets OMB and stakeholders know what agencies are planning. Once a
rulemaking arrives at OMB, through our tracking mechanisms we know how
quickly the rule moves through the review process. Additionally, on the
OIRA website the public can see when a rule arrives, and when it is cleared.
To our knowledge, the passing of the Information Quality Law has not affected
the pace or length of rulemakings.
Nonetheless,
as is seen with implementing many new administrative processes or statutes
within a government agency, implementation of the Information Quality
Law has been a learning experience. Agencies are finding that it takes
longer than they expected to respond to correction requests. Similarly,
it is also taking longer than expected for agencies to implement the appeals
process. At some of the larger agencies, finding the correct specialist
to respond to specific requests has not been an easy task. Furthermore,
ensuring that the correct specialist has sufficient time to give priority
to an information quality complaint, among all his or her other duties,
has also been challenging.
Perception:
Implementation
of the guidelines may chill agency disseminations.
Reality:
This was
a concern that was frequently brought to OMBs attention as the
guidelines were being developed. We currently have no evidence that points
to a reduced number of agency disseminations. Additionally, at least in
the disseminations that OIRA has reviewed through the agencies
rulemaking processes, we do believe that the agencies are making efforts
to ensure that information disseminated through this process meets the
standards set forth in the Information Quality Act.
Perception:
The
appeals process, the publics opportunity to ask for reconsideration
of a complaint, will not improve anything.
Reality:
Most of
the Information Quality responses to requests for correction that were
denied have subsequently been appealed. The majority of the appeals are
still in the process of being answered; thus, it is too early to assess
the value added. However, this added step appears to have fostered corrections.
The appeals process requires independent agency review of the reconsideration
request, its justification, and its strength. We recently saw this process
play out at HHS where, upon appeal, a complaint to the National Toxicology
Program resulted in the discontinuation of the webpage dissemination of
a draft abstract that contained results that were flawed (the compound
tested contained a contaminant that was believed to have influenced the
test results). In this situation, the appeals step was critical in order
for the agency to recognize that a correction was needed.
Perception:
The
Information Quality Law is aimed primarily at information in federal rulemakings.
Reality:
Most complaints
that agencies have received have not been directly related to rulemakings.
The correction requests have been directed towards information that is
predominantly disseminated to the public as reports, notices, or as a
means of sharing agency findings on webpages. These disseminations may
eventually lead to regulations at federal, state and local levels, but
the disseminations themselves are not rules nor are they typically contained
in rulemaking notices. This is not surprising, as I have been told by
the OIRA staff that many of the concerns in the previous administration,
which led to the creation of the Information Quality Act, were related
to questionable disseminations of agency information on their websites,
not necessarily via rulemakings.
Perception:
The Information
Quality Act is only about numerical data.
Reality:
If you think
that the word data, as defined by Webster, includes information
organized for analysis or used as the basis for decision-making,
then there has been no misperception.2 However,
if you are like many people and think that data covered by the Information
Quality Act must be numerical information, then you are incorrect. The
Information Quality Act has been used to address complex issues and analyses
that go beyond correcting errors entered into a spreadsheet. For instance,
whether or not the Trumpeter Swans (native North American swans characterized
by their unmistakable trumpet-like call) constitute a distinct population
around the Yellowstone area, and whether or not the nickel section of
the 10th edition of the Report on Carcinogens is representative of the
full body of scientific studies, are not questions that can be answered
solely by looking at numerical inputs. These are just two examples of
the types of complaints we have seen that deal with the information and
analysis used in the decision-making process.
Perception:
Colleges
and Universities are regulated by the Information Quality Law.
Reality:
OMB has
heard claims that college professors and their students, if funded by
the federal government, are covered by the Information Quality Law and
agency guidelines. OMB believes this is a misreading of the law. The Information
Quality Act covers only disseminations by federal agencies, specifically
those agencies covered by the Paperwork Reduction Act. The Law does not
cover colleges and universities, even when federal research funding is
involved. More generally, the law covers only agency disseminations, not
disseminations made by third parties (e.g., academics, stakeholders and
the public). As a practical matter, it may nonetheless make sense for
third parties to consider the quality of information that they disseminate
or submit to the federal government. If third-party submissions are to
be used and disseminated by federal agencies, it is the responsibility
of the federal government, under the Information-Quality Act, to make
sure that such information meets relevant information-quality standards.
The agency guidelines establish performance goals and procedures to assist
in the agencys evaluation of all information for which agency dissemination
is under consideration, whether that information was generated by the
agency or by third parties.
OMBs
LEARNING CURVE
At OMB, we
have also learned that interpreting the Information Quality Law involves
some complications. For instance, we have learned, along with the agencies,
that the notion of what constitutes a dissemination is not
straightforward. Agencies have had to figure out if an oral statement
by a regional employee at a public meeting, or if statements in an email
to a citizen, constitutes a dissemination. Similarly, determining when
an agency-commissioned study becomes subject to the Information Quality
Guidelines raises complex questions. When one agencys dissemination
is used by another agency, determinations become trickier. The Department
of Education recently grappled with this issue when it received a correction
request that asked the department to stop disseminating one of the Secretarys
Commission Reports because of the belief that it relied upon a study that
was flawed. The study in question was produced by the GAO. Deciphering
the correct or best answer to questions such as these has
been challenging.
On the bright
side, many of the non-influential Information Quality correction
requests have identified specific information that can be corrected, and
these requests have described clear corrections. These corrections have
usually been made by the agencies.
We have also
learned that passing a statute on information quality is easier than improving
the quality of information. Often complaints hinge on the interpretations
of science or analyses. When dealing with uncertain scientific issues,
it is possible to draw several reasonable inferences depending on the
perspective of the reviewer. Thus more than one plausible answer or methodology
may exist. We are learning that it is possible for neither the agency
nor the requestor to be incorrect. Thus far, the majority of correction
requests have been denied, usually on the basis that a reasonable scientist
could interpret the available information in the way that the agency had.
Such complaints might have been better focused if they had addressed the
inadequate treatment of uncertainty rather than the accuracy of information.
LOOKING
FORWARD
Despite all
the misperceptions, kinks, and surprises, we feel that we are moving closer
to achieving the goals of the Information Quality Act. Agencies are aware
that ensuring the high quality of government information disseminations
is a high priority of the Bush Administration.
By January
1, 2004, all the agencies will be sending to OMB their first annual reports
covering implementation of the Information Quality Act for Fiscal Year
2003. These reports will help us to better understand how each of the
agencies is handling specific correction requests. The reports should
also allow us to better gauge what parts of the process are working well
and what parts need improvement.
We are still
in the early phases of implementation, and we expect that the process
will evolve and change as we look back and learn from our successes and
mistakes. The importance of information quality justifies a strong commitment,
and at this point in time we plan to continue our original stance of making
this a priority.
THE
PROPOSED BULLETIN ON PEER REVIEW
Id
like to devote a few minutes to the Proposed Bulletin on Peer Review that
was recently released for comment by OMB. The proposal was triggered by
the concern that existing agency predissemination review mechanisms have
not always been sufficient to ensure the reliability of regulatory information
disseminated or relied upon by federal agencies. While most agencies have
policies that require or encourage peer review, they do not always conduct
peer review according to their own policies even for major rulemakings.
This proposed
bulletin will give operational force to the pre-dissemination review objectives
of the Information Quality Guidelines. The proposed bulletin suggests
stricter peer review for the most important information. For especially
significant regulatory information, which includes information where there
may be a possible public or private sector impact of over $100 million
dollars, the proposed bulletin establishes three requirements: 1) External
peer reviewers should possess the necessary expertise and independence
from the agency sponsoring the review; 2) the agencys must provide the
peer reviewers with sufficient information and an appropriately broad
charge; and 3) the agencies must publicly respond to the peer reviewers
written reports, and make other appropriate disclosures. The bulletin
also proposes annual agency reports on peer reviews, including a summary
of scientific or technical studies that might constitute or support significant
regulatory information and the agencys plan for conducting the
peer review of such studies. I would be very appreciative if each of you
could take the time to look at our proposed bulletin and provide OMB with
your feedback. OMB will be accepting comments on the proposed bulletin
until December 15th.
CONCLUDING
REMARKS
We have appreciated
the many thoughtful and provocative suggestions and comments we have received
throughout this first year, and we look forward to your further involvement.
Thank you very much for the opportunity to speak today.
1
The law is Section 515 of the Treasury and General Government Appropriations
Act for Fiscal Year 2001
2
Websters II New Riverside Dictionary, Houghton Mifflin Company, Boston
MA, 1984. |