Detailed Information on the
Reactor Inspection and Performance Assessment Assessment

Improve integration of budget and performance information.

Complete an independent evaluation of the program consistent with guidance in OMB Circular A-11.

Improve and limit the number of performance measures for the program, ensuring that measures capture the scope of program activities, demonstrate year-to-year progress, and promote continued improvement.

The NRC will better demonstrate contributions of program activities and resources to outcomes and outputs. Through an agency-wide working group, NRC will improve the efficiency of operating plans.

More transparency is needed with respect to how resource allocation decisions are made and how safety indicator goals and program goals contribute to the agency's long-term goals. In response, NRC will better explain the contributions of program activities and resources to outputs. Complete the NRC's review of operating plan format and content to improve the plan's effectiveness as management tools. The scope of the project was separated into two phases to address: (1) improvements that could be implemented in the short-term; and (2) improvements that would require longer-term planning and evaluation. The short-term improvement efforts were completed in December 2004 through the development of a performance reporting framework containing common reporting criteria and format. This framework was implemented during the first quarter of FY 2005. The longer-term efforts to improve the efficiency of operating plans are currently being addressed by an agency wide working group.

Measure: No nuclear reactor accidents.

Explanation:Defined as those accidents which result in substantial damage to the reactor core, whether or not serious offsite consequences occur.

Measure: No acute radiation exposures resulting in fatalities.

Explanation:

Measure: No releases of radioactive materials that result in significant radiation exposures.

Explanation:Significant radiation exposures defined as those that result in unintended permanent functional damage to an organ or physiological system.

Measure: No releases of radioactive materials that cause significant adverse environmental impacts.

Explanation:Releases that have the potential for an adverse impact is taken to mean those that exceed the reporting limits given by Abnormal Occurrence Criterion 1.B.1 of Appendix B to 10 CFR Part 20.

Measure: No instances where licensed radioactive materials are used domestically in a manner hostile to the security of the United States.

Explanation:

Measure: Stakeholders are informed and involved in NRC processes as appropriate.

Explanation:

Measure: No significant licensing or regulatory impediments to the safe and beneficial uses of radioactive materials.

Explanation:

Measure: Number of new conditions evaluated as red by the NRC's reactor oversight process.

Explanation:

Measure: Number of significant accident sequence precursors (ASPs) of a nuclear reactor accident.

Explanation:

Measure: Number of operating reactors whose integreated performance entered the Manual Chapter 0350 process, the multiple/repetitive degraded cornerstone column or the unacceptable performance column of the ROP Action Matrix with no performance exceeding Abnormal Occurrence Criterion I.D.4.

Explanation:

Measure: Number of significant adverse trends in industry safety performance.

Explanation:

Measure: Numbers of events with radiation exposures to the public and occupational workers that exceed Abnormal Occurrence Criterion I.A.

Explanation:

Measure: Number of radiological releases to the environment that exceed applicable regulatory limits.

Explanation:

Measure: Unrecovered losses or thefts of risk-significant radioactive sources.

Explanation:

Measure: Number of security events and incidents that exced the Abnormal Occurrence Criteria I.C 2-4

Explanation:

Measure: Number of significant unauthorized disclosures of classified and/or safeguards information.

Explanation:

Measure: Percentage of stakeholders that perceive the NRC to be open in its processes is equal to or greater than other Federal Agency measures, when available.

Explanation:The Federal Agency Weighted Average is obtained from a survey done by the American Customer Satisfaction Index, which has ratings for a number of federal regulatory agencies. The measure is new in FY 2006. There was no survey of NRC stakeholders in FY 2005, though one was conducted in FY 2004.

Measure: Percentage of selected openness output measures that achieve performance targets.

Explanation:

Measure: Percentage of selected processes that deliver desired efficiency improvement no less than 70 percent.

Explanation:

Measure: Number of instances where licensing or regulatory activities unnecessarily impede the safe and beneficial uses of radioactive materials.

Explanation:

Measure: Percentage of stakeholder formal requests for information that receive an NRC response within 60 days of receipt.

Explanation:

Measure: Percentage of non-sensitive, unclassified regulatory documents generated by the NRC and sent to the agency's Document Processing Center that are released to the public by the sixth working day after the date of the document.

Explanation:

Measure: Percentage of non-sensitive, unclassified regulatory documents received by the NRC that are released to the public by the sixth working day after the document is added to the ADAMS main library.

Explanation:

Measure: The NRC achieves a user satisfaction score for the agency's public website greater than or equal to the Federal Regulatory Agency Mean score based on results of the yearly American Customer Satisfaction Index for Federal Web sites.

Explanation:

Measure: Median number of days for responding to Freedom on Information Act requests.

Explanation:

Measure: Percentage of final Significance Determination Process determinations that are made within 90 days of inspection report issue date for all potentially greater-than-green findings.

Explanation:

Measure: Percentage of stakeholders that believe they were given sufficient opportunity to ask questions or express their views.

Explanation:

Measure: Percentage of Category 1, 2, and 3 meetings on regulatory issues for which the NRC issues public notices 10 days in advance of the meeting.

Explanation:

Measure: Complete all the key stakeholder and public interactions for the reactor performance assessment cycle consisting of mid-cycle review and letter report, end-of-cycle review report and letter, public meetings, agency action review, and Commission meeting.

Explanation:Conduct 103 mid-cycle reviews and 103 end-of-cycle reviews. Issue mid-year annual assessment letters and conduct annual meeting with licensee at site.

Measure: Percentage reduction in average enforcement processing time.

Explanation:Enforcement Process for Handling Discrimination Allegations

Measure: Percentage reduction in resources expended in support of each interagency exercise while still accomplishing agency goals for each exercise.

Explanation:Incident Response and Emergency Preparedness Exercises

Measure: No radiological sabotage at nuclear reactors

Explanation:

Measure: No events resulting in radiation overexposures from nuclear reactors that exceed applicable regulatory limits

Explanation:Overexposures are those that exceed limits as provided by 10 CFR 20.2203(a)(2)

Measure: No breakdowns of physical security that significantly weaken the protection against radiological sabotage or theft or diversion of special nuclear materials in accordance with abnormal occurrence criteria.

Explanation:

Is the program purpose clear?

Explanation: The U.S. Nuclear Regulatory Commission (NRC) regulates the Nation's civilian use of byproducts, source, and special nuclear materials to ensure adequate protection of public health and safety, to promote common defense and security, and to protect the environment. The NRC has several programs to fulfill its responsibility to protect public health and safety, one of which is the Reactor Inspection and Performance Assessment program. This purpose of this program is to ensure that the 104 power reactors licensed to operate identify and resolve safety issues before they affect safe plant operation.

Evidence: The Atomic Energy Act of 1954, as amended Mission Statement from the NRC FY2002 Performance and Accountability Report, pages 2 and 5. FY2004 Budget Estimates and Performance Plan, page 50.

Does the program address a specific and existing problem, interest, or need?

Explanation: The Reactor Inspection and Performance Assessment program provides the NRC regulatory oversight of commercial operating power reactors. The reactor inspection program provides the means for the NRC to gather information on licensee performance and oversee safe operation. The assessment process provides the means for the NRC to use this information to identify performance deficiencies and determine appropriate Agency actions in response.

Evidence: The Atomic Energy Act of 1954, as Amended, Section 25 states that an Inspection Division shall be responsible for gathering information to show whether or not licensees are complying with the provisions of this Act and the appropriate rules and regulations of the Commission.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The NRC has the sole responsibility to license commercial power reactors and ensure that these facilities are being operated in accordance with license conditions and other Federal regulations. As discussed later, the NRC does collaborate with other State and Federal agencies on some aspects of the oversight of operating commercial power reactors.

Evidence: The Atomic Energy Act of 1954, as Amended, Section 101, states that it shall be unlawful for any person within the United States to acquire, possess, or use any utilization facility except under and in accordance with a license issued by the Commission pursuant to section 103.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Reactor Oversight Process (ROP) is the key component of the Reactor Inspection and Assessment program and was designed to improve the oversight processes by making them more objective, predictable, understandable, and risk-informed. This initiative resulted from internal reviews, external stakeholder input, and direction from the Commission, and was specifically designed to address the interests, problems, and needs of all stakeholders. The ROP also includes a built-in self-assessment process, including senior management review, to ensure that the program continues to meet the interests and needs of its stakeholders. Independent external stakeholders have responded favorably to the ROP as a significant improvement over the previous oversight programs, and annual self-assessments have concluded that the ROP is effective.

Evidence: NRC Commission paper SECY-03-0062 dated April 21, 2003, provides the results of the latest self-assessment of the ROP. Also Reference SECY-99-007 and 007A, SECY-00-0049, SECY-01-0114, and SECY-02-0062, as well as ACRS letters dated March 13, 2003 (ML030730366) and February 13, 2002 (ML020500775), and the OIG report dated August 21, 2002. Commission memo on "Results of the NRC Agency Action Review Meeting, April 22-23, 2003," dated May 2, 2003.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Resource allocations for the program target the direct work (70% of program resources) performed either by NRC technical and professional staff or by NRC contractors, as well as the overhead and support activities (30% of resources) needed to implement the program. Approximately 90% of the total program resources are directed to the four regional offices to conduct inspections, assess reactor performance, respond to events, and address allegations. The remaining 10% of resources are directed to NRC Headquarters to: support continuing program development, improvement, and oversight; address emergency preparedness; maintain liaison with State, local, and tribal organizations and other Federal agencies; and conduct legal, investigative, and enforcement activities. In addition to targeting specific resources to support the program as described in the NRC budget, the Agency also monitors the use of funding and staff during the execution year to ensure that resources are expended as planned.

Evidence: Budget Estimates and Performance Plan - Fiscal Year 2004. NRC MD 4.2, "Administrative Control of Funds". NRR Rainbow Reports

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The strategic goal for the oversight of power reactors through the Reactor Inspection and Performance Assessment program is to prevent radiation related deaths and illnesses, promote the common defense and security, and protect the environment in the use of civilian nuclear reactors. The NRC has identified five measures to determine if it is meeting this strategic goal.

Evidence: FY2002 Performance and Accountability Report, Chapter 2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets for the Nuclear Reactor Safety Strategic Goal performance measures are very ambitious. In fact they are zero for all five measures. These are also long-term performance measures that generally have an unlimited timeframe. These targets and timeframes are appropriate given the extremely low frequency and high consequence of the events that would contribute to these performance measures.

Evidence: FY2002 Performance and Accountability Report, Chapter 2, page 36

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: The NRC has established performance measures such as "No statistically significant adverse industry trends in safety performance" and "No more than one event per year identified as a significant precursor of a nuclear accident" that provide indication on an annual basis of the programs ability to meet its long-term goal of maintaining safety. Another key performance goal of the program is to make it more effective and efficient. The ROP self-assessment program includes several measures that promote continuous improvement and drive the staff to evaluate the program annually for effectiveness and efficiency improvements.The ROP tracks and trends 39 performance metrics related to its four principal functional areas and 19 performance metrics related to the overall effectiveness of the ROP.

Evidence: FY2002 Performance and Accountability Report, Chapter 2. Budget Estimates and Performance Plan FY2004. IMC 0307, Reactor Oversight Process Self-Assessment Program , Appendix A. For example, performance measure IP-9 in IMC 0307 requires the analysis of inspection hours expended against budgeted resources. Reactor Oversight Process Self-Assessment for Calendar Year 2002 (SECY-03-0062). FY 2003 Operating Plan and quarterly updates.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Data for the annual performance measures related to maintaining safety have been collected and reported for several years, thus establishing an adequate baseline level of performance for each measure. Ambitious targets have also been set for each measure, with several of the targets being zero. These targets are appropriate given the extremely low frequency and high consequence of the events being measured. The ROP self-assessment program includes several measures that promote continuous improvement and drive the staff to evaluate the program annually for effectiveness and efficiency improvements. However, the Commission determined early during the development of the ROP that establishing resource demands artificially would be inconsistent with the goal of maintaining safety. Therefore, specific measures and targets for cost-effectiveness of the program have not been developed. Resource requirements for the program are determined by using risk-insights to determine those Agency actions required to provide reasonable assurance of public health and safety.

Evidence: FY2002 Performance and Accountability Report, Chapter 2, pages 32 and 37. IMC 0307, Reactor Oversight Process Self-Assessment Program, Appendix A. Item 8 of the Staff Requirements Memorandum for SECY-99-007 and SECY-99-007A dated June 18, 1999 provides Commission guidance establishing resource measures for the program.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: There are several key partners for the Reactor Inspection and Performance Assessment program. Most important are the four NRC Regional Offices which implement the program on a day-to-day basis. Each regional office has developed and implemented an operating plan and performance metrics that measure program performance against the strategic and performance goal measures. In addition, the NRC's performance measure of "No statistically significant adverse industry trends in safety performance" links the performance of the regulated entities to the performance goal of maintaining safety. Industry performance is a key input in evaluating the effectiveness of the Reactor Inspection and Performance Assessment program. An Industry Trends Program (ITP) has been developed by the staff to identify and evaluate adverse trends, and take appropriate action. The results of this program are documented in an annual Commission Paper and reviewed by senior NRC managers as part of the annual Agency Action Review Meeting and Commission briefing on the status of the ROP.

Evidence: FY2002 NRC Regional Office Operating Plans. FY2002 Performance and Accountability Report, pages 32 and 33. Commission Paper SECY-03-0057 "Results of the Industry Trends Program for Operating Power Reactors and Status of Ongoing Development." Management Directive 8.14 "Agency Action Review Meeting."

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Several independent advisory committees reviewed the ROP prior to implementation and continue to evaluate the program on a regular basis, including the Advisory Committee on Reactor Safeguards (ACRS) and the Office of the Inspector General (OIG). Specific panels were established, by charter under the rules of the Federal Advisory Committees Act, to independently evaluate ROP effectiveness; namely, the Pilot Program Evaluation Panel (PPEP) and the Initial Implementation Evaluation Panel (IIEP). In addition, annual surveys via Federal Register notice are administered to obtain stakeholder input regarding the efficacy of the ROP and provide insights for improvement. These critical reviews have resulted in several program enhancements as described in the annual self-assessments, including developing a structured self-assessment program, streamlining the Significance Determination Process, refining several performance indicators, and clarifying the inspection reporting guidance.

Evidence: Reference IIEP report dated May 10, 2001 (ML011290025, attachment 4 to SECY-01-0114), PPEP report dated December 21, 1999, (ML993550449, attachment 2 to SECY-00-0049), as well as ACRS letters dated March 13, 2003 (ML030730366) and February 13, 2002 (ML020500775), and the OIG report dated August 21, 2002 (Review of NRC's Significance Determination Process, OIG-02-A-15). NRC Inspection Manual Chapter 0307, "Reactor Oversight Process Self-Assessment Program"

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget for the Inspection and Performance Assessment program reflects the activities and anticipated level of effort that contributes to achieving the four performance goals that support the Agency's mission. Program resources are aligned annually in accordance with the concept of "prioritization" '' defined in NRC's Planning, Budgeting and Performance Management (PBPM) process as 'the ranking of activities ... based on their contribution to performance goals.' In the past, the direct and overhead costs for this program have been clearly identified in the NRC budget, and beginning in the FY 2005 budget, full cost for the program will be shown in the budget document. Although the current budget presentation is more descriptive than analytical, future NRC budgets will provide additional analytical information and will reflect the impact of resource allocation on effectiveness and efficiency.

Evidence: NRC Strategic Plan (FY 2002-FY 2005); FY 2004 Budget Estimates and Performance Plan; FY 2002 Performance and Accountability Report; Memorandum to the Program Review Committee, 'Prioritized Listing of Program Office Activities by Arena for FY 2004 and FY 2005 Budgets,' dated April 16, 2003.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The NRC is currently revising strategic goals and performance goal measures, and including some ROP performance attributes in these. These revised measures, which are primarily output measures, are then incorporated into the annual performance plan. The ROP is the main process for regulatory oversight under the Reactor Inspection and Performance Assessment program. NRC has been developing and using risk-informed and less-prescriptive performance-based regulatory approaches, where appropriate, to maintain safety and promote efficiency. As a direct result of this process, efficiencies have been identified for FY 2004, freeing up staff and budget to address unanticipated developments, such as the Davis-Besse performance issues.

Evidence: NRC Strategic Plan (FY 2002-2005); FY 2004 Budget Estimates and Performance Plan;FY 2002 Performance and Accountability Report.

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: NRC regulations issued are considered necessary to provide assurance that licensees operate their facilities in a safe manner and the goals are met to protect public health and safety. The Reactor Inspection and Performance Assessment program ensures that licensees are complying with these requirements. NRR has issued office instructions for rulemaking that provide procedures and guidance to its staff. Any rule imposing requirements needs a backfit analysis (per 10CFR50.109 - Backfit Rule) either justifying that the requirements are necessary for adequte protection or are cost-beneficial safety enhancements. An internal review committee (Committee to Review Generic Requirements) reviews these analyses before any rule is forwarded to the Commission for consideration. Additionally, the NRC has undertaken various efforts to review its regulations to reduce unnecessary regulatory burden and ensure consistency with NRC goals. For example, the NRC has embarked on a number of rulemakings to risk-inform requirements in 10 CFR Part 50 and remove unnecessary regulations.

Evidence: NRC Regulations Handbook, NUREG/BR-0053, Rev 5 (ADAMS Assession No. ML011010183 and ML011010201) and Supplement 1 (ML021990398); and NRC Regulatory Analysis Guidelines, NUREG/BR-0058, July 2000 (See 3RG3 for web page references). LIC 300, 'Rulemaking Procedures' and Commission White Paper, 'Risk-informed and Performance-Based Regulations,' SRM to SECY-98-144, dated March 1,1999. SECY-98-300, Options for Risk-informed Revisions to 10 CFR Part 50, December 23, 1998; SECY-00-0198, Status Report on Study of Risk-informed changes to 10 CFR Part 50, September 14, 2000; SECY-02-0057, 4th Status Report on Study of Risk-informed changes to 10 CFR Part 50, March 29, 2002; SECY-03-0044, Update to Risk-informed Implementation Plan, March 21, 2003. Some current regulatory actions underway to conform with the initiatives of the program are: (1) Performance-Based Risk-Informed Fire Protection, (2) §50.69 - Risk-Informing 10 CFR Part 50, Option 2 (Special Treatment Requirements), (3) Risk-Informed 50.44 Rulemaking, (4) Fitness For Duty Rulemaking, and (5) Risk-Informed Part 73/Exercise Rule.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ROP's self-assessment program annually evaluates the program's success in meeting its intended objectives. A detailed program assessment, using objective criteria, is conducted annually to evaluate program effectiveness. The sources of the data include Regional Operating Plans, performance indicators, internal and external stakeholder surveys, independent audits, program documents reviews, and agency databases. The self-assessment program has resulted in significant improvements in the effectiveness and efficiency of the ROP, including streamlining the Significance Determination Process, refining several performance indicators, and clarifying the inspection and assessment guidance. The results of the annual self-assessment are discussed and confirmed by senior NRC management during the annual Agency Action Review Meeting, and are subsequently provided to the Commission and interested stakeholders. Plant-specific performance indicator data is used to improve regulatory oversight.

Evidence: Reference IMC 0307 and SECY-03-0062. MD 8.14 describes the Agency Action Review Meeting. FY2002 Performance and Accountability Report, pages 32 and 33.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: Each manager in the Office of Nuclear Reactor Regulation is responsible for development and implementation of specific items in the Strategic Plan and Operating Plan. These items are in their SES contracts, elements and standards for performance appraisals, which are used, in part, to determine promotions and awards, and our work tracking and assignment system (ticketing). The SES program is being modified for FY2004 to link individual goals even more explicitly to NRC goals. In addition, the ROP characterizes the performance of our licensee partners in an ongoing manner, updating this assessment quarterly. When licensee performance declines, the ROP has predictable, clearcut linkages to regulatory responses which include additional inspection and increased regulatory interface with licensee management. Licensee partners are held accountable for the safety performance of their plants, adherence to all regulatory requirements, and strive to operate in a manner that the ROP will explicitly reflect as acceptable performance.

Evidence: Management Directive 10.137, "Senior Executive Service Performance Management System." Mid-year review of each manager's performance; annual performance review which includes the performance review board compensation adjustments. Example of SES Performance Plan for Chief, Inspection Program Branch. IMC 0305 "Operating Reactor Assessment Program" creates an 'accountability structure' in that licensee's are given quarterly performance reviews that are made public in a clear and understandable manner.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: It is the policy of the NRC that agency systems for budget execution and the administrative control of funds conform to policies, procedures, and standards that comply with the requirements set forth in OMB circulars, the Antideficiency Act, the Impoundment Control Act of 1974, Chief Financial Officers Act of 1990, etc. Agency policies and procedures are documented in NRC Management Directive, Volume 4 'Financial Management'. NRC's Office of the Chief Financial Officer monitors commitments, obligations, and expenditures on a monthly basis and reports findings in monthly and quarterly reports in the Budget Execution Reports. NRC's Office of Nuclear Reactor Regulation tracks funding and staff utilization, and projects annual resource expenditures for the majority of resources in the Inspection and Performance Assessment program through the NRR Rainbow Reports which are issued monthly.

Evidence: NRC Management Directives, Manual Chapter 4.2 'Administrative Control of Funds'; Budget and Reporting Number Structure Guide; Management Directive and Handbook 10.43, Time and Labor Reporting; monthly Budget Execution Reports; NRR Rainbow Reports; Acquisition Certification and Training program for project managers, technical monitors, and all personnel who are part of the acquisition process as defined in the May 2000 memorandum to Office Directors and Regional Administrators from the Executive Director for Operations.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: As noted in the response to item 2.3, procedures are in place that establish clearcut measures and goals to monitor ROP performance. The annual ROP self-assessment further includes analysis of resources expended as compared to resources budgeted, with established goals as targets. The combined thorough evaluation of performance and cost provides regular insights from which ROP changes to improve efficiency and cost effectiveness have been made and are being made. NRC annually analyzes inspection resources required for preparation, travel, communication, conduct of the inspection, and documentation of results to identify ways to make inspection more efficient and effective. The NRC also formed an efficiency focus group to explore additional ways to achieve efficiency gains within the ROP. Ongoing efficiency evaluations include the consolidation of inspection procedures, the streamlining of the inspection reporting process, and reevaluating the scope and frequency of the annual public meetings.

Evidence: IMC 0307, Appendix A, measure IP-9, "Analysis of Inspection Hours". Commission paper SECY-03-0062, Attachment 7 "ROP Resource Analysis". NRC Letter to Congress on Efficiencies Gained Through Implementation of the ROP, dated March 31, 2003 (ML030690522)

Does the program collaborate and coordinate effectively with related programs?

Explanation: The NRC collaborates with many other Federal and State regulatory bodies on certain aspects of the Reactor Inspection and Performance Assessment program. For example, NRC works in conjunction with FEMA to provide oversight of Emergency Preparedness activities at and around the power reactor sites. Many states, such as the State of Illinois, regularly participate in inspections of power reactors lead by the NRC. The NRC also collaborates and coordinates internally with other programs related to power reactor oversight, such as the security and safeguards programs run by the Office of Nuclear Safety and Incident Response.

Evidence: Collaboration with FEMA as described in the NRC/FEMA memorandum of understanding, dated August 26, 1993. Per the Atomic Energy Act of 1954, as Amended, Section 274 (i), the Commission in carrying out its licensing and regulatory responsibilities under this Act is authorized to enter into agreements with any State, or group of States, to perform inspections or other functions on a cooperative basis as the Commission deems appropriate. Management Directive 5.2, "Memoranda of Understanding With States." NRC's Policy Statement on "Cooperation With States at Commercial Nuclear Power Plants and Other Nuclear Production or Utilization Facilities" (57 FR 6462, February 25, 1992).

Does the program use strong financial management practices?

Explanation: NRC financial management practices governing control of funds and resource allocation are codified in MD4.2 and are fully implemented by the reactor oversight process (ROP). The adequacy of these practices is reflected in the fact that NRC's financial statements have earned unqualified opinions for nine consecutive years. NRC's cost accounting system was identified as having a material weakness because the system is not in full compliance with SFFAS Number 4 by capturing the full cost of program outputs. NRC is implementing a remediation plan to resolve the instance of non-compliance; all other financial systems are in full compliance. NRC offers a financial management training seminar to staff twice a year on Administrative Control of Funds and Financial Management.

Evidence: NRC's Performance and Accountability Report for FY 2002, Monthly Budget Execution Reports (BER), Quarterly review of BER by top Agency management, NRC Management Directive 4.2, 'Administrative Control of Funds;' NRC Financial Management Seminar, 'Audit of the Nuclear Regulatory Commission's FY 2002 Financial Statements,' OIG-03-A-04.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The NRC has identified management challenges for the Reactor Inspection and Performance Assessment program in developing and implementing risk-informed and performance-based regulatory oversight. Actions taken to address these challenges resulted in significant changes and improvements to the program structure and its implementation. For example, during ROP development the assessment process was streamlined and consolidated from three separate processes to one and the core/baseline inspection program was revamped using risk-informed evaluations. Annual program self-assessments have produced improvements as a result of program deficiencies identified and lessons learned, as shown by the last one conducted for CY 2002. The NRC IG has also identified challenges in the implementation of the ROP, which the staff has taken actions to address. A recent internal employee survey aimed at determining trends in the NRC's 'safety culture' identified certain areas where improvement could be made. The NRC created a task force to review this safety culture survey and they published a report with recommendations (NRC Safety Culture & Climate, ADAMS number ML031630816), for which the staff comment period is still open.

Evidence: NRC Management Directive 4.4; Annual Reasonable Assurance Statements; FY2002 Performance and Accountability Report. NRC MD 8.14, "Agency Action Review Meeting," IMC 0305, "Operating Reactor Assessment Program," SECY-03-0062, "Reactor Oversight Process Self-Assessment for CY2002," SECY-00-0049, "Results of the Revised Reactor Oversight Process Pilot Program," IMC 0307,"Reactor Oversight Process Self-Assessment Program"; 'Management Challenges Appendix of Budget Estimates and Performance Plan FY 2004; OIG-03-A-02, 'Inspector General's Assessment of the Most Serious Management Challenges Facing NRC,' NRC Safety Culture & Climate, ADAMS number ML031630816.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: In support of the program, the rulemaking process at the NRC takes into account the views of the affected parties, recognizes the public's interest in the proper regulation of nuclear activities, and provides opportunities for citizens to make their opinions known. The NRC seeks to elicit public involvement early in the regulatory process so that safety concerns that may affect a community can be resolved in a timely and practical manner. All rulemakings provide the public with at least one opportunity for comment. Often, there are several opportunities. In some cases, NRC holds meetings and workshops before a proposed rule is drafted so that members of the public can express their concerns early in the process. Sometimes, the NRC may publish an Advance Notice of Proposed Rulemaking in the Federal Register to obtain public comments and provide clarification of certain issues before developing a proposed rule. NRC is subject to the Small Business Regulatory Enforcement Fairness Act, thus evaluating any expected impact on small businesses.

Evidence: NRC Regulations Handbook, NUREG/BR-0053 and NRC Regulatory Analysis Guidelines, NUREG/BR-0058. The handbooks assist NRC staff in drafting and preparing rulemaking documents for publication in the Federal Register. LIC 300, 'Rulemaking Procedures' and Commission White Paper and SRM to SECY-98-144, 'Risk-informed and Performance-Based Regulations,' dated March 1,1999. The following rulemakings pertinent to the Reactor Inspection and Performance Assessment program requested public participation and are stored in the NRC Document Management system ADAMS under the following accession nos. - ML021080576, ML021300030, & ML022630007.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: NRC is covered by SBREFA and the Regulatory Flexibility Act and is in full compliance with their requirements on applicable rulemakings. For example, the final Fee Rule for FY2003 (10CFR Parts 170 and 171), contains a Regulatory Flexibility Analysis and a SBREFA determination. As an independent agency, NRC is not bound by the Unfunded Mandate Reform Act, or for the most part, by Executive Order 12866. The one exception is the requirement in the Executive Order to regularly post the overall agency regulatory agenda, which the NRC does in full compliance with the order. However, as a matter of normal practice, the NRC performs cost-benefit analyses on proposed rules which are not on matters of immediate safety concern. The NRC's guidance directs the analyst to use OMB's 'Regulatory Impact Analysis Guidance,' Appendix V in Regulatory Program of the United States Government: April 1, 1992 - March 31, 1993, and 'Benefit-Cost Analysis of Federal Programs; Guidelines and Discounts,' Circular No. A-94, Federal Register, Vol. 57, November 10, 1992, pp. 53519-53528.

Evidence: 6/18/2003 Federal Register Notice 1010 CFR Parts 170 and 171 Revision of Fee Schedules; Fee Recovery for FY2003; Final Rule". SECY-00-0111. Regulatory Analysis Guidelines of the U.S. Nuclear Regulatory Commission, NUREG/BR-0058, Rev. 3, July 2002 The following proposed rulemakings pertinent to the Reactor Inspection and Performance Assessment program prepared regulatory impact analyses and are stored in the NRC Document Management system ADAMS under the following accession nos. - ML021080807, ML022630028, ML021080576, ML021300030, & ML022630007.

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: As fostered by Commission policy, and in recognition of risk insights, the NRC has undertaken various efforts to review its regulations to reduce unnecessary regulatory burden and ensure consistency with NRC goals. For example, the NRC has embarked on a number of rulemakings to risk-inform requirements in 10 CFR Part 50. Significant examples include the revision to 10 CFR 50.44 and the proposed addition of 10 CFR 50.69. Most recently, the NRC has been engaged in an initiative to improve the coherence among its risk-informed regulatory programs. As part of this effort, the NRC will be systematically looking at its regulations to determine whether they are built on a unified safety concept (and consistent with our performance goals) and are properly integrated. This effort is using information from the ROP to identify candidate areas where our regulations and our risk-informed oversight process may not be fully compatible.

Evidence: SECY-98-300, Options for Risk-informed Revisions to 10 CFR Part 50, December 23, 1998; SECY-00-0198, Status Report on Study of Risk-informed changes to 10 CFR Part 50, September 14, 2000; SECY-02-0176 Proposed Rule Risk-informed Categorization and Treatment of Structures, systems and components, September 30, 2002; SECY-02-0080 Proposed Rule Risk-informed revision to 10 CFR 50.44, Combustible Gas Control, May 13, 2002; SECY-02-0057, 4th Status Report on Study of Risk-informed changes to 10 CFR Part 50, March 29, 2002; SECY-03-0044, Update to Risk-informed Implementation Plan, March 21, 2003; Meeting Summary March 28, 2003 on Coherence (ML031210499); LIC-300 'Rulemaking Procedures'; White Paper, 'Risk-Informed and Performance-based Regulations,' SRM to SECY-98-144, dated March 1, 1999; SECY-00-0191, High-level Guidelines for Performance-based Activities, September 1, 2000.

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: NRC conducts regulatory impact analyses (RIAs) to determine whether proposed changes maximize benefits. NRC guidance states that 'OMB maintains that the regulatory analysis should select the regulatory alternative that achieves the greatest present value-the discounted monetized value of expected net benefits.' The NRC guidance also states 'Selecting the alternative with the largest net value is consistent with obtaining the largest societal gain from among the alternatives analyzed.' However, not all benefits can be quantified; and in some cases qualitative benefits were determined to justify the costs. Examples of significant RIAs are noted under Evidence.

Evidence: Regulatory Analysis Guidelines of the U.S. Nuclear Regulatory Commission, NUREG/BR-0058, Rev. 3, July 2002. Some of the supporting RIAs for rulemakings pertinent to the Reactor Inspection and Performance Assessment program are stored in the NRC Document Management system ADAMS under the following accession nos. - ML021080807 and ML022630028.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: The Reactor Inspection and Performance Assessment program has achieved its long term strategic goal to prevent radiation related deaths and illnesses, promote the common defense and security, and protect the environment in the use of civilian nuclear reactors. As previously discussed, the effectiveness of the program in achieving these goals has been has been the subject of frequent independent evaluations. While areas of program improvement have been noted, and the NRC continues to work to improve the program, these evaluations have concluded that the program is effective at providing reasonable assurance of the adequate protection of public health and safety.

Evidence: The program has met all of its strategic goal measures as demonstrated on page 36 of the FY2002 Performance and Accountability Report. For the results of independent evaluations, reference IIEP report dated May 10, 2001 (ML011290025, attachment 4 to SECY-01-0114), PPEP report dated December 21, 1999, (ML993550449, attachment 2 to SECY-00-0049), as well as ACRS letters dated March 13, 2003 (ML030730366) and February 13, 2002 (ML020500775), and the OIG report dated August 21, 2002 (Review of NRC's Significance Determination Process, OIG-02-A-15).

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Reactor Inspection and Performance Assessment program has achieved its performance goal of maintaining safety at operating power reactors as demonstrated through achieving the five annual performance measures for this goal. The key program partners are the four NRC regional offices, which commit to and achieve this goal through their regional operating plans. Measures reflecting industry performance also link external partners to the program. Systematic integration of licensee performance into ROP programmatic and resource decisions occurs routinely as described in MD 8.14 and as documented in the annual ROP self-assessment Commission Papers. Also as described in the annual Commission Papers, the staff has been succesful at identifying ways to contnually improve the program. Examples include pursuing improved performance indicators and continuing to revise the Signficance Determination Process to make it more effective. However, as previously discussed, appropriate targets for continuous improvement and cost effectiveness have not been able to be established.

Evidence: The program has met all of its performance goal measures for maintaining safety as demonstrated on page 37 of the FY2002 Performance and Accountability Report. Measures on industry performance are shown on pages 32 and 33 of the FY2002 Performance and Accountability Report. MD 8.14 describes the NRC's Agency Action Review Meeting Process. SECY-03-0062 and SECY-03-0057 document the NRC's most recent assessments of the ROP and the industry trends programs respectively. Commission paper SECY-03-0062 also describes the results of evaluating the program against its annual self-assessment performance measures to identify areas for continued improvement and increased effectiveness.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: NRC has taken several steps to continuously evaluate the Reactor Inspection and Performance Assessment program to identify ways to make it more cost efficient. Resource analyses conducted as part of the annual ROP self-assessment have demonstrated improved efficiencies and cost effectiveness in each of its three years of program implementation. The staff has identified methods to make inspection preparation and documentation more efficient, and has also identified certain inspection procedures that can be conducted together, possibly requiring less resources. These efficiency gains were recognized by identifying a 15 FTE resource savings that could be applied to the conduct of the Baseline Inspection program effective for FY 2004 and beyond.

Evidence: Commission paper SECY-03-0062 Attachment 7 "ROP Resource Analysis". FY2004 Budget Estimates and Performance Plan, page 52.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: During the development of the ROP inspection program, the NRC benchmarked the concepts with similar programs in the Environmental Protection Agency and the Federal Aviation Administration. The purpose was to glean insights into how these agencies incorporated risk into their inspection programs.

Evidence: Commission Paper SECY-99-007, Attachment 3, dated January 8, 1999

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Recent reports from the Advisory Committee on Reactor Safeguards and the Office of the Inspector General have indicated that the ROP is generally effective, though suggested improvements were noted. The Pilot Program Evaluation Panel and Initial Implementation Evaluation Panel also provided favorable results with some noted improvements. External stakeholders have responded favorably to the ROP as a significant improvement over the previous oversight programs.

Evidence: Reference SECY-03-0062, as well as ACRS letters dated March 13, 2003 (ML030730366) and February 13, 2002 (ML020500775), and the OIG report dated August 21, 2002.

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: The NRC Regulatory Impact Analysis (RIA) Guidelines state that the value-impact analyses must consider implementation of the regulation both upon affected entities and on the NRC. These RIAs are subject to public comment. The NRC Office of Nuclear Reactor Research has also conducted a number of 'look-back' studies on the regulatory effectiveness of particular regulations. Examples include: (1) Section 50.63- Loss of all alternating current (Station Blackout); (2) 50.62, Requirements for reduction of risk from anticipated transients without scram; and (3) Option B of Appendix J (Containment Leak Rate Testing). For each study, a draft version of the report was circulated for both internal NRC and external comment before finalization. Examples of this are noted under Evidence Section.

Evidence: Regulatory Analysis Guidelines of the U.S. Nuclear Regulatory Commission, NUREG/BR-0058, Rev. 3, July 2002; Station Blackout Study (ML003741781); Anticipated Transient Without Scram Study (ML011200001), and 10 CFR 50 Appendix J Study (ML023100201)