NATIONAL
ADVISORY COUNCIL September 13, 2002 The Performance Measurement Advisory Council (PMAC) was convened for its second meeting at 9:00 A.M. on September 13, 2002, at the White House Conference Center, Washington, D.C. Thomas Reilly, the Council’s Designated Federal Officer, opened the meeting along with Mortimer Downey, the Council’s Chair. In accordance
with the provisions of Public Law governing Federal Advisory Committees,
the meeting was open to the public.
Staff members of the Office of Management and Budget (OMB) attending were:
Other OMB representatives present for all or portions of the meeting were:
Mortimer Downey opened the meeting. He stated that the main focus of the meeting would be to offer advice and generate feedback on the program assessment exercise, so that this information can be processed and included in the 2004 Budget. • Discussion of Ratings Exercise Thomas Reilly provided a summary update for the Council on the ratings exercise. After considering advice from the Council and other stakeholders, such as the President’s Management Council (PMC), the Budget Director issued a memorandum to agency heads on July 16th that described the process. Included in this memo was a list of agency programs that will be assessed as a part of the 2004 Budget. OMB’s Program Effectiveness Team held a series of training sessions for agency and OMB staff designed to provide an overview of the exercise and ensure the consistent application of the Program Assessment Ratings Tool (PART). The Council noted that the definition of programs will affect presentation issues and the selection of relevant performance goals. Thomas Reilly responded that included in OMB’s guidance was a set of criteria for defining programs and selecting programs to be rated this year. Thomas Reilly described how OMB staff and agency program staff had been collaborating on completing the PARTS and reported that there is anecdotal evidence of excellent dialogue on performance measures. He also reviewed the proposed schedule for completing the PARTs and settling agency appeals. OMB has formed an Interagency Review Panel (IRP), which is comprised of agency representatives and OMB staff, to audit a sample of completed PARTs, ensure consistent application of guidance, and make recommendations on specific agency appeals. The IRP is scheduled to meet at the end of September and complete its work by October 7th. Council members inquired about the engagement of Congress. Joe Wright stressed the importance of working with Congress to institutionalize the review of program performance. He indicated the possibility of linking this effort to the existing Government Performance and Results Act framework, and then proposing ways to connect the GPRA format to the presentation of the Budget. This could be proposed as a refinement to GPRA. Harry Hatry suggested that OMB might ask the General Accounting Office (GAO) to obtain feedback on the assessment process. The Council asked Paul Posner of GAO to comment. He said that he would like this process to lead to congressional interest in performance evaluation and the current inventory of GAO analysis could be used in the assessment process. Council suggested that a review of the entire exercise should be conducted to refine the process for future years. This should include a survey of the agencies to obtain their feedback as to how the process was supporting their analytical and managerial needs. • Fall Review Thomas Reilly reported that OMB is currently considering how to reflect PART assessments in the Director’s Fall Budget Review. As a part of this process, OMB has developed a database which will facilitate the tracking of budget decisions to the program assessments. The Council offered several suggestions for future efforts, including focusing on measurements of cost effectiveness and equity in program results • Presentation Issues Reid Cramer reported that the Budget Director explained in his July 16th memo that final decisions regarding the presentation of PART results in the FY 2004 Budget have not been made. Further, the Director stated that presentation options would be discussed by the Performance Measurement Advisory Council (PMAC) at its September meeting. The Council discussed a range of presentation options and considered their advantages and disadvantages in the context of the Budget documents. Several issues were highlighted. Issue 1: Transparency. How should performance information be made available? OMB staff reported that they intended to recommend that at a minimum, the name of each program assessed with the tool should be identified, along with section scores, and an overall effectiveness rating should be included in the Budget. The Council agreed with this recommendation. OMB staff reported that they are proceeding with the assumption that the PART worksheets for each program will be accessible to the public on an OMB website. The Council agreed with this approach to transparency but several members were concerned that the manner in which the summary assessments were presented would greatly influence the public’s perception of the initiative. The Council recognizes that there are many places within the four-volume budget where PART related information can be presented, such as in Analytical Perspectives or in a summary chapter on the President’s Management Agenda. While the Council recognized the time constraints for this preparing the 2004 Budget, they strongly recommended pursuing web-based presentation alternatives for future years. OMB should explore ways to use technology to reduce the size of the printed Budget and develop a flexible database that could be useful to many different types of stakeholders Issue 2: Format. How should the PART score be converted into program assessments? Effectively communicating program assessments for such a large number of programs places a premium on simplicity, distinguishing performance among programs, and ability to measure improvements over time. The Council recommended that the numeric scores of the PART and the section weights should be disclosed somewhere in the Budget but this information was not necessary to include in a summary table in each department chapter. Summary tables could use verbal assessments, such as “effective,” “moderately effective,” “minimally effective,” and “ineffective”. Issue 3: Presentation. What information from the PART should be shown in the Budget and where should it be shown? The Council recommended that a summary table be included in each departmental budget chapter with the name of assessed programs, overall assessments, explanations of the assessments, and descriptions of the budget recommendation or policy proposal for each program. They recommended that this table be supported by a summary report for each assessed program in another portion of the Budget. This summary report would include additional information derived from the assessment including key performance measures and results. Several members of the Council recommended that OMB consider additional alternatives in the future. Harry Hatry asked that OMB continue to consider ways to highlight the results score from the PART as distinct from the overall assessment. For example, the overall effectiveness rating could be based on the results score and be relabeled as an “Outcome Rating.” The first three sections could be combined into one “program management” rating and used as a part of the explanation of program results. Hatry noted that if OMB does not take this suggestion, it should footnote on each program rating sheet, that the overall rating is based on the weighted average of all four sections and identify the weights used. Joe Wright recommended that program assessments include a more explicit focus on cost effectiveness in addition to results and program purpose. Planning and management will feed into results and therefore do not need to be separated out. Council members were concerned with how the presentation of program assessments would compare to the presentation of the scorecard for the President’s Management Agenda. The stoplight format of the scorecard attracts attention and even though these assessments are made at the department level, while the PART assessments are performed at the program level, the Budget must communicate how they are connected. • Appeal and Audit Process Thomas Reilly described the appeal and audit process that has been outlined in the Budget Procedures Memo No. 855. Central to this process is the formation of an 8 member interagency Review Panel that will include OMB and agency staff. • Common Measures Marcus Peacock described the current state of the common measures exercise. Council members discussed ways to expand and institutionalize the effort, such as developing benchmarks from the private sector, focusing on efficiency measures, and controlling for divergent quality in programs that have diverse constituencies. • Next Steps The next meeting of the PMAC is not scheduled. It may be held in February. Potential agenda items include agency presentations on their experience with PART, examination of assessment process, refinements for the 2005 Budget. ADJOURNMENT The meeting adjourned at 1:45 P.M. I hereby certify that, to the best of my knowledge, the foregoing minutes are accurate and complete. Thomas Reilly, Designated Federal Officer The Council at its next meeting will formally consider these minutes, and any corrections or notations will be incorporated in the minutes.
|
 |
|||
President's Budget
Management
Information &
Regulatory Affairs
Legislative Information
Agency Information
|
|
||
 | |||
|
Federal Register
| Jobs at OMB | FOIA | No FEAR | OMB Locator | USA.gov | Accessibility
| Privacy Policy
| Site Search
| Help
|