ExpectMore.gov


Detailed Information on the
International Information Programs Assessment

Program Code 10004622
Program Title International Information Programs
Department Name Department of State
Agency/Bureau Name Other
Program Type(s) Direct Federal Program
Assessment Year 2006
Assessment Rating Adequate
Assessment Section Scores
Section Score
Program Purpose & Design 80%
Strategic Planning 62%
Program Management 57%
Program Results/Accountability 33%
Program Funding Level
(in millions)
FY2007 $50
FY2008 $55
FY2009 $59

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2007

Create a more comprehensive system of identifying target audiences and ensuring that products and services reach target groups and have intended effects.

Action taken, but not completed As part of the Public Diplomacy and Strategic Communication Policy Coordination Committee, IIP will be participating in in-depth analyses of country audience data, starting with Afghanistan and Nigeria. Data from multiple sources will be consolidated for an additional 20 countries. This data will greatly augment the current program audience data collected by the Mission Activity Tracker (MAT) database.
2007

Develop an enhanced system for measuring electronic outreach.

Action taken, but not completed IIP has developed a measurement team across multiple offices that seeks to consider how best to combine traditional web metrics (visits/views), Web 2.0 issues (interactivity) as well as satisfaction scores. In the absence of industry standards, the team is seeking to implement a calculated scoring system that provides meaningful measures with the intent of understanding usage patterns, Search Engine Optimization, measuring marketing efforts and consideration of content changes.
2007

Coordinate more closely with regional counterparts in the Department to plan programming and outreach efforts.

Action taken, but not completed As part of its reorganization, IIP created the role of regional policy officer to enhance coordination on regional issues within the Department and the field. New initiatives include: 1) development of policy and guidance materials to be posted on the INFOCENTRAL site for use by PD practitioners. 2) monthly consultations on priority themes and issues to be addressed in IIP??s products. 3) establishing a regular PD role in crisis management in cooperation with the Operations Center.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2007

Complete development of performance measure framework. Obtain data for baselines and develop ambitious targets.

Completed In coordination with the Public Diplomacy Evaluation office, IIP has developed a comprehensive strategy for performance measurement, with baselines and targets in place for seven of our eight measures, and data collection beginning on the eighth.
2007

Provide resource requests with direct links between funding levels and performance objectives and goals.

Completed The Department has institutionalized the process of including performance measures in its annual budget requests. As part of the FY 2009 Congressional Budget Justification, PART indicators have been added to show direct linkage between Bureau performance in reducing anti-American sentiment and improving understanding and the requested funding. This is part of the Department??s effort to achieve full budget-performance integration.

Program Performance Measures

Term Type  
Annual Efficiency

Measure: Operational Support Costs as a Percentage of Total Costs


Explanation:Reveals the efficiency of the Bureau. A lower percentage indicates that more funds are spent on programs and fewer on administrative overhead. Operational Support includes: costs of the Coordinator and Deputy Coordinator to direct and coordinate Bureau offices toward efficient and effective fulfillment of IIP's mission; the Office of Policy, Planning, and Evaluation; and 27 positions for administrative management of IIP programs, from budget oversight to personnel services.

Year Target Actual
2004 11.4% 11.4%
2005 11.15% 10.6
2006 10.9% 9%
2007 10.65% 10.6%
2008 10.4% 10.6% to date
2009 10.2
2010 10.1
2011 10
Long-term Outcome

Measure: Initiation or implementation of positive change in local organizations or communities by IIP audience.


Explanation:IIP's mission is to engage, inform, and influence foreign audiences. This measure reflects the extent that the participants take concrete actions that apply knowledge gained from IIP activities to local institutions, community groups, government, or civil society organizations - a behavioral "positive change" that indicates the influence IIP's program have on their target audiences. Because this is a measure of long-term influence, increases of one percent should be considered ambitious. The Performance Measurement Data Collection Project (PMDCP) collected data from 1,858 members of key foreign audiences worldwide. The center of the study design is a comparative analysis of survey responses from 541 participants in PD programs and 537 non-participants. PMDCP included 21 focus groups involving 165 members of foreign audiences, and event feedback surveys from 19 events from 615 individuals. The data reflects the percentage of participants who indicated in a survey or focus group that they applied information gained from an IIP program to affect a positive change in their local community or organization. No new data is expected until the end of FY09 due to budget timing and the complexities of collecting and analyzing data from multiple regions, countries and languages.

Year Target Actual
2007 Establish baseline 66%
2008 67% Available in FY09
2009 68%
2010 69%
2011 70%
2012 71%
Long-term Outcome

Measure: Percentage of key audience members with a better understanding of U.S. policy, society and values as a result of IIP products/programs.


Explanation:IIP's mission is to engage, inform, and influence foreign audiences. This measure reflects the long-term impact of IIP programs in improving foreign publics understanding and appreciation of the United States' policy, society and values, and is key to measuring how well IIP programs are performing the mission of engaging and informing key target audiences. Building on the shorter term measure regarding topic specific understanding, this long-term measure captures improvements in audience understanding of broad policies, society, and values. Because this is a measure of long-term changes in understanding, increases of one percent should be considered ambitious. The Performance Measurement Data Collection Project (PMDCP) collected data from 1,858 members of key foreign audiences worldwide. The center of the study design is a comparative analysis of survey responses from 541 participants in PD programs and 537 non-participants. PMDCP included 21 focus groups involving 165 members of foreign audiences, and event feedback surveys from 19 events from 615 individuals. The data reflects the percentage of participants who indicated in a survey or focus group that they gained a better understanding of broad US policies, society, and values as a result of IIP programs and products. No new data is expected until the end of FY09 due to budget timing and the complexities of collecting and analyzing data from multiple regions, countries and languages.

Year Target Actual
2007 Establish baseline 83%
2008 84% Available in FY09
2009 85%
2010 86%
2011 87%
2012 88%
Annual Outcome

Measure: Percentage of key audience members with a better understanding of specific U.S. policy, society and values after using/attending IIP products/programs on that subject.


Explanation:This measure captures the audience's improved understanding of a specific topic or subject that was the focus of an IIP program or product, which is directly related to IIP's mission to engage, inform and influence foreign audiences. It measures annual improvements in awareness of a particular issue, position, policy, or value. IIP designs programs and products to have a cumulative impact in raising the level of understanding concerning broad U.S. policy, society, and values. As there is only baseline data available at this time, the targets may be adjusted upward after further data is received. The Performance Measurement Data Collection Project (PMDCP) collected data from 1,858 members of key foreign audiences worldwide. The center of the study design is a comparative analysis of survey responses from 541 participants in PD programs and 537 non-participants. PMDCP included 21 focus groups involving 165 members of foreign audiences, and event feedback surveys from 19 events from 615 individuals. The data reflects the percentage of participants who indicated in a survey or focus group that they gained a better understanding of a specific issue, position, policy, or value as a result of an IIP program and/or product. No new data is expected until the end of FY09 due to budget timing and the complexities of collecting and analyzing data from multiple regions, countries and languages.

Year Target Actual
2006 Under development 55% (anticipated)
2007 Establish baseline 79%
2008 81% Available in FY09
2009 83%
2010 85%
2011 87%
Annual Outcome

Measure: User satisfaction scores for America.gov


Explanation:In January, IIP introduced America.gov, a new website that serves as the Department's primary website to reach foreign audiences. America.gov offers the latest in multimedia, video, forum discussions, polls, articles and quizzes. Because this is a new website, new baselines and targets need to be established. The level of satisfaction of the users of America.gov will be measured using the American Customer Satisfaction Index, an independently validated and verified model that surveys users' experience with the website.

Year Target Actual
2005 baseline 71.9%
2006 71.9% 69%
2007 71.9% 72%
2008 Estab. new baseline Under development
2009 Establish new target
2010 Establish new target
2011 Establish new target
Annual Outcome

Measure: User satisfaction scores for IIP publications


Explanation:Survey data shows the level of reader satisfaction with IIP publications. The Performance Measurement Data Collection Project (PMDCP) collected data from 1,858 members of key foreign audiences worldwide. The center of the study design is a comparative analysis of survey responses from 541 participants in PD programs and 537 non-participants. PMDCP included 21 focus groups involving 165 members of foreign audiences, and event feedback surveys from 19 events from 615 individuals. The data reflects the percentage of participants who indicated in a survey or focus group that they were satisfied with IIP's print products. No new data is expected until the end of FY09 due to budget timing and the complexities of collecting and analyzing data from multiple regions, countries and languages.

Year Target Actual
2007 Establish baseline 61%
2008 63% Available in FY09
2009 65%
2010 67%
2011 69%
Annual Output

Measure: Audience reached through www.America.gov


Explanation:In January IIP introduced America.gov, a new website that serves as the Department's primary website to reach foreign audiences. America.gov offers the latest in multimedia, video, forum discussions, polls, articles and quizzes. Because this is a new website new baselines and targets will need to be established. IIP has developed a measurement team across multiple offices that seeks to consider how best to combine traditional web metrics (visits/views), Web 2.0 issues (interactivity) as well as satisfaction scores. In the absence of industry standards, the team is seeking to implement a calculated scoring system that provides meaningful measures. Until this new scoring system is fully in place, IIP will continue to track page views.

Year Target Actual
2005 baseline 25,000,000 pageviews
2006 4% increase 52,679,990 (110%)
2007 4% increase 57,319,981 (8.8%)
2008 Estab. new baseline Under development
2009 Establish new target
2010 Establish new target
2011 Establish new target
Annual Output

Measure: The number of key audience members attending IIP speaker programs.


Explanation:IIP's mission is to engage, inform, and influence foreign audiences. This measure shows the number of people reached through IIP's speaker program, both in person and via digital video conference. IIP continually works to make more programs available to the field to reach our key audiences. Data for this measure is collected through the Department's Mission Activity Tracker (MAT). MAT gathers information from posts on their public diplomacy activities, the size and general demographics of audiences reached and any results that have been achieved.

Year Target Actual
2007 Avail. spring 2008 Avail. spring 2008
2008 Establish baseline Avg. 7,869 per month
2009 5% increase
2010 5% increase
2011 5% increase

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The purpose of the Bureau of International Information Programs is to engage, inform, and influence international audiences about U.S. policy and society to help create an international environment receptive to America's interests as stated in the Bureau's Mission and Vision Statements and consistent with its authorizing legislation.

Evidence: 1. IIP Mission and Vision Statements 2. State Department Basic Authorities Act of 1956 (22 USC 2651 et seq.), page 73. 3. Foreign Affairs Manual 1, page 1 4. Foreign Affairs Manual 2, pages 15 and 16

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: The International Information Program (IIP) provides the official U.S. position on policy and critical interests for all U.S. Missions and also for direct international audiences. The program is especially critical as public opinion gains greater influence on government action around the world and in light of the ongoing challenge of anti-Americanism and the threat it poses to U.S. strategic interests. IIP products and services focus on ensuring that the U.S. perspective and policy is fully available for public discourse. Toward these ends, IIP increases understanding of U.S. policy society and values and creates an atmosphere more receptive to America's interest through services and products like the U.S. speakers and specialists, websites, and electronic and print publications.

Evidence: 5. Pew Report on Global Attitudes, June 13, 2006, pages 1-5 6. Report of the Defense Science Board on Strategic Communication, September 2004, pages 11 and 19 7. Advisory Group on Public Diplomacy for the Arab and Muslim World, Changing Minds, Winning Peace ("Djerejian Report") page 16

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While numerous groups reach out to international audiences, the Department of State has the sole mandate to lead the U.S. Government effort to articulate foreign policy objectives of the United States to foreign audiences. IIP's role is unique because it provides global audiences with comprehensive guidance and background on the full scope and range of U.S. policy concerns. To support this effort, IIP provides contextual information about U.S. society and values by disseminating authoritative texts and offering expert interpretations. IIP delivers America's message both through U.S. missions abroad and directly to international media, government officials, opinion leaders, and the public in more than 140 countries and in seven languages.

Evidence: 8. Memo: Establishment of the Public Diplomacy and Strategic Communications Policy Coordinating Committee 3. Foreign Affairs Manual 1 4. Foreign Affairs Manual 2

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Public Diplomacy and IIP have been examined recently in various reports and by an inspection in 2004 by the State Department Office of the Inspector General. These reports do not challenge the major elements of the current program nor do the report provide conclusive evidence that an alternative approach would be more efficient or effective. In order to continuously improve the effectiveness of the program, IIP evaluates different services and products of its programs on an ongoing basis. Recent reviews have been conducted on products such as Hi magazine. In response to the findings of that review the magazine was cancelled. From a management perspective, IIP is reviewing how to realign different units in the bureau to increase overall effectiveness, efficiency and accountability. Performance measure frameworks are reviewed and updated on a yearly basis as part of the strategic planning process.

Evidence: 9. Office of Inspector General Report of Inspection: Bureau of International Information Programs Report Number ISP-I-04-31, July 2004 10. OIG Compliance Review Memo 11. "Public Diplomacy: A Strategy For Reform, A Report of an Independent Task Force on Public Diplomacy Sponsored by the Council on Foreign Relations, July 2002"

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Currently, IIP does not have mechanisms in place that ensure that resources are being used directly to reach target audiences. To address this problem, the public diplomacy program as a whole, with IIP as a core partner, have begun to survey public affairs officers and chief of mission to ensure that the right beneficiaries are being targeted. The next step to ensure that program resources are effectively targeted will be to study how well resources are linked with the targeted beneficiaries and outcomes. Institutionalizing both the evaluation of target audiences and the linkage between resources and intended beneficiaries will be essential to ensure effective targeting in the long term.

Evidence: 12. Congressional Budget Justification document13. FY2007 Bureau Performance Plan14. BPP and MPP process document15. I-Bucks policy statement

NO 0%
Section 1 - Program Purpose & Design Score 80%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: IIP has two long-term outcome measures that meaningfully capture the long term success in informing and influencing target audiences and foreign publics. More specifically. The program purpose to engage, inform and influence are measured by the following: 1) Percentage of key audience members with a better understanding of U.S. policy, society and values after engagement with IIP products and programs 2) The number of IIP audience participants that initiate or implement positive change in local organization or communities The first outcome indicator builds directly on the progress of shorter term IIP efforts to engage foreign publics and target audiences. Improved understanding over the long term reflects a well informed audience that is more receptive environment for U.S. policies and positions. The second measure reflects the ability of the program to influence participants to the extent that they take concrete actions to directly apply knowledge gained from an IIP product or program. The change that can be directed toward positive improvements to local institutions, community groups, government, or civil society organizations. These measures were developed through a performance measure framework (logic model) that links annual outputs such as improved reach and satisfaction to outcomes like understanding and behavior change.

Evidence: IIP Performance Measures IIP logic model PD logic model

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Because both long term outcome measures have been recently developed, data does not currently exist to establish baselines and targets. IIP will be working with the new Public Diplomacy Evaluation Office to design data collection systems to support IIP's performance measurement framework.

Evidence: IIP Performance Measures

NO 0%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: IIP has six annual measures that track progress toward achieving long-term goals. Two annual output indicators are included to reflect the growth in outreach for IIP websites and speakers programs. Expanding the reach of the program through electronic and events is critical to achieving the annual and long term outcome indicators. Three annual outcome indicators are included to measure the satisfaction of participants and the short term impact of the engagement. Specifically, satisfaction measures for websites and publications reflect the quality of engagement with the audience as well as provide an indication of the likelihood of continued engagement in the future. The annual measure for audience understanding reflects the increase in participants understanding of the topic or subject of a particular program or publication. This links directly toward building the deeper and more broad understanding of U.S. policy, society, and values that is reflected in the long term. The final measure reflects the efficiency of the program to channel resources to program efforts rather than administrative and overhead costs. These measures were developed through a performance measure framework (logic model) that links annual outputs such as improved reach and satisfaction to outcomes like understanding and behavior change.

Evidence: IIP Performance Measures, IIP Logic model.

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: IIP has baseline data for three of the six performance measures. Because three of the measures are new, baselines and targets will be forthcoming. IIP will be working with the new Public Diplomacy Evaluation Office to institutionalize data collection systems so that comprehensive program performance data will be available strategic planning and management purposes in the near future.

Evidence: IIP Performance Measures

NO 0%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: IIP's principal partners are the DOS regional and functional bureaus and U.S. missions abroad. In the Department's strategic planning process, the bureau, posts, and regional bureaus commit to goals, target audiences, the means by which they will be reached, and the resources that will be committed to each task. The Bureau holds posts financially responsible for reporting on progress toward performance goals, and reviews annual performance when allocating bureau funds for posts' use of IIP programs, products and services.

Evidence: I-Bucks Policy Statement FY07 Bureau Performance Plan

YES 12%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department of State Inspector General evaluated IIP in 2004 and will review the speaker program in FY07. IIP program evaluations include the independent evaluation of the American Corners program by ORC Macro, which is in the data collection phase, the recently completed assessment of Hi magazine, and the ongoing the American Customer Satisfaction Index assessment. An evaluation of the Sabre book program began in FY06, and an evaluation of the speaker program begins in FY07. IIP's evaluation unit has developed a monitoring and evaluation (M & E) program. The M & E program includes criteria for selecting independent evaluators and a multi-year program evaluation schedule of independent evaluations of all IIP programs. The evaluation schedule ensures that at least one of IIP's major programs is independently evaluated each year. Independent professional evaluators, selected through a competitive process, do evaluations. Methodology for all evaluations (contracted or through the PD Evaluation Office) is reviewed for quality and appropriateness by a team of professional evaluators within the evaluation office. Quality control mechanisms ensure that evaluation methods include quasi-experimental design (comparison groups), pre and post surveying, and multi-site data collection in order to validate the data and assess program outcomes. Statistical sampling is also used to increase validity in the evaluations. Performance measurement assessments are core components of each evaluation. Data gathered from the evaluations is used to make program decisions, such as the discontinuation of Hi Magazine.

Evidence: American Corners evaluation documents Evaluation Schedule American Customer Satisfaction Index (ACSI) report IIP Monitoring and Evaluation Program document

YES 12%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: International Information Programs currently do not explicitly tie direct and indirect resource requests to the accomplishment of annual and long-term performance goals. More specifically, budget requests to OMB and Congress do not directly link resource levels to annual or long-term outcome targets identified in the public diplomacy performance measure framework. As a first step toward addressing this issue, IIP has begun developing a new budget development process that more clearly links resource requests to achievement of strategic goals. This bureau-based initiative will become the basis of the Bureau Performance Plan (BPP) that contains long-term goals for achieving the bureau's program priorities, and the budget request. In future years, the Congressional Budget Justification (CBJ) should provide the total funding request, accomplishments and priorities and funding by program activities.

Evidence: FY08 Budget Proposal documents FY07 Senior Review PowerPoint

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Recognizing the core need to tie its budget more closely to its goals, IIP staff created and begun implementing a new budget development process. The new process requires that each office create a plan that breaks down their current budget, shows how resources are used to create outputs, outcomes and achieve strategic goals, then uses that information to form its budget requests. This process will ultimately lead to budget information that is presented in a more transparent manner, greater accountability and a stronger linkage between budget and performance.

Evidence: FY08 Budget Proposal documents FY07 Senior Review PowerPoint

YES 12%
Section 2 - Strategic Planning Score 62%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program has just established a comprehensive performance measurement framework, so baseline performance data and targets are not yet available. As a result, it is not currently possible for the program program to integrate performance data and targets in decision making regarding the adjustment of priorities or resource allocations.

Evidence: American Customer Satisfaction Index (ACSI) report American Corners evaluation documents Hi Magazine Assessment Tracker reports Webtrends reports Memo on Washington File changes

NO 0%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The job requirements of each manager outline strict budget management standards he/she is expected to meet. All IIP office managers are required to write a yearly office performance plan outlining goals and the steps they will take to meet them. Managers brief the Bureau coordinator (Assistant Secretary equivalent) every week on the status of their programs. They are also required to submit weekly written reports to the Coordinator on the status of their programs, progress toward goals, corrective measures if necessary, innovations, challenges and successes. IIP requires that program partners report on their use of IIP programs and progress toward performance goals. For example, posts that do not report adequately on IIP speaker programs will have funding for future programs withheld and/or reduced until they have met their reporting requirements.

Evidence: Office Directors' Weekly Reports I-Bucks policy statement

YES 14%
3.3

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: IIP is funded from the Diplomatic and Consular Programs appropriation (Point P Limitation) with budget authority that is available for obligation during the current fiscal year expires, if not obligated at the end of that time. A financial plan is developed at the beginning of the fiscal year and approved by the Bureau's Coordinator. This plan is executed throughout the year. IIP uses the Department's Central Financial Management System (CFMS) and IIP's internal Financial Management Information System (FMIS) to provide managers with monthly status of funds reports. All funds (100%) allocated to the Bureau are obligated by the end of each fiscal year.

Evidence: Allotment reports

YES 14%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: IIP tracks its increasing efficiency using an efficiency measure of operational costs as a percentage of total costs. IIP has implemented several other programs to improve the efficiency and cost effectiveness of public diplomacy (PD) programs, including the development of the INFOCENTRAL and Best Practices websites and the International Events Calendar, which have improved the ability of public diplomacy practitioners to find critical information in a timely manner. IIP is leading the way with IT enhancements in the State Department that improve program efficiency, including the adoption of CMS for IIP and post websites, and the transition to OpenNet Plus that led to enhancements to the OpenNet Plus network. An active participant in the DOS Competition Council, IIP is developing a streamlined competition study of graphic design functions in FY 05.

Evidence: Efficiency Measure Quarterly Management Reports

YES 14%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: There is no strong evidence that interagency or private collaboration has led to meaningful resource allocation decisions. For instance while IIP staff regularly participate in the coordination meetings and interagency teams (such as the Fusion Team), there is no clear evidence that coordination efforts have led to meaningful resource allocation decisions or devleopment of joint efforts with integrated performance and resource plans.

Evidence: INFOCENTRAL screen prints PDEO memo

NO 0%
3.6

Does the program use strong financial management practices?

Explanation: The Bureau obligates and expends funds only for the purposes authorized in governing legislation. Appropriate reprogramming authority is requested before any funds are reprogrammed for other purposes. The Bureau uses a cuff record system to check obligations against commitments. The Bureau also uses the Ariba buying system for all purchase orders and contracts. The 2003 OIG Reports for IIP found no material weaknesses with respect to financial management.

Evidence: Allotment reports

YES 14%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: IIP has not established and institutionalized the management systems necessary to improve accountability, responsiveness, and ultimately performance. To address the need for greater accountability of managers, IIP has reviewed its current structure and is realigning its offices and program areas. The realignment will bring the bureau's core programs, electronic outreach, field support programs, and print material into more coherent units within a clear chain of command. This realignment also establishes a new planning office to support the Department's long-range public diplomacy strategy, as well as IIP's participation in evaluation and BPP processes.

Evidence: Memo to Under Secretary Hughes on restructuring

NO 0%
Section 3 - Program Management Score 57%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Because the two long term measures are new, comprehensive baseline data and targets are not currently available. Initial data from a survey of American Corners in one region indicate that progress is being made toward one of the outcome measures.

Evidence: American Corners evaluation documents

NO 0%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: For the three annual goals where data is available, IIP is clearly on track to achieving its goals. However, since data is available for three of the six measures, the program can only clearly demonstrate that a small extent of progress is being made toward annual performance goals. Evidence that IIP is on track to meet its annual goals comes from Webtrends reporting of website performance, website monitoring and research conducted by program staff using independent tools (such as Alexa and Google), other usage and attendance information, and qualitative evidence from RESULTS and the U.S. Speaker Tracker system. Evaluations that began in FY05 and planned FY06 and FY07 evaluations will provide data on new measures in the near future.

Evidence: Measures data

SMALL EXTENT 7%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: For Fiscal Year 2006, IIP is on track to exceed its goal for improving efficiency. Current data shows IIP with operational support costs at nine percent of total costs. IIP has efficiency measure - operational support costs as a percentage of total costs and is meeting its efficiency goals. IIP is working to improve efficiency in all areas by focusing its resources on program priorities and program performance.

Evidence: Efficiency Measure Quarterly Management Reports

LARGE EXTENT 13%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While numerous studies of effectiveness and impact of IIP have been undertaken, none have executed a direct comparison between IIP products and programs within a governemnt context. For example, a recent reoprt by the United Kingdom is not directly comparable to IIP because the UK public diplomacy program includes media (such as radio) that is not handled by IIP the U.S. public diplomacy which can cause distortion in the interpretation.

Evidence: Report on UK public diplomacy

SMALL EXTENT 7%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The most recent independent evaluations of the IIP program are not at a sufficient scope to clearly indicate whether the overall program is effective and achieving result. A recent independent evaluation of the Hi Magazine concluded the the program was not effective enough to justify its resource level. However, while an evaluation of the American Corners program is still in the data collection phase, more positive results regarding overall effectiveness are expected. From a broad perspective, the Office of the Inspector General has indicated that IIP is effective and achieving results. Data from the American Customer Satisfaction Index indicates that the program is on target and is achieving results, as did the 2004 OIG Report that evaluated the performance of IIP. Numerous evaluations of specific IIP programs, including OMB's "all green" evaluation of the transition to CMS, the Foresee survey of web users and a survey of posts' use of IIP products and services have found IIP's programs to be effective and achieving results. IIP is in the process of enhancing its evaluation capabilities and has developed and implemented a rigorous monitoring and evaluation program for the Bureau, including regular independent evaluations of IIP programs through evaluation contractors. These evaluations will ensure the comprehensive evaluation of all IIP programs on a regular basis.

Evidence: American Corners evaluation documents Evaluation Schedule American Customer Satisfaction Index (ACSI) report Office of Inspector General Report of Inspection: Bureau of International Information Programs Report Number ISP-I-04-31, July 2004 Control Review Scorecard: OMB Evaluation of IIP's Content Management System

SMALL EXTENT 7%
Section 4 - Program Results/Accountability Score 33%


Last updated: 09062008.2006SPR