Detailed Information on the
United State Patent and Trademark Office - Intellectual Property Protection Activities Assessment

OIPPE will contract with an independent organization to conduct an evaluation on the long-term outcomes and impact OIPPE assistance has on Intellectual Property Protection and Enforcement.

Increase the number of countries that achieve 75 percent of actions as part of their joint agreements to improve intellectual property protections.

Measure: Percentage of countries on the USTR 301 list or awaiting WTO ascension that have positively amended or improved their IP systems

Explanation:Measures the results of OIPPE engagement to moving toward improving IP systems worldwide. To move off the 301 list or to gain WTO membership, countries must have IP protection systems in place that meet international standards. "Positively amended" denotes changes that are not harmful to U.S. interests. Amendments and improvements include actions such as new and improved IPR laws and newly acceded WTO members such as Saudi Arabia.

Measure: Percent of foreign officials trained by GIPA who have initiated or implemented a positive IP change in their organizations

Explanation:Measure is an aggregate of multiple projects. Positive change includes items such as changing policy, creating new law or treaty, revising enforcement mandates, in a way that protects IP, while not harming U.S. business interests. Data gathered from follow-on surveying of participants. Goal is to reach 75% in five years.

Measure: Number of countries that implement at least 75% of action steps which improve IP protections in their joint cooperation, action or work plans.

Explanation:Measures the progress of countries in implementing the joint action plans. Measurement taken every two years. In 2008, OIPPE will establish its baseline data. By 2014, OIPPE's goal is to have 10 countries achieve 75% of their action steps. As more countries implement their action plans, intellectual property protections around the globe will be strengthened.

Measure: Number of Memoranda of Agreement for IP joint cooperation, plans of action, mechanisms and support programs initiated or implemented by developing countries as a result of OIPPE.

Explanation:Measures the scope of results of OIPPE engagement toward the development of improved IP systems. The action plans will lead to results in detailed plans for training, technical assistance, capacity building and cooperation. This measure was first introduced in FY 2005 to demonstrate the number and variety of training and technical assistance activities provided to the intellectual property offices and staff of countries with developing economies in need of strengthening the protection of intellectual property rights as part of their economic and trade development. Attorney specialists from the Office of International Relations and the Office of Enforcement provide country-specific review of intellectual property laws, and recommend strengthened enforcement provisions along with training of judges, prosecutors, customs officials, and intellectual property office technical staff.

Measure: Percent of foreign officials trained by OIPPE with increased expertise in IP

Explanation:Measures the scope of those reached by GIPA who master learning of critical IP topic areas. The long-term target is to achieve 97% (because of + 3% margin of error in sampling) within five years and to then to maintain a minimum of 95% per year. This high percentage is also a proxy indicator that OIPPE is selecting the right current and emerging IP leaders.

Measure: Administrative costs as a percentage of total costs.

Explanation:Measures OIPPE's administration costs to the total program cost. This measure focuses on the relationship between spending on administration and the effectiveness of the organization in carrying out its mission. By decreasing administrative costs' share of total program costs, OIPPE will be able to use more resources on program activities.

Is the program purpose clear?

Explanation: The mission of the Office of External Affairs, comprised of the Office of Intellectual Property Policy and Enforcement (OIPPE) and the Office of Governmental Affairs, is to improve intellectual property protection and enforcement domestically and abroad by delivering intellectual property information and education worldwide and guiding domestic and intellectual property policy. It supports and advances this mission by: (1) working with Congress on major legislation to reform current patent law and practice, to coordinate informational and educational events on piracy and counterfeiting, and to implement international intellectual property treaties; (2) providing technical assistance to foreign governments that are looking to develop or improve their intellectual property laws and systems; (3) providing capacity-building training programs to foreign intellectual property officials on intellectual property enforcement; (4) advising the Department of State and the United States Trade Representative (USTR) on drafting, reviewing and negotiating of intellectual property sections in bilateral and multilateral investment treaties and trade agreements; (5) advising the USTR on intellectual property issues in the World Trade Organization (WTO); and (6) examining the adequacy and effectiveness of intellectual property rights protection under the Special 301 provisions of the Trade Act of 1974.

Evidence: Legislative mandate: 35 USC Section 2 (b) 8-13: http://www.uspto.gov/web/offices/pac/mpep/documents/appxl_35_U_S_C_2.htm) 2007-2012 Strategic Plan: http://www.uspto.gov/web/offices/com/strat2007/stratplan2007-2012.pdf; page 23) 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 24) 2008 USPTO Budget Presentation: http://www.uspto.gov/web/offices/ac/comp/budg/fy08pbr.pdf (pages 32-36) OIPPE Website: http://www.uspto.gov/web/offices/dcom/olia USPTO Policy Making PowerPoint (Internal document) Technical Assistance Overview (internal document)

Does the program address a specific and existing problem, interest, or need?

Explanation: OIPPE addresses the interest of encouraging American innovation and strengthening the nation's ability to compete in the global economy by tackling the complex problems of counterfeiting, piracy, and the lack of simplified and harmonized international IP systems. OIPPE responds to these problems by providing IP advice and technical assistance to U.S. Government (USG) agencies, and other countries through reviewing policies, providing training to Intellectual Property (IP) officials, making U.S. businesses more aware of IP issues, supporting international IP organizations, and having IP experts stationed around the world in high priority countries. The interest and the problems continue to grow and are still relevant requiring the attention and activities of OIPPE. IP represent 40% of U.S. economic growth and employ 18 million Americans -- who earn 40% more than the average U.S. wage. (2006 NIPLECC Report, page. 21). Copyright industries add nearly $820 billion (6.5 %) to the U.S. GDP (Copyright Industries Report, page 2). The scope of the problems are also great. Copyright piracy costs the U.S. economy up to $250 billion and 750,000 U.S. jobs every year. The estimated value of counterfeit and pirated goods seized increased in 2007 by approximately 27%, up to 200 million (2008 NIPLECC Report, pages 28-29). U.S. Copyright industries estimate that in 2006, 85 -93 percent of all copy righted material sold in China was pirated. The number of counterfeit product seizures from China at the U.S. border increased by 81%. (Special 301 Report Pg. 18). In Russia, U.S. copyright industries estimate they lost in excess of $2.1 billion due to piracy (Special 301 Report, page 23).

Evidence: Report to the President and Congress on Coordination of Intellectual Property Enforcement and Protection, September 2006, National Intellectual Property Law Enforcement Cooperation Council, page. 21. URL: http://www.commerce.gov/opa/press/Secretary_Gutierrez/2006_Releases/September/2006%20IP%20report.pdf 2008 NIPLECC Report: http://www.stopfakes.gov/pdf/2008_NIPLECC_Report_and_Appendices_Final.pdf Special 301 Report: Url: http://www.ustr.gov/Document_Library/Reports_Publications/2007/2007_Special_301_Review/Section_Index.html (page 6) 2006 Copyright Industries report: Url: http://www.iipa.com/pdf/2006_siwek_full.pdf President's Economic Report: http://www.gpoaccess.gov/eop/index.html (pages 79-95) February 12, 2008 Daily News Clip (Internal document) Published Articles Recorded Music piracy cost the U.S. $12.5 billion in economic output; 71,060 jobs ($2.7 billion in lost earnings); and $422 million in tax revenue annually. According to the Institute for Policy Innovation (IPI), Intellectual property products, such as sound recordings, are responsible for nearly 40% of economic growth and nearly 60% of growth in U.S. exports. Billboard.biz - Report: Piracy Has Cost U.S. $12.5 Billion Motion picture piracy costs U.S. workers $5.50 billion in lost annual earnings and 141,030 in jobs lost; $ 837 million in lost annual tax revenue; and $20.5 billion in lost annual output to all U.S. industries. Motion Picture Association of America - Motion Picture Piracy Costs U.S. Economy Thousands of Jobs, Billions in Lost Wages Taipei Times - U.S. Business Lobby Targets Piracy in India and China

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: OIPPE is the only agency charged by statute with providing policy advice to the President and other USG agencies on intellectual property matters. There are no other federal, state, local or private entities that serve as the technical lead on intellectual property policy, treaty and trade negotiations, or that provide basic technical assistance to other countries on IP matters. In conducting programs, OIPPE leverages training resources within the State Department, Customs, Justice, FDA and the Federal Courts. OIPPE coordinates overarching IP efforts with other federal agencies through the Office of the IP Coordinator and through the National Intellectual Property Law Enforcement Coordination Council (NIPLECC) to ensure a coordinated and comprehensive approach to solving IP related issues. While USTR leads trade negotiations and WTO representation, OIPPE provides technical IP policy advice. Similarly, State has the lead for all United Nations agencies, and OIPPE provides technical IP policy advice on IP issues that arise within the United Nations. The International Trade Administration - Market Access and Compliance (ITA - MAC) focuses on individual company casework, as opposed to IP policy.

Evidence: Legislative mandate: 35 USC Section 2 (b) 8-13 http://www.uspto.gov/web/offices/pac/mpep/documents/appxl_35_U_S_C_2.htm Catalog of Federal Domestic Assistance (www.cfda.gov) DOC Departmental Administrative Order DOO-10-14 (Section 4) Role of NIPLECC members - Report to the President and Congress on Coordination of Intellectual Property Enforcement and Protection, September 2006, National Intellectual Property Law Enforcement Cooperation Council, page. 15. URL: http://www.commerce.gov/opa/press/Secretary_Gutierrez/2006_Releases/September/2006%20IP%20report.pdf

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: OIPPE receives 100 percent of its funding from user fees collected from patent and trademark applicants and from reimbursable inter-agency agreements, which allows the program to easily identify where the program's benefits should be delivered. There is no evidence of major design flaws or indication that a different design would be more effective. In addition, the ongoing oversight provided by the statutorily established Patent and Trademark Public Advisory Committees (PPAC and TPAC) have not identified any major flaws that would limit the program's effectiveness or efficiency.

Evidence: No legislative initiatives, or independent studies, suggest major flaws in the mission or organization of OIPPE. The statute provides exclusive authority to USPTO and OIPPE. Patent Public Advisory Committee (PPAC) and Trademark Public Advisory Committee (TPAC) Documents: http://www.uspto.gov/web/offices/com/advisory/index.html OIPPE Organizational Chart (internal document) OIPPE Logic Model (internal document) MOUs with Department of Commerce for IP Attaches (Internal documents) 2008 NIPLECC Report Scorecard: http://www.stopfakes.gov/pdf/2008_NIPLECC_Report_and_Appendices_Final.pdf (page 8)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: OIPPE resources are directly targeted to its promotion and advocacy of Intellectual Property. OIPPE has two primary, intended audience groups - USG officials who need advice on IP issues, and foreign officials who are responsible for IP policy and enforcement in their countries. By engaging these groups, OIPPE works to establish conditions that ultimately benefit American businesses and inventors, who provide the program's resources through patent and trademark application fees. OIPPE directly reaches its ultimate beneficiaries, U.S. businesses, mainly small and medium sized enterprises, through Intellectual Property Awareness Campaigns (IPAC), and the Strategy Targeting Organized Piracy (STOP!) Fakes website, brochure and toolkits. Other U.S. audience groups, such as other USG agencies, directly access OIPPE program services through phone calls, emails, and interagency agreements. The foreign audience is directly engaged through U.S. Embassy contacts, international IP organizations, and on-line registration for training opportunities. OIPPE's technical assistance is targeted to countries in the process of accession to the World Trade Organization (WTO), Free Trade Agreement (FTA) and various Intellectual Property Offices to assist them in meeting their obligations under the WTO Agreement on Trade Related Aspects of Intellectual Property Rights (TRIPS). OIPPE also works closely with the U.S. Trade Representative on the 301 review for countries that deny adequate and effective protection of intellectual property, which often results in improvements in IP systems. OIPPE prioritizes its education and outreach to the BRIC (Brazil, Russia, India, and China) countries and uses its annual action plans to set strategic priorities. To ensure that the right foreign nationals are reached, OIPPE relies on U.S. Embassy contacts within countries to nominate officials for training.

Evidence: 2007-2012 Strategic Plan http://www.uspto.gov/web/offices/com/strat2007/stratplan2007-2012.pdf; page 23-28) 2007 Performance and Accountability Report http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (pages 24-28) STOP! Fakes Website: http://www.stopfakes.gov/ IPAC Road Show Agenda - Phoenix IPAC Road Show Surveys (internal documents) GIPA Participant End-of-Course Survey Form Example (internal document) 2007 Special 301 Report Documents (internal documents) Country Action Plans for Brazil, Russia, India, China, Middle East, ASEAN (internal documents) Training Nominee Cable (internal document) MEPI Training Tour Cable (internal document) Advanced Patents Program Cable (internal document) MOUs with other countries for requested services of OIPPE (internal documents) Country Law Reviewed - example document from CAFTA - DR, more available (internal document) 2007 monthly technical assistance reports (internal documents) October Performance Review of acceptance of IP advice (internal document)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: To achieve its goal of improving IP protection worldwide, OIPPE has established three long-term outcome measures and one long-term output measure that reflect OIPPE's span of influence and effectiveness of its activities. These are: 1) percentage of countries on the USTR 301 list, which details IP related problems in developing countries, or awaiting WTO ascension that have positively amended or improved their IP systems because of OIPPE involvement - this measure tracks the long-term goal of improving global IP systems at the country level; 2) percent of foreign officials trained by Global Intellectual Property Academy (GIPA) who have initiated or implemented a positive IP change in their organizations - this measure tracks the long-term effectiveness of OIPPE's training and education efforts conducted in the United States; 3) number of countries that implement at least 75% of the action steps in their joint cooperation plans over a two year period - this measure tracks the implementation of action plans that are jointly developed between the United States and host government, and serves as a measure of in-country effectiveness of OIPPE activities; and 4) the number Memoranda of Agreement for IP joint cooperation plans of action, mechanisms and support programs initiated or implemented by developing countries as a result of OIPPE guidance (output) - this measure tracks the scope and influence of OIPPE in getting developing countries to commit to steps that support the long-term outcome of helping U.S. businesses by increasing IP protections.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 24) 2005, 2006 and 2007 Special 301 Reports: http://www.ustr.gov/Document_Library/Reports_Publications/2007/2007_Special_301_Review/Section_Index.html (url for 2007 report) WTO protocols of accession: http://www.wto.org/english/thewto_e/acc_e/completeacc_e.htm PART listing of long-term and annual performance measures. OIPPE Logic Model (internal document)

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: OIPPE established quantified targets for all of its long-term measures through a five year timeframe to 2013. Three of the four measures have baseline data. Baseline data for the new measure (number 3 above) will be established in 2008. Targets for the number of section 301 countries implementing changes that improve IP protections is envisioned to double in five years, while the percentage of foreign offices trained through GIPA, who implement measures to improve IP protections is planned to increase by 50 percent. Also, the targets for the number of IP joint plans of action signed remains at a high level for each year.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 24) PART Listing of long-term and annual performance measures. OIPPE Logic Model (Internal document)

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: OIPPE has annual performance measures that demonstrate progress toward achieving its long-term goals that envision improved IP protections in other countries. OIPPE has one outcome measure, which they will develop baseline data for in 2008. That measure tracks the number of individuals, who undergo IP training from OIPPE, that show increased knowledge of proper IP protection practices. Increased IP protection knowledge is an intermediate outcome towards actual implementation of this knowledge in other countries.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 24) PART Listing of long-term and annual performance measures. OIPPE Logic Model (internal document) OIPPE Outcome Chain (internal document)

Does the program have baselines and ambitious targets for its annual measures?

Explanation: OIPPE has baselines for all of its output measures and has set ambitious targets to meet each year for all measures. The one annual measure without baseline data (percent of foreign officials with increased IP expertise) is a new measure in 2008 and baseline data will be gathered in 2008. The targets for the aforementioned output measures envision increasing output, except for the number IP policies reviewed by OIPPE. However, that measure has experienced great volatility in recent years and the target is above the average of recent year's actuals.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 24) PART Listing of long-term and annual performance measures. OIPPE Logic Model (internal document) OIPPE Outcome Chain (internal document) 2006, 2007 and 2008 (1st Quarter) Process production reports, (page 10 for each document) (internal documents)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: While OIPPE does not have any grantees, contractors or other non-government partners that receive program funding or contribute to the long-term goals, it does work with several U.S. Government partners; however, none of these partners directly report on OIPPE's performance measures, rather each is commits certain shared goals and performance measures through interagency agreements. OIPPE has interagency agreements with the Department of State providing funding for IP Rights Enforcement training through the Middle East Partnership Initiative (MEPI), the ASEAN Cooperation Plan, International Narcotics and Law Enforcement (INL) for program activity in the ASEAN region, Russia, Africa and Central America, as well as interagency agreements with NIPLECC, Department of Justice, the Federal Bureau of Investigation. The MEPI, INL and ACP programs are conducted under the agreements based on proposals by OIPPE. The interagency agreements cover specific joint activities, with articulated goals, to be conducted in various regions and on particular topics by OIPPE.

Evidence: 2008 NIPLECC Report: http://www.stopfakes.gov/pdf/2008_NIPLECC_Report_and_Appendices_Final.pdf STOP ! Initiative Description STOP! Fakes website: http://www.stopfakes.gov/ Copy of MEPI and INL (State Department) MOUs (internal documents) Examples of MOUs with other countries in Patents Rights in India and China: Office-to-Office Initiatives PowerPoint slides 18-20 (internal document) Heads of Patent Offices Meeting Agenda (internal document) Heads of Patent Offices Record of Discussion (internal document)

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: OIPPE has contracted with external, independent evaluation professionals, with experience in international program evaluation, to conduct several evaluations of its activities over the next three years that bring the appropriate methodology and rigor to assessing OIPPE programs and their effectiveness. These evaluations include 1) a meta-evaluation of the end-of-program surveys collected from GIPA and in-country training participants and impact survey; 2) a process implementation evaluation of the IP Attach?? Program (jointly coordinated with an OIG review of the program); 3) designing and deploying a pre-program, post-program, and follow-on evaluation system for IP training participants; and 4) a GIPA impact evaluation. These assessments are scheduled to be completed by the end of FY 2009 and will provide valuable information for program improvements and will also serve as baseline data for repeated, regular evaluations every three-to-five years. OIPPE plans that it will also have independent evaluators conduct process and improvement and impact assessments of its policy review function and IPAC road shows in FY 2009 and 2010. In addition, the Patent Public Advisory Committee and Trademark Public Advisory Committee, as well as industry groups, like the Free Trade Agreement Industry Functional Advisory Group, review the work of OIPPE within larger contexts of Intellectual Property protection issues.

Evidence: OIPPE Evaluation Plan/Schedule (Internal document) OIG review - expected June 2008 (internal document) Patent Public Advisory Committee (PPAC) and Trademark Public Advisory Committee (TPAC) Documents: http://www.uspto.gov/web/offices/com/advisory/index.html Free Trade Agreement Industry Functional Advisory Group Reports (CAFTA-DR example): http://www.ustr.gov/assets/Trade_Agreements/Regional/CAFTA/CAFTA_Reports/asset_upload_file571_5945.pdf

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The USPTO performance budget provides a linkage between USPTO's budget request and the results the public can expect. The performance budget defines the performance goals and measures used to manage progress toward our strategic objectives. An activity-based-costing methodology is used to identify various factors that drive program costs by strategic goal as well as by primary business line to depict a transparent relationship of costs by strategic goal to outcome measures. All budget estimates include direct and indirect costs. The Performance and Accountability Report provides annual performance results against targets that are set in the budget request.

Evidence: 2009 USPTO Budget Presentation: http://www.uspto.gov/web/offices/ac/comp/budg/fy09pbr.pdf (pages 30-35) 2007 Performance and Accountability Report (url: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (page 28) Project Program Activity Codes (internal document)

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The 2007-2012 Strategic Plan was release along with the transmission of the FY 2008 President's Budget to Congress. As part of the strategic plan OIPPE developed long-term measures, which are reflected in the measures tab, to better assess its impact on improving global IP protections. OIPPE also created a program logic model to reflect its program theory of change and how the annual outputs and outcome measure relate to the program's long-term goals.

Evidence: 2007-2012 Strategic Plan: http://www.uspto.gov/web/offices/com/strat2007/stratplan2007-2012.pdf; (page 23) Issue Action plans (internal documents) Technical Assistance Overview (internal documents) OIPPE Logic Model (Internal documents) PART Listing of annual and long-term performance measures Statement by OIPPE Director (Internal documents)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: OIPPE regularly collects information on policies reviewed, GIPA trainee success and satisfaction, technical assistance activities including Free Trade Agreement Implementation Activities, China Related Activities, U.S. Government representation at Multilateral Conferences on IP issues, WIPO meetings, International Cooperation Meetings and Plans of Action Initiated or Implemented in Developing Countries. These are captured in weekly reports prepared by the OIPPE staff, who gather the information from partners and international counterparts, as well as based on their own activity-based accounting. Information in weekly reports is tallied and included with internal performance data, such as teleworking levels, for the quarterly USPTO Organizational Assessment report. The USPTO Organizational Assessment includes highlights and challenges, is discussed by program managers at weekly meetings, is used to rate SES level managers, and is compared to targets for OIPPE's performance measures.

Evidence: Weekly Report Example (internal document) Process Production Reports, 2006, 2007, 2008 - First Quarter (page 10 each document) (internal documents) USPTO FY 2008 Qtr 1 Quarterly Organizational Assessment Report (internal document) USPTO Quarterly Performance Card (internal document) Management Council Meeting Agenda with discussion notes (internal document) PCT Yearly Review for 2006 (available online at http://www.wipo.int/pct/en/activity/pct_2006.html) PCT Explanation Document (internal document) GIPA Survey Responses (2-26-07) example (internal document)

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Managers at each level have specific task in support of the strategic goals and objectives, including cost, schedule, and results. The Performance Standards for all OIPPE SES managers include responsibility to achieve the organization's performance measures. Senior and mid-level managers are held accountable for the schedule, cost and performance results through the pay for performance system, where promotions and awards are linked to the accomplishment of results. The alignment also goes to the programmatic and partner level. For example, the IP Attaches' critical performance elements align with the missions of the Department of Commerce's Commercial Service. The IP Attach?? annual plans link to long-term results and align with the Commercial Service's mission and vision.

Evidence: SES Performance Plan Example (internal document) Senior/Mid-Level manager Performance Plan Example (internal document) Quarterly Performance Card (internal document) IP Attach?? work (performance) plans (internal documents)

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: OIPPE obligated 100 percent of its reimbursable funds in FY 2005, 2006 and 2007. USPTO develops operating plans for each organization based on appropriated funding levels and USPTO goals and priorities with routine monitoring of spending against established program plans. USPTO uses the MOMENTUM System, which provides the means to monitor and control all accounting related to the execution of Program Offices' budgets, including establishing spending limits, exercising funds control, and accounting for revenue and reimbursements. The system tracks plan versus actual amounts by month, quarter and year, and records payable amounts resulting from liquidated obligations and schedules payments according to the Prompt Payment Act. Monthly budget execution reports produced by the Office of Corporate Planning are shared with managers and staff.

Evidence: Spending Report (RBESPO - FM 13) (internal document) 2007 External Affairs Operating Plan (internal document) MOMENTUM systems document (internal document) Vouchers and Invoices with U.S. Embassy Partners Exhibits (internal documents)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: OIPPE has taken steps to increase efficiency, such as partnering with the International Chamber of Commerce to reduce costs of the IPAC road shows by 91 percent. Also, OIPPE automated its knowledge management system, which eliminated multiple copying of cables and other documents, and helped ease access to critical information from overseas. OIPPE has one cost efficiency measure that tracks administrative costs versus total program costs. By decreasing the administrative costs share of total program costs, OIPPE will be able to operate a more efficient program by spending more resources on program activities.

Evidence: Documentum Description and Systems documents including screen shots (internal document): Documentum 5.3 functions as a stand-alone, electronic storage/retrieval database responsible for storing: official records, historical documents, working documents, and multimedia presentations for the Office of External Affairs. Presently, paper documents and records are uploaded to Documentum 5.3 through a scanning process while electronic documents and records are transmitted to Documentum administrators via e-mail and are uploaded. Documentum 5.3 can store and retrieve records and non-records; however, it is unable to classify documents as records. Documentum 5.3 currently services 51 end-users in the Office of External Affairs and is currently not accessible to IP Attaches posted worldwide. IPAC Savings: In FY 2006 OIPPE conducted 6 IPAC programs at an average cost of $76,017 per program for a total of $456,103. In FY2007 OIPPE partnered with the International Chamber of Commerce on the IPAC events and conducted 6 programs for a total cost of $47,370 or an average cost of $6,767 per program. The Chamber covered the venue costs and refreshments (coffee, etc.) for most of the events and our costs were travel for OIPPE staff who were speaking. Visa savings: In the past EA paid VISA application fees for each foreign traveler to attend GIPA programs. EA now informs attendees to request an A2 VISA that is issued by their Embassy free of charge. At an application fee of $131 pp and 30-40 program attendees, this is a savings of $4,000- $5,200 per program. By researching and using A2 visas for GIPA attendees, OIPPE saved approximately $95,000 in 2007. Hotel savings document (internal document): Lodging costs are another area where OIPPE has achieved efficiencies. Lodging costs per attendee at government rates could cost up to $214 per night at local hotels. In October 2006, GIPA solicited bids from nearby hotels for rates (including breakfast) and was able to secure local hotels at lower than government rates. The total hotel contract savings for FY 2007 was $33,300 and so far in FY 2008 OIPPE has saved over $41,500 relative to the government per diem rate for lodging. In addition, OIPPE has been able to reduce per diem costs by one meal for each attendee because the hotels we contracted with includes breakfast.

Does the program collaborate and coordinate effectively with related programs?

Explanation: USPTO, as one agency on the National Intellectual Property Law Enforcement Coordination Council, collaborates on the Bush Administration's Strategy Targeting Organized Piracy (STOP) campaign, which resulted in OIPPE taking the lead on certain IP educational activities where it had expertise. For example, OIPPE operates a call center, as part of the STOP initiative, to educate businesses on their IP rights. In addition to its primary coordination through NIPLECC, OIPPE also consults with the Copyright Office of the Library of Congress on intellectual property policy formulation and proposed legislation. OIPPE partners with Customs and Border Protection of the U.S. Department of Homeland Security by assisting in the identification and eventual seizure of counterfeit goods and services imported into the U.S. OIPPE, through its Global IP Academy (GIPA), coordinates with other agencies on training and routinely includes other USG speakers for a given program, such as Federal Judges, Customs Officers, Department of Justice prosecutors, and FBI officials (all for various Enforcement of IPR programs), Department of Agriculture officials (on plant varietals protections and the UPOV treaty for specialized Patents programs), NIH specialists (on Traditional Knowledge/Genetic Resources programs), USAID and Department of State (on Convention on Biological Diversity programs) and main Commerce and NIST officials (on Tech Transfer programs).

Evidence: Trilateral website: http://www.trilateral.net/ Trilateral Pre-conference reports from USPTO, EPO and JPO (internal documents) Workshare PowerPoint (internal document) Daily News Clips, page 2 (internal document) 2008 NIPLECC Report: http://www.stopfakes.gov/pdf/2008_NIPLECC_Report_and_Appendices_Final.pdf STOP ! Initiative Description STOP! Fakes website: http://www.stopfakes.gov/ MEPI Training Tour Cable (internal document)

Does the program use strong financial management practices?

Explanation: For the FY 2007 Financial Statement Audit performed by the OIG and KPMG, USPTO received an unqualified audit opinion with no material weaknesses or reportable conditions. FY 2007 marked the 15th year of an unqualified audit opinion and the 11th year without material internal control weaknesses supporting the financial management practices. The USPTO complied substantially with the FFMIA for FY 2007. Also, the USPTO operates a mature Activitiy-Based Cost (ABC) accounting system that captures costs of core mission activities and both direct and indirect costs for the entire Office.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) (pages 46-49) USPTO's FY 2007 Financial Statements Audit Report, OIG Audit Report Number FSD-18531-8-0002 (hyperlinked) FFMIA Compliance Report and Statement: http://www.uspto.gov/web/offices/com/annual/2007/30600_mng_comply.html USPTO 2007 Combined Report from KPMG.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: After rapid and accelerated growth in FY 2005 (from 14 to 98 FTE), OIPPE realigned its functions in FY 2007 to integrate better policy review and enforcement. It also separated out the training function through the GIPA, so that there is a dedicated set of resources and focus to bring high quality training to key foreign counterparts. The office has implemented the following measures for continued improvement: Individual Action Plans, Senior Level Counsel, an Interagency Agreement Management Process, GIPA Course Evaluations and Performance Measures.

Evidence: OIPPE Organizational Order (internal document) OIPPE Organization chart (internal document) Individual action plans (internal, confidential documents) GIPA Participant End-of-Course Survey Form Example (internal document) Technical Assistance Overview (internal document) PART list of annual and long-term performance measures

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: OIPPE has targets to compare actual performance for three of its four long-term measures. The program missed its target in 2006 for the number of section 301 countries that made a positive change to their IP protection systems and met a substantially reduced goal for 2007. However, the OIPPE exceeded its targets in 2007 for the number of foreign officials who have initiated a positive IP change and the number of joint cooperation plans in place with other countries. In addition, improvements in the intellectual property regimes of numerous countries resulted from OIPPE working closely with USTR in the Special 301 process and on Free Trade Agreements (FTAs) and WTO accessions.

Evidence: PART list of long-term performance measures Report to the President and Congress on Coordination of Intellectual Property Enforcement and Protection, September 2006, National Intellectual Property Law Enforcement Cooperation Council, pages 38-42. URL: http://www.commerce.gov/opa/press/Secretary_Gutierrez/2006_Releases/September/2006%20IP%20report.pdf Special 301 Report 2005 (pp. 51-65) http://www.ustr.gov/Document_Library/Reports_Publications/2005/2005_Special_301/Section_Index.html Special 301 Report 2006 (pp. 2-3) http://www.ustr.gov/Document_Library/Reports_Publications/2006/2006_Special_301_Review/Section_Index.html Special 301 Report 2007 (pp. 3-4) http://www.ustr.gov/Document_Library/Reports_Publications/2007/2007_Special_301_Review/Section_Index.html Free Trade Agreement Industry Functional Advisory Group Reports (CAFTA-DR example): http://www.ustr.gov/assets/Trade_Agreements/Regional/CAFTA/CAFTA_Reports/asset_upload_file571_5945.pdf WTO protocols of accession: http://www.wto.org/english/thewto_e/acc_e/completeacc_e.htm 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html)

Does the program (including program partners) achieve its annual performance goals?

Explanation: OIPPE has not developed targets for its annual outcome measures yet, which makes program effectiveness difficult to definitively determine. However, the program has been able to increase the number of attendees at intellectual property training sessions, especially with officials from Brazil Russia, India, and China, which may indicate furture success on thier outcome measures.

Evidence: 2007 Performance and Accountability Report: http://www.uspto.gov/web/offices/com/annual/2007/30204_sg3perfrm.html) 2007 Participants by Country spreadsheet (Internal document) PART list of annual performance measures

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: OIPPE has reduced the per participant cost of GIPA training classes. The reduction of cost per GIPA participant trained, resulted in a savings of $42,252, which allowed an additional 14 participants. OIPPE also realized cost savings by partnering with the International Chamber of Commerce on IPAC events, which resulted in a total cost savings of $408,733. Also, OIPPE has one cost efficency measure that tracks administrative costs share of total costs. By decreasing administrative costs' share of total costs, OIPPE will be more efficent and use a greater share of resources on program activities.

Evidence: Budget tracking sheets (Internal documents)

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Interagency Working Group for International Exchanges and Training (IAWG - Department of State) in a report on performance measurement did not set common performance measures for exchange and training programs across the USG, which makes comparison difficult. On the international front, EPO and JPO have also not established outcome-related performance measures for their international policy functions by which OIPPE can benchmark. OIPPE anticipates that a benchmarking study will begin in 2010 to study EPO, JPO as well as USG agencies engaged in international training. The existence of measures to judge OIPPE programs is one indicator that they may compare favorably to other countries' programs.

Evidence: IAWG Report on Measuring Performance of International Training and Exchanges: http://www.iawg.gov/rawmedia_repository/dc07a56b_8539_42a2_8e48_0a67fb0c7fe3 2008 NIPLECC Report: http://www.stopfakes.gov/pdf/2008_NIPLECC_Report_and_Appendices_Final.pdf Trilateral website: http://www.trilateral.net/ Trilateral Pre-conference reports from USPTO, EPO and JPO (internal documents) EPO Website: http://www.epo.org/about-us/office/academy.html JPO Website: http://www.jpo.go.jp/

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A meta-evaluation of GIPA end of course surveys, conducted by independent evaluators, found that GIPA was increasing IP knowledge of foreign participants country. However, the evaluation also recommended that GIPA 1) develop an alumni affairs function to remain in contact and provide follow-on assistance to participants; 2) work more closely with US Embassies to track participants' career trajectories and contributions to IP on a yearly basis; 3) develop a certification course that allows participants to take multiple courses in person and/or on-line on a well-rounded battery of IP focused topics; 4) benchmark similar training institutes and programs to identify best practices; and 5) utilize standardized pre, post and follow-up surveying to more rigorously track participant outcomes. Currently, an independent contractor is conducting a review of the IP Attach?? program, which should provide more information on the actual impact of OIPPE's work.

Evidence: CAFTA - DR Report: http://www.ustr.gov/assets/Trade_Agreements/Regional/CAFTA/CAFTA_Reports/asset_upload_file571_5945.pdf (page 2) GIPA meta-evaluation (internal document) OIG IP Attach?? Review project overview (internal document)