OMB Circular A-11 -- SECTION 240 - ANNUAL PERFORMANCE PLANNING AND REPORTINGThe Annual Performance Plan (APP) is a description of the level of performance to be achieved during the fiscal year in which the plan is submitted and the next fiscal year. The plan should also be specific in describing the strategies the agency will follow, explaining why those strategies have been chosen, and identifying performance targets and key milestones that will be accomplished in the current and next fiscal year. It should be comprehensive of the agency’s mission by showing the plan for each strategic objective.The Annual Performance Plan should align to the agency's strategic goals and objectives, explaining how they will be achieved. Strategic goals are advanced by strategic objectives, which in turn, are supported by specific performance goals and indicators. For each strategic goal, the annual plan should show the supporting strategic objectives and performance goals. The indicators that will be used to track, interpret or improve progress on performance goals must also be included in the performance plan. Agencies should add performance goals as needed to reflect multiple objectives: * Mission. The core functions and activities of Federal agencies that are reflected in statutory requirements or leadership priorities and which serve to drive their efforts in addressing pressing and relevant national problems, needs, and challenges. * Service. The activities that reflect the interaction(s) between individual citizens or businesses and Federal agencies in providing a direct service on behalf of the Federal Government, and which is core to the mission of the agency. * Stewardship. The responsibilities of Federal agencies to provide appropriate safeguards in executing mission and service related activities effectively and efficiently, including minimizing instances of waste, fraud, and abuse. ^ The Annual Performance Plan supports the agency’s budget request by identifying the performance goals and key milestones that an agency will pursue in the coming year. Results of agency progress on strategic objectives and performance goals are presented and discussed in the Annual Performance Report. Office of Management and BudgetOMB_fc2d8890-ba77-4c51-bbad-1cc10871c14eRussell T. VoughtActing Director Federal AgenciesAgencies will develop their FY 2022 APP to align with the strategic plan framework of goals and objectives reflected in the agency FYs 2018-2022 Strategic Plan. Agencies will also have an opportunity in the FY 2022 APP to update as needed performance goals, measures, and targets from the FY 2021 APP to support the agency’s strategic objectives and priorities reflected in the agency Strategic Plan. The FY 2023 APP published in February, 2022 will be developed throughout the course of 2021 and aligned to the new Strategic Plan covering FYs 2022-2026. _aada480c-a90a-11e9-a082-d69e4d1a686aTo provide guidance on the preparation of annual performance plans and reports._aada4910-a90a-11e9-a082-d69e4d1a686aPerformance PlansDescribe the level of performance to be achieved during the fiscal year in which the plan is submitted and the next fiscal year._40bb7f1c-1a52-11eb-81bb-e133f482ea00240.1What is an Annual Performance Plan? The Annual Performance Plan (APP) is a description of the level of performance to be achieved during the fiscal year in which the plan is submitted and the next fiscal year. The plan should also be specific in describing the strategies the agency will follow, explaining why those strategies have been chosen, and identifying performance targets and key milestones that will be accomplished in the current and next fiscal year. It should be comprehensive of the agency’s mission by showing the plan for each strategic objective._aada4bb8-a90a-11e9-a082-d69e4d1a686aPurposeCommunicate strategic objectives and performance goals with other elements of budget requests._40bb8214-1a52-11eb-81bb-e133f482ea00240.2What is the purpose of an Annual Performance Plan? Agencies prepare an Annual Performance Plan to communicate the agency’s strategic objectives and performance goals with other elements of the agency budget request. The plan describes how the goals will be achieved, identifies priorities among the goals and explains how the agency will monitor progress. The APP also updates the previous APP to reflect changes in plans, funding decisions, and changes in the environment._40bb84d0-1a52-11eb-81bb-e133f482ea00Strategic AlignmentAlign performance plans to strategic goals and objectives._40bb887c-1a52-11eb-81bb-e133f482ea00240.3How does the Annual Performance Plan relate to the Strategic Plan? The Annual Performance Plan should align to the agency's strategic goals and objectives, explaining how they will be achieved. Strategic goals are advanced by strategic objectives, which in turn, are supported by specific performance goals and indicators.Goals & ObjectivesShow the supporting strategic objectives and performance goals_40bb8a34-1a52-11eb-81bb-e133f482ea00240.3.1For each strategic goal, the annual plan should show the supporting strategic objectives and performance goals. The indicators that will be used to track, interpret or improve progress on performance goals must also be included in the performance plan. Agencies should add performance goals as needed to reflect multiple objectives:MissionInclude goals for core functions and activities_428f13c2-1b19-11eb-aab2-1e641a83ea00240.3.1.1Federal AgenciesMission. The core functions and activities of Federal agencies that are reflected in statutory requirements or leadership priorities and which serve to drive their efforts in addressing pressing and relevant national problems, needs, and challenges.ServiceInclude goals reflect the interaction(s) between individual citizens or businesses and Federal agencies_428f1732-1b19-11eb-aab2-1e641a83ea00240.3.1.2CitizensBusinessesFederal AgenciesService. The activities that reflect the interaction(s) between individual citizens or businesses and Federal agencies in providing a direct service on behalf of the Federal Government, and which is core to the mission of the agency.StewardshipInclude goals related to safeguards in executing mission and service related activities effectively and efficiently_428f1a66-1b19-11eb-aab2-1e641a83ea00240.3.1.3Federal AgenciesStewardship. The responsibilities of Federal agencies to provide appropriate safeguards in executing mission and service related activities effectively and efficiently, including minimizing instances of waste, fraud, and abuse.Budget RequestsIdentify the goals and milestones to be pursued in the coming year_428f1e4e-1b19-11eb-aab2-1e641a83ea00240.3.2The Annual Performance Plan supports the agency’s budget request by identifying the performance goals and key milestones that an agency will pursue in the coming year. Results of agency progress on strategic objectives and performance goals are presented and discussed in the Annual Performance Report.Strategic PlansAlign performance plans with the goals and objectives in the agency FYs 2018-2022 strategic plans_428f211e-1b19-11eb-aab2-1e641a83ea00240.3.3Agencies will develop their FY 2022 APP to align with the strategic plan framework of goals and objectives reflected in the agency FYs 2018-2022 Strategic Plan. Agencies will also have an opportunity in the FY 2022 APP to update as needed performance goals, measures, and targets from the FY 2021 APP to support the agency’s strategic objectives and priorities reflected in the agency Strategic Plan. The FY 2023 APP published in February, 2022 will be developed throughout the course of 2021 and aligned to the new Strategic Plan covering FYs 2022-2026.Additions & DeletionsIdentify performance measures that are added or dropped_428f22cc-1b19-11eb-aab2-1e641a83ea00240.3.4Agencies may choose to drop or add measures in the Annual Performance Plan, in consultation with OMB. Agencies should include a consolidated list or summary of changes in the FY 2022 APP that identifies performance measures that are added or dropped. Agencies should still report FY 2020 results of any performance goals and indicators that will be discontinued in the FY 2022 APP; targets for dropped measures no longer need to be set as part of the FY 2021 APP.Budget JustificationsUse the performance plan to structure budget submissions._40bb8bd8-1a52-11eb-81bb-e133f482ea00240.4What is the relationship between the Annual Performance Plan, Annual Performance Report and Congressional Budget Justification? Section 51 outlines agency budget justification requirements. The performance plan may be used to structure the budget submission, or at minimum, be part of the agency’s budget submission to OMB and to Congress.ChangesReflect changes to the program activities in the budget request_40bb8e6c-1a52-11eb-81bb-e133f482ea00240.4.1Changes in the plan should reflect changes to the program activities in the budget request.ConsolidationConsolidate the Annual Performance Plan with the Annual Performance Report_428f2614-1b19-11eb-aab2-1e641a83ea00240.4.2To reduce duplication and to communicate future plans in the context of historical trends, agencies are strongly encouraged to consolidate the Annual Performance Plan with the Annual Performance Report to deliver them concurrent with the Congressional Budget Justification and on Performance.gov.ConsultationConsult with relevant congressional appropriations committees_428f2812-1b19-11eb-aab2-1e641a83ea00240.4.3Congressional Appropriations CommitteesAgencies should consult with relevant congressional appropriations committees to confirm their support for modifications to the format of the Congressional Budget Justification.Priority GoalsInclude contributions to Cross-Agency Priority (CAP) goals as well as establish performance goals, measures, and targets in management and policy priority areas required by Executive Orders or OMB memoranda and guidance._40bb8ffc-1a52-11eb-81bb-e133f482ea00240.5Does the agency Annual Performance Plan include contributions to the Cross-Agency Priority Goals and other Administration priorities or initiatives? Yes, as applicable, agency Annual Performance Plans shall, include contributions to Cross-Agency Priority (CAP) goals as well as establish performance goals, measures, and targets in management and policy priority areas required by Executive Orders or OMB memoranda and guidance. In developing the APP, agency PIOs should coordinate across the organization to ensure the APP is comprehensive of the agency’s mission, including identified policy or management priorities the agency is working to achieve. For example:Environment & InfrastructureInclude goals related to the completion of environmental reviews and authorization for infrastructure projects_40bb915a-1a52-11eb-81bb-e133f482ea00240.5.1Per Executive Order 13807, agencies with environmental review, authorization, or consultation responsibilities for infrastructure projects shall include in the agency’s Annual Performance Plan agency performance goals related to the completion of environmental reviews and authorization for infrastructure projects consistent with the CAP Goal on Infrastructure Permitting Modernization; andRegulatory ReformEstablish goals and indicators related to regulatory reform_40bb93d0-1a52-11eb-81bb-e133f482ea00240.5.2Per OMB Memorandum M-17-23, agencies are required to establish performance goals and associated indicators related to regulatory reform accountability under E.O. 13777.Real PropertyIncorporate goals and metrics related to capital planning for real property_40bb956a-1a52-11eb-81bb-e133f482ea00240.5.3Landholding AgenciesPer OMB Memorandum M-20-03, all landholding agencies, beginning within their FY 2022 Annual Performance Plan, should incorporate performance goals and metrics developed as part of the agency’s capital planning for real property.CAP GoalsDevelop reporting to support progress on CAP goals_40bb9696-1a52-11eb-81bb-e133f482ea00240.5.4CAP Goals TeamsInstances where CAP Goals teams, as part of the goal’s implementation strategy, have developed specific reporting to support goal progress (e.g., Category Management). See sections 210 and 220 for additional details on how to incorporate these policy and management priority areas in the APP.ResourcesLink resources to performance plans._40bb9a24-1a52-11eb-81bb-e133f482ea00240.6How will agencies be expected to link resources to the performance plan this year?BudgetsInform agency budget decisions_40bb9bdc-1a52-11eb-81bb-e133f482ea00240.6.1Performance information in the Annual Performance Plan (APP), especially the goals, indicators of past performance and other evidence such as evaluations, should inform agency budget decisions, complementing other factors considered in the budget process. The funding proposed in agency FY 2022 final budget submissions should reflect funding levels the agency believes are needed to meet proposed FY 2022 targets. The performance goals in Annual Performance Plans should be consistent with those set through agency strategic and performance planning processes.UpdatingUpdate FY 2022 performance plans to reflect budgeted resources as well as other policies or directives provided to the agency_428f2970-1b19-11eb-aab2-1e641a83ea00240.6.2As funding levels are finalized through the Budget Process, agencies must update their FY 2022 APP as applicable to reflect budgeted resources as well as other policies or directives provided to the agency through OMB Passback. In addition to incorporating budget policies or other guidance provided in Passback, current fiscal year performance goals should also be updated to reflect final congressional action on appropriations and other changes in external conditions or management priorities as necessary.ProgramsInclude goals and objectives for all program activities_428f2d62-1b19-11eb-aab2-1e641a83ea00240.6.3Program activities are a key organizing component that connects budget and funding requests as part of the coordinated strategies for achieving goals and objectives in the agency Strategic Plan. Thus, strategic goals and objectives in the performance plan should capture efforts for all program activities in the budget request, and support the implementation strategies for achieving strategic objectives and performance goals.Aggregation & DisaggregationAggregate, disaggregate, or consolidate program activities as appropriate_428f2f88-1b19-11eb-aab2-1e641a83ea00240.6.4Agencies may aggregate, disaggregate, or consolidate program activities for the purposes of aligning performance information and resources as appropriate for the agency size, except that any aggregation or consolidation may not omit or minimize the significance of any program activity constituting a major function or operation for the agency.ContentInclude the specified content in annual performance plans._aada4cb2-a90a-11e9-a082-d69e4d1a686a240.7What content should be included in the Annual Performance Plan and how will it be published? The content table in section 210 establishes what information must be included in the Annual Performance Plan. This section should be considered in conjunction with section 51, on Basic Justification Materials as well as section 210 which describes requirements for publication of the plan on Performance.gov. Agencies that are required to establish FYs 2020–2021 Agency Priority Goals on Performance.gov will also be expected to publish Annual Performance Plan information on Performance.gov. The FY 2022 Annual Performance Plan will be developed to align with Administration policies and should reflect, where applicable, the agency’s FYs 2020-2021 APGs. The FY 2022 Annual Performance Plan will be published concurrent with the agency’s final FY 2022 congressional budget justification, with all agencies producing a full agency performance plan for posting on the agency’s website with all required content. The agency's congressional committees may also require additional information for the performance plan that is submitted to Congress. Machine-ReadabilityProduce performance plans and reports in searchable, machine-readable format._aada4d70-a90a-11e9-a082-d69e4d1a686a240.7.1Machine-Readable. To more fully implement provisions of the GPRA Modernization Act of 2010 that require agency plans and reports be produced in a searchable, machine-readable format, OMB will continue to work with a small subset of Federal agencies on an exploratory initiative to produce machine-readable components or portions of the FY 2022 Annual Performance Plans and FY 2020 Annual Performance Reports. Producing agency performance plans and reports in a machine-readable format improves accessibility of agency performance data contained within the APP and APR, as well as enhances transparency through advanced data visualization tools. This machine-readable initiative for agency performance plans and reports will be conducted in conjunction with OMB and agency efforts that is also focused on exploring the production of machine-readable components of congressional budget justifications as part of the Budget cycle (see section 22.6). The purpose of the effort will be to identify processes, resources, and best practices in order to inform a timeline and additional guidance for full implementation of machine-readable performance plans and reports required by all Federal agencies in future Budget and Performance Planning and Reporting cycles. Management ChallengesReport performance improvement actions for major management challenges._aada4e38-a90a-11e9-a082-d69e4d1a686a240.8How should agencies report performance improvement actions for items identified as major management challenges in the Annual Performance Plan? The GPRA Modernization Act of 2010 (Pub. L. No. 111-352) requires agencies to describe the major management challenges the agency faces as part of the Agency Annual Performance Plan. Major management challenges are programmatic or management functions, within or across agencies, and may have greater vulnerability to waste, fraud, abuse, and mismanagement (such as issues the Government Accountability Office identifies as high risk or issues that an Inspector General (IG) identifies) or where failure to perform well could seriously affect the ability of an agency or the Federal Government to achieve its mission or goals.IG RecommendationsConsider IG recommendations on serious management and performance challenges_aada4f64-a90a-11e9-a082-d69e4d1a686a240.8.1Inspectors GeneralAgencies may consider IG recommendations on serious management and performance challenges as well as management issues and risks or areas most critical to the agency’s mission delivery when developing performance goals as part of performance planning, or as an input to the agency’s enterprise risk management planning and processes (see OMB Circular No. A-123).Performance GoalsHighlight instances where a performance goal has been developed as a result of a major management challenge_40bb9d1c-1a52-11eb-81bb-e133f482ea00240.8.2Where applicable, agencies should highlight instances where a performance goal has been developed as a result of a major management challenge, and include detailed performance information required in section 210.Performance PlansAddress major management challenges in Annual Performance Plans_40bb9f24-1a52-11eb-81bb-e133f482ea00240.8.3Agencies should address identified major management challenges in the Agency Annual Performance Plan as a part of Other Information if not addressed as agency priority or performance goals elsewhere in the performance plan. Agency discussion of major management challenges in the Annual Performance Plan should include: planned actions to address major management challenges; performance goals, indicators and/or milestones used to measure progress for the major management challenges identified; and the agency official (title and office) responsible for resolving such challenges. See section 210.Planned ActionsInclude planned actions to address major management challenges_40bba0dc-1a52-11eb-81bb-e133f482ea00240.8.4Indicators & MilestonesInclude performance goals, indicators and/or milestones used to measure progress for the major management challenges identified_40bba230-1a52-11eb-81bb-e133f482ea00240.8.5Responsible OfficialsName the agency official (title and office) responsible for resolving such challenges_40bba442-1a52-11eb-81bb-e133f482ea00240.8.6DataGather data to improve agency progress on goals._40bba5fa-1a52-11eb-81bb-e133f482ea00240.9What are data of “significant value?” What attributes and dimensions should agencies consider when selecting and gathering data to improve agency progress on goals? Data are most valuable when they are meaningful for analyzing progress and identifying ways to improve performance. Data need to be sufficiently accurate, and timely to inform a decision, behavior, or outcome by those who have authority to take action to drive progress towards mission, service, and stewardship outcomes. For information to be actionable, it must be prepared in a format appropriate for the user.SourcesIdentify data sources that are already available and assess whether they adequately measure performance goals_40bba758-1a52-11eb-81bb-e133f482ea00240.9.1A first step for agencies will be to identify data sources that are already available and assess whether they adequately measure the performance goal or other indicator of interest. Agencies may be able to adapt existing data, or they may find that new data sources will need to be developed. Data attributes to consider include: FrequencyDocument how often the data should be collected_40bba96a-1a52-11eb-81bb-e133f482ea00240.9.2Frequency—How often should the data be collected in order to impact future performance? How quickly is it needed to make sound decisions? Annually, quarterly, monthly, weekly, daily?TimingDocument whether the time of day, day of the week, or week of the year is likely to correlate with performance or causal factors affecting performance_40bbab2c-1a52-11eb-81bb-e133f482ea00240.9.3Time—Is the time of day, day of the week, or week of the year likely to correlate with performance or causal factors affecting performance (e.g.; time of incidents, demand patterns)? If so, is it worth tagging performance indicators with this information so that they can be sorted to see variations in performance patterns across time?UsersDocument who uses the data to learn from experience, improve performance, or make decisions and for what purpose_40bbac8a-1a52-11eb-81bb-e133f482ea00240.9.4Users—Who uses the data to learn from experience, improve performance, or make decisions and for what purpose? Are they responsible for achieving the task, accountable for accomplishing it, potentially supportive of the endeavor, likely to be consulted due to their expertise, or do they need to be kept informed? Are they in central offices, field, among delivery partners, public or the Congress? What does that information-using role imply for the form, timing, collection and dissemination of the collected indicators? FormattingDocument how the information will be used and what format is most conducive to such use_40bbaeb0-1a52-11eb-81bb-e133f482ea00240.9.5Format—How will people use the information and what format is most conducive to its use?Feedback & ImprovementDocument how are data collected or delivered and what methods are used to get feedback on the data to continually improve its quality or usefulness_40bbb072-1a52-11eb-81bb-e133f482ea00240.9.6Methods—How are data collected or delivered and what methods are used to get feedback on the data to continually improve its quality or usefulness? What challenges related to data collection may impact its use?Context & AnalysisConsider what analysis and evaluation will be needed to use the data to make decisions or improve performance_40bbb1e4-1a52-11eb-81bb-e133f482ea00240.9.7Context and Analysis—What analysis and evaluation will be needed in order to be able to use the data to make decisions or improve performance?CostConsider the cost and burden associated with collecting and analyzing the data_40bbb40a-1a52-11eb-81bb-e133f482ea00240.9.8Cost—How costly or burdensome is the data collection or analysis before it can be put to use.Performance FrameworkDevelop evidence to support decision-making, understand how well policies and programs are working, and improve performance._40bbb5e0-1a52-11eb-81bb-e133f482ea00240.10How do performance measurement and evaluation complement each other in the Federal Performance Framework? Performance measurement and evaluation should generally be viewed as two of the key tools available to help policymakers and program managers develop systematic evidence in order to support decision-making, understand how well policies and programs are working, and identify or promote possible changes that improve performance. Both evaluation and performance measurement generate information that falls along the continuum of evidence, serve as methods for systematic assessment, and aim to facilitate learning about and improve results of government activities. While often undertaken separately, collaboration between performance measurement and evaluation teams can lead to stronger evidence-building. For example, opportunities in which the two may work hand in hand include:QuestionsIdentify priority questions to be addressed by evaluations_40bbb748-1a52-11eb-81bb-e133f482ea00240.10.1Performance measurement can help identify priority questions to be addressed by evaluations, informing decisions about allocating evaluation resources.Predictive IndicatorsClarify which indicators are predictive of an activity’s success and should be tracked_40bbb9b4-1a52-11eb-81bb-e133f482ea00240.10.2Evaluation findings can clarify which indicators are predictive of an activity’s success and should be tracked in performance measurement.Metrics & EvaluationIdentify outliers in performance that warrant evaluation and provide context and potential explanations for variation_40bbbb9e-1a52-11eb-81bb-e133f482ea00240.10.3Performance measurement can identify outliers in performance (either poor or strong) that warrant evaluation, while evaluation can provide context and potential explanations for variation over time or across sites revealed by performance measurement.OutcomesCompare outcomes to confirm or refute whether they are attributable to programs_40bbbd10-1a52-11eb-81bb-e133f482ea00240.10.4When performance measures suggest that many participants in a program experience a certain outcome, evaluation can confirm (or refute) whether that is directly attributable to the program by comparing outcomes seen in a control or comparison group when possible.Program IndicatorsDetermine what types of indicators are important to program operators_40bbbfe0-1a52-11eb-81bb-e133f482ea00240.10.5EvaluatorsProgram OperatorsPerformance measurement can suggest to evaluators what types of indicators are important to program operators and might thus be useful to include in selecting evaluation measures.EvidenceConsider evidence in managing performance_40bbc1d4-1a52-11eb-81bb-e133f482ea00240.11What kind of evidence is considered appropriate for use in managing performance under the GPRA Modernization Act? For the purpose of managing performance under the GPRA Modernization Act of 2010 and the Federal Performance Framework, evidence as a general construct should be viewed and approached as the available body of facts or information indicating whether a belief or proposition is true or valid. This view of evidence does not displace the definitions of key terms provided for in the Foundations for Evidence-Based Policymaking Act of 2018 (“Evidence Act”) related to ‘evidence,’ ‘evaluation,’ ‘statistical activities,’ and ‘statistical purposes’ (see section 200.22), as information produced by ‘statistical activities’ with a ‘statistical purpose’ is potentially useful when assessing policies or programs. However, approaching evidence more broadly as a body of information in the context of performance management and the Federal Performance Framework is used to illustrate how it should be applied to support and advance the organizational performance and goals/objectives/outcomes articulated by agencies’ strategic and performance plans.Types & SourcesConsider quantitative and qualitative evidence from a variety of sources_40bbc350-1a52-11eb-81bb-e133f482ea00240.11.1Evidence can be quantitative or qualitative, and may come from a variety of sources, including foundational fact finding (e.g., aggregate indicators, exploratory studies, descriptive statistics, and other research), performance measurement, policy analysis, and program evaluations. Evidence has varying degrees of credibility, and the strongest evidence generally comes from a portfolio of high-quality, credible sources rather than a single study.ConclusionsUnderstand what conclusions can be drawn from the information, and equally important, what conclusions cannot be drawn_428f310e-1b19-11eb-aab2-1e641a83ea00240.11.2The credible use of evidence in decision-making requires an understanding of what conclusions can be drawn from the information, and equally important, what conclusions cannot be drawn. For example, multiple impact and implementation evaluations may provide strong evidence that a particular intervention is effective with a particular population, but it may be less definitive on how effective that intervention would be in other settings or with different populations. Quasi-experimental evidence from large, diverse samples of administrative data may address concerns about generalizability, but could lack definitive evidence on causality or be silent on important outcomes not captured in the administrative data.Descriptive AnalysesProvide context to examine societal and economic trends over time_428f3654-1b19-11eb-aab2-1e641a83ea00240.11.3Descriptive analyses from Federal statistical series provide context to examine societal and economic trends over time, but do not speak to program outcomes or impacts. Qualitative evidence can complement other evidence on outcomes and impacts by providing insight or context into how programs and practices can be successfully implemented with particular populations and under what circumstances.Tools & MethodsUse the appropriate tools and methods to answer questions of interest_428f391a-1b19-11eb-aab2-1e641a83ea00240.11.4This is a broad definition of evidence, and portfolios of evidence have varying degrees of credibility. The ‘intended use’ of evidence compels agencies to set expectations for levels of credibility that are aligned to and appropriate for the specific purpose for which the portfolio of evidence, including specific activities, will be used. It is important that agencies use the appropriate tools and methods to answer the questions of interest.Unquantifiable PerformanceMeasure performance when quantifiable performance goals cannot be developed_40bbc5bc-1a52-11eb-81bb-e133f482ea00240.12What can be used to measure performance in areas where quantifiable performance goals cannot be developed? When agencies cannot express a performance goal in a quantifiable form for a particular program, an “alternative form” performance goal or suite of indicators may be used instead.MilestonesConsider milestones_40bbc7b0-1a52-11eb-81bb-e133f482ea00240.12.1For example, milestones are often used as the basis of an alternative form performance goal.StandardsConsider third-party standards_428f3b5e-1b19-11eb-aab2-1e641a83ea00240.12.2In other cases, the attainment or maintenance of a third-party, established, standard can be a qualitative, measureable performance goal, such as obtaining an unmodified audit opinion on the agency’s financial statements.IndicatorsConsider a suite of indicators in lieu of performance goals_428f4054-1b19-11eb-aab2-1e641a83ea00240.12.3For certain programs, a suite of indicators in lieu of a performance goal will be appropriate.Evaluations & AssessmentsConsider evaluations and other assessment tools_428f42de-1b19-11eb-aab2-1e641a83ea00240.12.4Evaluations and other assessment tools may also be helpful.Other EvidenceInclude other evidence besides performance goals and indicators_40bbc94a-1a52-11eb-81bb-e133f482ea00240.13How should evidence, aside from performance goals and indicators, be incorporated in the Annual Performance Plan? Facts, Metrics, Analyses & EvaluationsInclude foundational facts, performance measurements, policy analyses, and program evaluations_40bbcbac-1a52-11eb-81bb-e133f482ea00240.13.1Evidence can include many sources, such as foundational fact finding, performance measurement, policy analysis, and program evaluation. Each of these sources can support agencies as they carry out their missions, though may do so in different ways.PortfoliosCombine sources to create a portfolio of evidence_428f44d2-1b19-11eb-aab2-1e641a83ea00240.13.2When combined, various sources of evidence can be combined to create a portfolio of evidence, each piece of which may provide information about a different aspect of a particular program, policy, or organization. While some evidence can help agencies determine whether a program is effective, others can address whether a policy is being implemented as intended or if an intervention is reaching its target population.Impacts & ImprovementsEnsure that resources are used in the smartest way possible to achieve intended impacts and continuous improvement_428f495a-1b19-11eb-aab2-1e641a83ea00240.13.3Evidence is a critical tool to help agencies ensure that resources are used in the smartest way possible to achieve intended impacts and continuous improvement. Thus, using evidence budget, management, programmatic, policy, and regulatory decisions is critical to make government work effectively.Budget SubmissionsUse evidence in FY 2022 budget submissions_428f4c84-1b19-11eb-aab2-1e641a83ea00240.13.4Federal AgenciesWhereas agencies will use their Annual Evaluation Plan to describe the significant evaluation that they plan to conduct following from the agency’s Learning Agenda, other priorities, and evaluations required by Congress, agencies should demonstrate the use of evidence throughout their FY 2022 budget submissions, in particular illustrating how evidence will support agency assessments of performance goals and implementation strategies in the Annual Performance Plan and 2021 final congressional budget justification.Needs & UsagesIdentify which evidence is needed and articulating how it will be used to measure progress_428f4ea0-1b19-11eb-aab2-1e641a83ea00240.13.5In the Annual Performance Plan, an agency’s use of evidence may focus on identifying which evidence is needed and articulating how it will be used to measure progress in achieving performance goals and objectives.Learning AgendaInclude evidence-building activities that go beyond performance measurement_428f530a-1b19-11eb-aab2-1e641a83ea00240.13.6This may be a part of the evidence-building activities outlined in the agency’s Learning Agenda, but the Learning Agenda will include evidence-building activities that go beyond performance measurement.Lower PrioritiesIdentify lower-priority program activities as a part of the FY 2022 performance planning process and final budget submission_40bbcdb4-1a52-11eb-81bb-e133f482ea00240.14What is required by the GPRA Modernization Act on lower-priority program activities? Agencies are required to identify lower-priority program activities as a part of the FY 2022 performance planning process and final budget submission.DisaggregationDisaggregate program activities to identify lower-priorities appropriate to the agency’s size_40bbcf4e-1a52-11eb-81bb-e133f482ea00240.14.1Small AgenciesIn cases where small agencies have only one program activity in the President’s Budget the agency may disaggregate the program activity for the purposes of identifying lower-priorities appropriate to the agency’s size.Program InformationPublish information on lower-priority program activities_40bbd1ce-1a52-11eb-81bb-e133f482ea00240.15How do agencies prepare and publish their lower-priority program activities to meet the reporting intent of this provision of the Act? Program Activities are the main organizing unit around which agency budget and funding requests are structured. As changes to resource and/or funding requests are reflected from one fiscal year to the next, agency budget submissions to OMB necessarily reflect those program activities upon which an agency places a lower-priority on the program activity relative to the funding requested in previous years. Thus, it is through the annual Budget process that facilitates production of the annual President’s Budget then that agencies prepare and publish their lower-priority program activities and therefore meet the reporting intent of this provision of the GPRA Modernization Act.Activity ListsApprove and finalize lower-priority program activity lists as part of publishing the President’s Budget_40bbd3cc-1a52-11eb-81bb-e133f482ea00240.15.1OMBAs in previous years, OMB works with agencies through the annual Budget process to approve and finalize lower-priority program activity lists as part of publishing the President’s Budget. Agencies must publish in the agency’s performance plan a clear reference to the President’s Budget for the agency’s lower priorities such as “The President’s Budget identifies the lower-priority program activities, as required under the GPRA Modernization Act, 31 U.S.C. § 1115(b)(10). The public can access the volume at: http://www.whitehouse.gov/omb/budget.”UpdatesMake available agency performance updates on the web_40bbd570-1a52-11eb-81bb-e133f482ea00240.16The GPRA Modernization Act requires each agency to make available on the web an update on agency performance. How and when will agencies publish the final Annual Performance Plan? Since the passage of the GPRA Modernization Act of 2010, agencies have been aligning the annual performance plan and report with the agency’s congressional budget justification, both to improve the accessibility and usefulness of agency performance reporting for stakeholders, as well as to reduce the burden of duplicative planning and reporting timelines.Budget JustificationsDevelop, align, and publish performance plans concurrent with final FY 2022 congressional budget justifications_40bbd7e6-1a52-11eb-81bb-e133f482ea00240.16.1The FY 2021 Annual Performance Plan should be developed to align with and publish concurrent with the agency’s final FY 2022 congressional budget justification. The timeline for development and publication of the FY 2022 Annual Performance Plan is in section 200. Agencies must first submit electronically the draft Annual Performance Plan to OMB for review in September 2020 by posting it on the agency’s Performance Submission Portal accessible from OMB’s Performance Portal page on MAX Community.Congressional NotificationNotify Congress electronically when performance plans are released publicly concurrent with the President’s Budget_428f55a8-1b19-11eb-aab2-1e641a83ea00240.16.2CongressAgency HeadsNotification to the Congress is transmitted electronically by the agency head when released publicly concurrent with the President’s Budget in February. When delivering notification to the Congress, agencies should also notify the President by emailing the OMB Director at performance@omb.eop.gov.Website PostingsPost performance plans on agency websites with hyperlinks on the Performance.gov site_428f5800-1b19-11eb-aab2-1e641a83ea00240.16.3Federal AgenciesAgencies shall post a copy of the final document on the agency’s website and provide a hyperlink to the plan on Performance.gov. Related submission questions should be emailed to performance@omb.eop.gov.Enterprise ArchitectureEnsure that enterprise architecture planning documents are consistent with agency goals and objectives_40bbd9f8-1a52-11eb-81bb-e133f482ea00240.17How does the Annual Performance Plan relate to the agency’s enterprise architecture? Once an agency’s performance plan is established, agencies should ensure that the enterprise architecture planning documents are consistent with achieving the agency goals and objectives.Strategic AlignmentAlign capital and enterprise architecture planning efforts to meet strategic objectives and performance goals_40bbdbb0-1a52-11eb-81bb-e133f482ea00240.17.1This will require direct alignment of the capital and enterprise architecture planning efforts to meet the strategic objectives and performance goals in agency strategic and annual performance plans, to the extent that information technology resources are critical to the achievement of those objectives and goals.Performance ReportsProvide information on progress achieving strategic goals and objectives_40bbde30-1a52-11eb-81bb-e133f482ea00240.18What is the Annual Performance Report (APR)? The Annual Performance Report (APR) provides information on the agency's progress achieving the goals and objectives described in the agency’s Strategic Plan and Annual Performance Plan, including progress on strategic objectives, performance goals and Agency Priority Goals. The term Annual Performance Report means the same as the performance section of the Performance and Accountability Report (PAR) published by agencies in November, or the Annual Performance Report that is published by agencies in February concurrent with their APP and congressional budget justifications._40bbe1d2-1a52-11eb-81bb-e133f482ea00Performance UpdatesMake updates on agency performance available on the web._aada5040-a90a-11e9-a082-d69e4d1a686a240.17The GPRA Modernization Act requires each agency to make available on the web an update on agency performance. How and when will agencies publish the final Annual Performance Plan? Since the passage of the GPRA Modernization Act of 2010, agencies have been aligning the annual performance plan and report with the agency’s congressional budget justification, both to improve the accessibility and usefulness of agency performance reporting for stakeholders, as well as to reduce the burden of duplicative planning and reporting timelines.Budget JustificationsDevelop, align, and publish FY 2021 Annual Performance Plans concurrent with final FY 2021 congressional budget justifications_aada5144-a90a-11e9-a082-d69e4d1a686a240.17.1The FY 2021 Annual Performance Plan should be developed to align with and publish concurrent with the agency’s final FY 2021 congressional budget justification. The timeline for development and publication of the FY 2021 Annual Performance Plan is in section 200.Submission & ReviewSubmit draft Annual Performance Plans to OMB electronically for review_428f5c42-1b19-11eb-aab2-1e641a83ea00240.17.2Federal AgenciesOMBAgencies must first submit electronically the draft Annual Performance Plan to OMB for review in September 2019 by posting it on the agency’s Performance Submission Portal accessible from OMB’s Performance Portal page on MAX Community.Congressional NotificationNotify Congress electronically when performance plans are released publicly concurrent with the President’s Budget_428f5f12-1b19-11eb-aab2-1e641a83ea00240.17.3CongressAgency HeadsNotification to the Congress is transmitted electronically by the agency head when released publicly concurrent with the President’s Budget in February.POTUS NotificationNotify the President by emailing the OMB Director_428f614c-1b19-11eb-aab2-1e641a83ea00240.17.4Federal AgenciesOMB DirectorPresident of the United StatesWhen delivering notification to the Congress, agencies should also notify the President by emailing the OMB Director at performance@omb.eop.gov.Website PostingsPost performance plans on agency websites and provide hyperlinks on Performance.gov site_428f65ac-1b19-11eb-aab2-1e641a83ea00240.17.5Agencies shall post a copy of the final document on the agency’s website and provide a hyperlink to the plan on Performance.gov. Related submission questions should be emailed to performance@omb.eop.gov.Priority GoalsReport progress quarterly on Priority Goals of the Administration on the Performance.gov site._40bbe3a8-1a52-11eb-81bb-e133f482ea00240.19The GPRA Modernization Act requires “more frequent updates of actual performance on indicators that provide data of significant value to the Government, Congress, or program partners at a reasonable level of administrative burden.” How will agencies meet this requirement? Agencies report progress quarterly on Priority Goals of the Administration on Performance.gov.CAP GoalsUpdate progress on Cross-Agency Priority (CAP) Goals quarterly_40bbe6e6-1a52-11eb-81bb-e133f482ea00240.19.1CAP Goal LeadersGSAOMBIn addition, Cross-Agency Priority (CAP) Goal progress is updated quarterly by CAP Goal Leaders in coordination with the GSA, OMB and contributing agencies.Other GoalsReport on other performance goals more frequently than annually_428f6908-1b19-11eb-aab2-1e641a83ea00240.19.2All agencies are encouraged to report performance on their other performance goals more frequently than annually, if cost-effective, valuable, or required by Executive Order or other OMB memoranda and guidance to agencies. Each agency should determine the areas and kinds of information where more frequent data will lead to better decisions by the public, field offices, and delivery partners that generate more value and/or lower cost.Agency WebsitesProvide more frequent performance updates on agency websites_428f6b56-1b19-11eb-aab2-1e641a83ea00240.19.3Agencies should use their own websites to provide more frequent performance updates, where cost effective or required, and ...APIsEstablish Application Programming Interface (API) feeds directly to the Performance.gov site_428f6fde-1b19-11eb-aab2-1e641a83ea00240.19.4explore opportunities for establishing Application Programming Interface (API) feeds directly to Performance.gov. See “actionable information/data of significant value” in sections 200, 240.9, and 280.Performance UpdatesMake available performance updates on agency websites_40bbe9b6-1a52-11eb-81bb-e133f482ea00240.20The GPRA Modernization Act requires each agency to make available on the website of the agency an update on agency performance. When are agencies required to publish the Annual Performance Report? With the passage of the GPRA Modernization Act of 2010, agencies have been aligning the annual performance plan and report with the agency’s congressional budget justification, both to improve the accessibility and usefulness of agency performance reporting for stakeholders, as well as to reduce the burden of duplicative planning and reporting timelines.2020 ReportsPublish FY 2020 Annual Performance Reports with FY 2022 Annual Performance Plans and final FY 2022 congressional budget justifications_40bbeb8c-1a52-11eb-81bb-e133f482ea00240.20.1Federal AgenciesAgencies should publish their FY 2020 Annual Performance Report with their FY 2022 Annual Performance Plan and final FY 2022 congressional budget justification in February 2021.Small agenciesPublish small agency Annual Performance Reports (APR) for FY 2020 on agency websites and provide links from the Performance.gov site_428f72ea-1b19-11eb-aab2-1e641a83ea00240.20.2Small agenciesSmall agencies will maintain the flexibility to publish the Annual Performance Report (APR) for FY 2020 on the agency’s website as a Performance and Accountability Report (PAR) (November 2020), but are encouraged to produce a combined annual performance plan and report. Agencies will also be required to link to the APR from Performance.gov.Performance.gov SitePublish content from Strategic Plans, Annual Performance Reports, and Annual Performance Plans through the Performance.gov site_428f7542-1b19-11eb-aab2-1e641a83ea00240.20.3Large Federal AgenciesCFOAct AgenciesConcurrent with the release of the agency’s final FY 2021 congressional budget justifications, large (CFOAct) agencies are required to publish content from the Strategic Plan, Annual Performance Report, and Annual Performance Plan through Performance.gov. See sections 200 and 210 for more information on development timelines and required content for each plan and report.CoordinationWork with OMB and Congress in the preparation of performance reports_40bbef24-1a52-11eb-81bb-e133f482ea00240.21OMBCongressFederal AgenciesHow are agencies expected to work with OMB or Congress in the preparation of the performance report?Presentation & ConcernsDiscuss the presentation and work out any concerns related to performance reports_40bbf2c6-1a52-11eb-81bb-e133f482ea00240.21.1Agency StaffOMBWhen preparing an agency-specific Annual Performance Report, agency staff and OMB should discuss the presentation and work out any concerns, if needed, in advance of the submission of the reports to the Congress.Congressional OutreachReach out to the Congress to obtain input on how to improve the communication of performance information_428f79d4-1b19-11eb-aab2-1e641a83ea00240.21.2CongressFederal AgenciesAgencies are encouraged to reach out to the Congress, where possible, to obtain input on how they might improve their communication of performance information to the Congress. Agencies should work with their legislative affairs offices to determine the best ways to consult with the Congress.DeliveryDeliver reports to the President, Congress and the public_40bbf4b0-1a52-11eb-81bb-e133f482ea00240.22President of the United StatesCongressThe PublicHow do agencies deliver the report to the President, Congress and the public?2020 ReportsMake FY 2020 performance reports available on agency websites_40bbf762-1a52-11eb-81bb-e133f482ea00240.22.1For the FY 2020 performance report, agencies should make Annual Performance Reports available on the agency website.HyperlinksPublish hyperlinks to agency performance reports on the Performance.gov site_428f7d1c-1b19-11eb-aab2-1e641a83ea00240.22.2A hyperlink to the agency performance report will also be published via Performance.gov.POTUS NotificationPost reports on the agency websites and email hyperlinks to the Director of OMB_428f7ff6-1b19-11eb-aab2-1e641a83ea00240.22.3President of the United StatesDirector of OMBFederal AgenciesFor notification to the President, agencies should post final reports on the agency’s website, and email the Director of OMB at email performance@omb.eop.gov with the hyperlink to the published report.Congressional NotificationNotify Congress electronically of the availability of the Annual Performance Reports_428f853c-1b19-11eb-aab2-1e641a83ea00240.22.4CongressAgency HeadsThe report notification must be from the head of the agency, but may be transmitted electronically by his or her delegate.Other SignatoriesAn agency may add other signatories, such as the Deputy Secretary, Chief Operating Officer, Performance Improvement Officer or Chief Financial Officer, as necessary to the transmittal, thus recognizing a shared responsibility within the agency.Deputy SecretariesChief Operating OfficersPerformance Improvement OfficersChief Financial OfficersAgencies should notify the Congress electronically of the availability of the final Annual Performance Report.Transmittal LettersAddress transmittal letters to the specified congressional leaders_428f888e-1b19-11eb-aab2-1e641a83ea00240.22.5Speaker of the House of RepresentativesPresident of the SenatePresident Pro Tempore of the SenateTransmittal letters to the Congress are addressed to the Speaker of the House of Representatives, the President of the Senate and the President pro tempore of the Senate.Congressional TransmittalsSend congressional transmittals electronically_428f8b40-1b19-11eb-aab2-1e641a83ea00240.22.6Copies of the congressional transmittal are sent electronically, unless otherwise requested in print by the Congress, to the chair and ranking minority members of the budget committees, relevant authorization and oversight committees, appropriation subcommittees, and the chair and ranking minority member of the Senate Committee on Homeland Security and Governmental Affairs and the House Oversight and Government Reform Committee.Notification MethodsWork with legislative affairs and congressional staff to determine optimal ways to notify Congress_428f9108-1b19-11eb-aab2-1e641a83ea00240.22.7Legislative Affairs StaffCongressional StaffAgencies should work with their legislative affairs and congressional staff to determine the optimal way to transmit notification to the Congress.Secret InformationMake secret information available in classified appendices_428f94aa-1b19-11eb-aab2-1e641a83ea00240.22.8If an agency performance update includes any program activity or information that is specifically authorized under criteria established by an Executive Order to be kept secret in the interest of national defense or foreign policy and is properly classified, the head of the agency will make such information available in a classified appendix.Plans & ReportConsolidate Annual Performance Reports with Annual Performance Plans._40bbf9b0-1a52-11eb-81bb-e133f482ea00240.23Should agencies consolidate the Annual Performance Report with the Annual Performance Plan? Generally, yes (see 210.4 for exceptions during Presidential election and transition years). To streamline agency planning and reporting of performance information for stakeholders and more efficiently manage duplicative planning and reporting timelines, agencies are strongly encouraged, but not required, to consolidate the Annual Performance Plan and Annual Performance Report.HyperlinksProvide hyperlinks to the Performance.gov site from agency websites where Annual Performance Reports are published_40bbfb90-1a52-11eb-81bb-e133f482ea00240.23.1Agencies are required to provide a hyperlink to Performance.gov from the agency’s website where the Annual Performance Report is published.ContentArticulate how the work of the agency benefits the public, enable the public to understand the actions agencies have taken to make progress and explain what the agency is doing to improve performance._40bbfe4c-1a52-11eb-81bb-e133f482ea00240.24What information should the Annual Performance Report contain? Annual Performance Reports should clearly articulate how the work of the agency benefits the public, enable the public to understand the actions agencies have taken to make progress and explain what the agency is doing to improve performance.FormattingFormat the FY 2020 APR by strategic goal and objective_40bc009a-1a52-11eb-81bb-e133f482ea00240.24.1The APR must address the content established in section 210, with agencies encouraged to format the FY 2020 APR by strategic goal and objective.ProgressInclude summary assessments of progress on strategic objectives_428f97a2-1b19-11eb-aab2-1e641a83ea00240.24.2The FY 2020 APR should include a summary assessment of progress as described in sections 210 and 270 for strategic objectives, and focus on comparing and reporting results achieved against performance goals and associated measures and indicators established in the agency’s Annual Performance Plan.Discontinued GoalsReport the FY 2020 results of any performance goals and indicators that will be discontinued_428f9d4c-1b19-11eb-aab2-1e641a83ea00240.24.3Agencies must still report the FY 2020 results of any performance goals and indicators that will be discontinued by the agency in future performance plans, and targets for dropped measures no longer need to be set or included in the FY 2022 Performance Plan.Selective ApplicationsSelectively apply parts of Annual Performance Reports as applicable_40bc028e-1a52-11eb-81bb-e133f482ea00240.25What other parts selectively apply to the Annual Performance Report, as applicable? The following parts selectively apply to agencies.Non-Federal RolesBriefly describe contributions by non-Federal entities in preparing performance reports_40bc07ca-1a52-11eb-81bb-e133f482ea00240.25.1Information on use of non-Federal parties. The GPRA Modernization Act of 2010 states that preparation of an annual report is an inherently governmental function. However, the report should include an acknowledgment of the role and a brief description of any significant contribution made by a non-Federal entity in supporting preparation of the report.Classified AppendicesWithhold classified appendices from the public_428fa0c6-1b19-11eb-aab2-1e641a83ea00240.25.2Classified appendices not available to the public. Agencies that conduct classified activities may prepare a classified appendix for the Annual Performance Plan. Also, if an agency believes that reporting of actual performance will impede goal achievement, a non-public appendix may be prepared for the Annual Performance Report. Agencies should consult with OMB to determine whether such an appendix is necessary.Completeness, Reliability & QualityAssess the completeness, reliability, and quality of performance data._40bc0a36-1a52-11eb-81bb-e133f482ea00240.26How should agencies assess the completeness, reliability, and quality of performance data reported in the Annual Performance Report? The GPRA Modernization Act of 2010 requires agencies to prepare information on the reliability of data presented. Agencies may develop a single data verification and validation appendix used to communicate the agency’s approaches, and/or may also choose to provide information about data quality wherever the performance information is communicated (e.g., websites). Agencies should discuss their verification and validation techniques with their respective OMB Resource Management Office, if necessary. The transmittal letter included in Annual Performance Reports must contain an assessment by the agency head of the completeness and reliability of the performance data presented and a description of agency plans to improve completeness, reliability, and quality, where needed.LimitationsIdentify significant and known limitations of the data_40bc0c2a-1a52-11eb-81bb-e133f482ea00240.26.1Data limitations. In order to assess the progress towards achievement of performance goals, the performance data must be appropriately valid and reliable for intended use. Significant or known data limitations should be identified to include a description of the limitations, the impact they have on goal achievement, and the actions that will be taken to correct the limitations. Performance data need not be perfect to be valid and reliable to inform management decision-making. Agencies can calibrate the accuracy of the data to the intended use of the data and the cost of improving data quality. At the same time, significant data limitations can lead to bad decisions resulting in lower performance or inaccurate performance assessments. Examples of data limitations include imprecise measurement and recordings, incomplete data, inconsistencies in data collection procedures and data that are too old and/or too infrequently collected to allow quick adjustments of agency action in a timely and cost-effective way. Verification & ValidationEnsure the accuracy and reliability of performance information_40bc0fe0-1a52-11eb-81bb-e133f482ea00240.26.2GAOThe GAO defines verification as a process of checking or testing performance information to assess other types of errors, such as errors in keying data. The GAO defines validation as an effort to ensure that data are free of systematic error or bias and that what is intended to be measured is actually measured. The GAO information can be found in the GAO publication GAO/GGD10.1.20 The Results Act, An Evaluator’s Guide to Assessing Agency Annual Performance Plans. See also GAO’s Verification and Validation of Performance Data.Federal AgenciesAgencies should have in place verification and validation (V&V) techniques that will ensure the completeness and reliability of all performance measurement data contained in their Annual Performance Plans and reports as appropriate to the intended use of the data.Performance Improvement Council (PIC)In addition, the Performance Improvement Council (PIC) through an interagency working group, has developed a Data Quality Maturity Model and Example Practices guidebook to assist agencies in improving their data quality programs over time. Copies of the guidebook can be obtained by emailing the PIC directly at PICStaff@pic.gov.Verification and validation. Verification and validation of performance data support the general accuracy and reliability of performance information, reduce the risk of inaccurate performance data, and provide a sufficient level of confidence to the Congress and the public that the information presented is credible as appropriate to its intended use... The guidance that follows provides agencies with a list of reasonable V&V criteria that when applied should increase the level of confidence the Congress and the public have in the performance information presented.FactorsConsider the following verification and validation factors_40bc1256-1a52-11eb-81bb-e133f482ea00240.26.2.1Agency internal assessments. Agencies are encouraged to consider the verification and validation factors outlined below.Standards & Procedures_40bc1468-1a52-11eb-81bb-e133f482ea00240.26.2.1.11. Standards and procedures: * Source data are well defined, documented; definitions are available and used. * Collection standards are documented/available/used. * Data reporting schedules are documented/distributed/followed. * Supporting documentation is maintained and readily available. * Collection staff are skilled/trained in proper procedures.Data Entry & Transfer_40bc1774-1a52-11eb-81bb-e133f482ea00240.26.2.1.22. Data entry and transfer: * Data entry methodology is documented and followed. * Data are verified as appropriate to the needed level of accuracy. * Procedures for making changes to previously entered data are documented and followed. * Data are available when needed for reporting, learning and critical decision making cycles. * Data entry staff are skilled and trained in proper procedures.Data Integrity_40bc19f4-1a52-11eb-81bb-e133f482ea00240.26.2.1.33. Data integrity: * Whenever possible, data should be returned to data suppliers with value added so that data suppliers benefit from the analysis of the data and are engaged to improve its quality over time. * Third-party measurement is often preferable to self-measurement. * Administrative data that is used for other purposes and validated by its use can be a source of high-quality performance data at a relatively low cost.Data Quality_40bc1c06-1a52-11eb-81bb-e133f482ea00240.26.2.1.44. Data quality and limitations: * Accuracy limits of all data are appropriate to their intended use. * Data limitations are explained and documented. * Method for handling anomalous data is established and used, not just to isolate data artifacts but also to search for promising practices to validate and possibly solve problems needing attention. * Third party evaluations are conducted. * Use of externally controlled data is documented.Oversight & Certifications_40bc1f30-1a52-11eb-81bb-e133f482ea00240.26.2.1.55. Oversight and certifications: * Accountability for data accuracy exists in a responsible employee’s performance standards. * Responsible officials certify that procedures were followed each reporting period. * Responsible officials certify that data accuracy has been checked each reporting period.AssessmentsConsider external assessments to determine data or information gaps and changes in performance trends_40bc21ba-1a52-11eb-81bb-e133f482ea00240.26.3External Assessments. External assessments, such as evaluations and peer reviews can be helpful to determine data or information gaps and whether changes in performance trends are attributable, in whole or in part, to agency action or to other factors. Agencies are expected to consider the available evidence, including any available evaluation results, when conducting this analysis. As appropriate, such analysis should consider whether the goals and indicators have been validated through 1) research to be well correlated with ultimate outcomes; 2) implications of available research on the appropriateness of the measure; and 3) the relative strength or weakness of the measure overall. Agencies should determine when and how to complement performance measurement with evaluations or other high-quality external assessments to improve the quality and comprehensiveness of the data being reported. AuditsDo not require the use of audits for performance data_40bc23e0-1a52-11eb-81bb-e133f482ea00240.26.4External Audits. It is important to note the GPRA Modernization Act of 2010 does not require the use of audits for performance data contained in Annual Performance Plans or reports.ScopeApply judgment when deciding which performance indicators will be verified and validated_40bc28b8-1a52-11eb-81bb-e133f482ea00240.26.5Scope. Because most agencies process a large amount of performance measurement data, agencies should apply judgment when deciding which performance indicators will be verified and validated. Agencies should consider priorities, spending, GAO high risk lists, IG reports and management challenges.FrequencyDetermine the appropriate frequency of validation and verification needed for the intended use_40bc2b4c-1a52-11eb-81bb-e133f482ea00240.26.6Frequency of Validation and Verification. Agencies should determine the appropriate frequency of validation and verification needed for the intended use and should allocate appropriate resources to carry out validation and verification on an appropriately periodic basis. Data presented annually should typically be validated annually or biennially. AccountabilityHold agency heads accountable for the accuracy and reliability of performance data_40bc2db8-1a52-11eb-81bb-e133f482ea00240.26.7Agency HeadsAgency Head Responsibility. Agency heads are officially accountable for the accuracy and reliability of performance data. The agency head shall include in the transmittal letter of the agency’s APR a brief statement on the completeness and reliability of the performance data, and on what data limitations exist.Data IntegrityIntegrate internal control over reporting (ICOR) with enterprise risk management (ERM) processes_40bc31aa-1a52-11eb-81bb-e133f482ea00240.27How does the update to OMB Circular No. A-123, Appendix A, Management of Reporting and Data Integrity Risk affect agency preparation of the Annual Performance Plan and Annual Performance Report? OMB Circular No. A-123, Appendix A provides updated guidance to agencies that integrates internal control over reporting (ICOR) with enterprise risk management (ERM) processes, and assurances over internal control. Specifically, the 2018 update to OMB Circular No. A-123, Appendix A expanded internal controls from financial reporting (internal controls over financial reporting, e.g., ICOFR) to all reporting objectives (internal controls over reporting, e.g., ICOR).RiskAnalyze risk across portfolio views of agency objectives_40bc34ac-1a52-11eb-81bb-e133f482ea00240.27.1By aligning the updated Appendix A to the agency’s ERM processes, agency management should apply their analysis of risk in the agency’s risk profiles across a portfolio view of the agency’s objectives (e.g., Strategic, Operations, Reporting, and Compliance Objectives – see OMB Circular No. A-123) when deciding where internal controls will be most effectively employed to those reporting objectives where inaccurate, unreliable, or outstanding reporting would significantly impact the agency’s ability to accomplish its mission and performance goals or objectives.Goals & IndicatorsApply ICOR at the individual performance goal and indicator level_428fa404-1b19-11eb-aab2-1e641a83ea00240.27.2Importantly, management decisions to apply ICOR should not be done against the entire Annual Performance Plan or Annual Performance Report. Rather, management decisions to apply ICOR should be made at the individual performance goal and indicator level, applying only in those instances where: * there is a significant risk that a material reporting error may impact achievement of the agency’s mission objectives; and * application of ICOR is likely to cost effectively mitigate that risk.2020-07-102020-10-30https://www.whitehouse.gov/wp-content/uploads/2018/06/a11.pdfOwenAmburOwen.Ambur@verizon.net