Recommendations for Making Federal Agencies Evidence-Based: The Key Role of Learning AgendasAs agencies proceed with developing learning agendas and evidence-building plans in coming years in the United States, they will continue to also learn how to improve the implementation process. Based on the experience of early adopting agencies and our own experience with implementing a user-centric design process, there are a series of clear priorities and recommendations for fulfilling the intent of the learning agenda in coming years.IBM Center for The Business of GovernmentIBMCBG_c0becfe0-8215-11eb-b357-d2f53183ea00Daniel J. ChenokExecutive Director, IBM Center for The Business of Government Leanne HaseldenPartner and Practice Leader, Cognitive & Analytics SupportKathryn NewcomerCo-Author | George Washington UniversityKarol OlejniczakCo-Author | SWPS University of Social Sciences and HumanitiesNick HartCo-Author | Data FoundationLearning agendas succeed_c0bed1a2-8215-11eb-b357-d2f53183ea00To offer recommendations for fulfilling the intent of the learning agenda_c0bed288-8215-11eb-b357-d2f53183ea00LearningLearning agendas are designed to provide an internal management tool, intended primarily foragency decision makers accountable for successful planning and delivery of programs andpolicies. The plan can also be used externally to promote evidence-building that may befunded by nongovernmental partners. However, the production of new knowledge and evidenceis funded or supported, the emphasis on ensuring ultimate users are part of the developmentprocess—and meaningfully engaged throughout the process—will likely result in a more useful,relevant, and salient plan.User-CentricityIn federal agencies, there are traditionally few user-centered approaches applied that result inmeaningful engagement and participatory processes. Many federal agencies rely on the use ofadvisory committees, which provide expert feedback on topics requested by the agency andmay meet on a recurring basis. Nearly all federal agencies rely on mechanisms like the FederalRegister for requesting comments in a unidirectional discussion. Some agencies, especiallyones issuing regulatory actions, may engage more directly with stakeholders in the regulatedcommunity by holding workshops or public forums for soliciting feedback on draft proposalsprior to finalizing an action. While these activities are attempts at engagement, they oftenresult in agencies and individuals listening, rather than collaborating with participants onpolicy design.ParticipationWhen developing learning agendas, additional active processes should be considered asmeaningful ways to engage stakeholders, such as including individuals from other parts ofgovernment in the formulation and use of the plans. The traditional, passive feedback mechanisms of the 20th century provided structured opportunities for input from stakeholders, suchas via written comments on draft plan. However, to develop an effective and meaningfullearning agenda that will create a sustained focus on evidence-building activities necessitatesa more active strategyUnderstandingIn recent years there has been a renewed interest in using design approaches in publicpolicy and public management. Although the notion of policy as a design is not new, theapplication of highly collaborative and user-centered design to develop complex publicmanagement processes is novel. This type of design method stresses focusing onunderstanding users of the particular solution, co-designing the solution with them, makingthe discussed concepts tangible. User-centered design approaches are well-aligned with thepurpose and intent of the learning agendas.IterationTimingProvide federal agencies flexibility to develop their learning agendas after thequadrennial strategic planning process has been completed_c0bed3be-8215-11eb-b357-d2f53183ea00Recommendation 1Congress should provide federal agencies flexibility to develop their learning agendas after thequadrennial strategic planning process has been completed, not concurrently. The EvidenceAct’s requirements for issuing an agency’s learning agenda in parallel with the issuance of itsstrategic plan present major obstacles for alignment of the information needs to the real questions faced by the agency. The concurrent deadline presents obstacles to incorporating the rightneeds and timelines for annual evaluation plans or agency statistical activities. A recognition byCongress that the inclusion “in the strategic plan” may include technically publishing in themonths after finalizing the strategic plan would support agencies in aligning missions, goals, priorities, and needs in a timely and accurate way._c0bed4c2-8215-11eb-b357-d2f53183ea00ResourcesAllocate resources for learning needs and evidence-building activities_c0bed594-8215-11eb-b357-d2f53183ea00Recommendation 2CongressPresidentSenior Agency LeadersCongress, the president, and senior agency leaders should prioritize allocating resources forlearning needs and evidence-building activities. Once agencies have formulated meaningfullearning agendas to address agency evidence needs, resources must be allocated to actuallybuild the requisite evidence. Resources may be allocated practically through direct appropriations, set-aside funding, reallocations and reprogrammings, or other means._c0bed65c-8215-11eb-b357-d2f53183ea00PublicationUpdate guidance to direct agencies to publish learning agendas publicly_c0bed71a-8215-11eb-b357-d2f53183ea00Recommendation 3Office of Management and BudgetThe Office of Management and Budget should update guidance to direct agencies to publishlearning agendas publicly. The Evidence Commission prioritized the principle of transparency, aconcept that is reflected in OMB’s Evaluation Standards and the underlying motivation for thelearning agenda construct itself. Agencies should publish and share their learning agendaswidely with the public and relevant stakeholders, including oversight officials and the researchcommunity. Wide distribution of agency learning agendas may foster external researchers inaddressing information needs, the prioritization of new resources, and other benefits. WhenOMB issues its new guidance on learning agendas, consistent with direction from President JoeBiden to OMB in January 2021, the OMB director should specify that learning agendas beshared and published as open resources._c0bed7e2-8215-11eb-b357-d2f53183ea00EngagementInvolve stakeholders when developing learning agendas_c0bed9cc-8215-11eb-b357-d2f53183ea00Recommendation 4Agency evaluation officers should strategically involve stakeholders when developing learningagendas using proactive engagement approaches tailored to the agency ecosystem. Everyagency will vary in the breadth of their existing stakeholder engagement processes, but agenciesshould consider the full range of possibilities for pursuing engagement and avoid the tendency torely exclusively on written feedback provided to the agency in response to a Federal Registernotice. Participatory processes such as implementing the Learning Agenda Design Sprint process, holding forums, workshops, and public meetings, and involving advisory committees canbe productive in broadening the feedback and dialogue on the learning and evidence needs ofdecision makers._c0bedada-8215-11eb-b357-d2f53183ea00LADSConsider applying the Learning Agenda Design Sprint (LADS) model to more effectively engage stakeholders_c0bedbb6-8215-11eb-b357-d2f53183ea00Recommendation 5Federal agency evaluation officers should consider applying the Learning Agenda Design Sprint(LADS) model to more effectively engage stakeholders when developing learning agendas. Themodel described in the previous section has been tested in different organizational contexts. Itcarries clear benefits when implemented as intended: staff and stakeholders can develop ashared understanding of the program decision journey and key information needs, as well asalign the demand and supply of evidence for decision making at both a strategic and operationallevel. LADS can be implemented in person or remotely, in a digital environment. They can beused to develop creative, user-centered input at different stages of a federal agency’s learningagenda life cycle._c0bedc88-8215-11eb-b357-d2f53183ea002021-03-10http://www.businessofgovernment.org/sites/default/files/Making%20Federal%20Agencies%20Evidence-Based.pdfOwenAmburOwen.Ambur@verizon.net