Audit of the Federal Risk and Authorization Management Program, Program Management Office's Goals and ObjectivesWhy We Performed This Audit -- This audit was included in the GSA Office of Inspector General Fiscal Year 2017 Audit Plan. Theinitial focus for this audit was to review GSA’s Federal Risk and Authorization ManagementProgram (FedRAMP), Program Management Office’s (PMO) authorization and accreditationprocess for third party providers. However, during our survey work, we determined that risksare present at the FedRAMP PMO level; therefore, we evaluated the FedRAMP PMO’s goals andobjectives to determine if they are sufficient to assess its effectiveness in accomplishing itsmission. What We Found -- The FedRAMP PMO has not established an adequate structure comprising its mission, goals,and objectives for assisting the federal government with the adoption of secure cloud services.Specifically, the mission statement does not provide a clear direction for the FedRAMP PMO;objective statements are missing key attributes; and the alignment of the mission, goals, andobjective statements makes it difficult to determine if the FedRAMP PMO is meeting its missionin an effective manner.GSA Office of Inspector GeneralGSAOIG_f188f9a0-5613-11e9-bf41-a57b291b279eU.S. General Services AdministrationAcquisition and Information Technology Audit OfficeAudit Team -- This audit was managed out of the Acquisition and Information Technology Audit Office and conducted by the individuals listed below:Sonya D. PanzoAssociate Deputy Assistant Inspector General for AuditingMichelle L. WestrupAudit ManagerBruce E. McLeanAuditor-In-ChargeOffice of Management and Budget (OMB)Background -- On December 8, 2011, the Office of Management and Budget (OMB) issued Security Authorization of Information Systems in Cloud Computing Environments, which resulted in a collaborative effort between GSA and other federal entities, including the Departments of Defense and Homeland Security, to develop FedRAMP.FedRAMPFedRAMP is a government-wide program designed to increase the pace at which the federal government adopts cloud computing services. It was developed to standardize: (1) the process of how the Federal Information Security Management Act of 2002 (FISMA) applies to cloud computing services;and (2) the way the government conducts security assessments, authorizations, and continuous monitoring of cloud computing services. FedRAMP has several key stakeholders:OMBOMB has outlined the key components of FedRAMP and its operationalcapabilities. OMB is also responsible for defining the requirements for executive branch departments and agencies using FedRAMP, and providing oversight of the program.Department of Homeland Security (DHS)Within FedRAMP’s operationalframework, DHS coordinates cybersecurity operations including incident response.Joint Authorization Board (JAB)The JAB consists of the Chief Information Officers from the Department of Defense, DHS, and GSA. The JAB members define and update FedRAMP security authorization requirements in accordance with FISMA and DHS guidance, review authorization packages, and grant Provisional Authorities to Operate for cloud services.Department of DefenseExecutive Branch Departments & AgenciesThe executive branch departments and agencies are required to use FedRAMP when conducting risk assessments, reviewing security authorizations, and granting Authorities to Operate for use of cloud services.FedRAMP PMOThe day-to-day operations of FedRAMP are performed by the FedRAMP PMO, which is managed by GSA’s Federal Acquisition Service, Technology Transformation Services.The FedRAMP PMO’s responsibilities include:• Creating a process for executive branch departments, agencies, and CSPs to adhere to FedRAMP security authorization requirements;• Prioritizing authorization requests for JAB review;• Establishing a centralized, secure repository of various authorization packages of cloud services that executive branch departments and agencies can leverage;• Collaborating with the National Institute of Standards and Technology to develop and implement a program to accredit third-party assessment organizations to provide independent reviews of how CSPs implement the FedRAMP requirements; and• Developing templates for executive branch departments and agencies to use to satisfy FedRAMP security authorization requirements.Alan B. Thomas, Jr.Federal Acquisition Service Commissioner -- The Federal Acquisition Service Commissioner agreed with the audit recommendations.The federal government rapidly adopts cloud computing services_f188fb4e-5613-11e9-bf41-a57b291b279eTo determine if the FedRAMP PMO’s goals and objectives are sufficient to assess its effectiveness in accomplishing its mission._f188fc02-5613-11e9-bf41-a57b291b279eEfficiencySince the program began, government agencies and CSPs have expressed concerns regarding FedRAMP’s efficiency, effectiveness, and transparency. They have noted that the Authority to Operate review and approval process is expensive and time consuming. A FedRAMP PMO official informed us that the PMO has been confronted with external pressures to speed up the FedRAMP process along with maintaining quality standards. While the FedRAMP PMO has taken action to address some of these concerns, additional action is needed to strengthen the PMO to better meet the needs and requirements of the program.EffectivenessTransparencyMissionRevise the FedRAMP PMO’s mission statement to a concise, singular statement._f188fcac-5613-11e9-bf41-a57b291b279e1According to OMB Circular No. A-11, Preparation, Submission, and Execution of the Budget (2017), a mission statement should be a brief, easy-to-understand narrative, usually no more than a sentence long. It should define the basic purpose of a program and be expressed in the broad context of a problem, need, or challenge. It enables employees to see how their work contributes to the broader mission and is a key component of a program’s strategic plan, explaining why the program exists and what it does, while bringing the program into focus...The FedRAMP PMO’s mission statement does not conform to the succinct format prescribed under OMB Circular No. A-11, but is rather a listing of directives established in the OMB memorandum. Therefore, the FedRAMP PMO’s mission statement is not presented in a way that is focused or easily communicated, creating confusion as to its central purpose and vision of what needs to be accomplished. Accordingly, the FedRAMP PMO should establish a concise, clearly defined mission statement._f188fd56-5613-11e9-bf41-a57b291b279eObjectivesRevise the FedRAMP PMO’s objectives to make them more specific and measurable._f188fe00-5613-11e9-bf41-a57b291b279e2The FedRAMP PMO’s objective statements are missing key attributes, hindering the understanding of what is to be accomplished and the effective communication of program results. Objective statements should be specific and measurable to assist in assessing performance.In the Standards for Internal Control in the Federal Government, the U.S. Government Accountability Office defines various attributes of objectives. Among other things, these standards provide that objectives should be:• Defined in specific terms, such as what is to be achieved and how it will be achieved, so they are understood at all levels of the entity.• Defined in measurable terms so that performance toward achieving those objectives can be assessed. Measurable objectives are generally free of bias and subjectivity.• Supported by performance measures that are appropriate for evaluating the program’s performance in achieving the objective.• Aligned with the organization’s mission, strategic plan, and performance goals...The FedRAMP PMO established eight objectives to assess progress against its four goals, aslisted below in Figure 1. [Figure excluded]The FedRAMP PMO’s goals and objectives provided in Figure 1 were documented in the FedRAMP PMO’s Goal Tracker and published on its website. The goal tracker contained abbreviated objective statements intended to provide a convenient way for the public to view the goals, as well as the objectives and corresponding performance metrics at a glance.However, these simplified objective statements excluded information necessary for understanding and measuring the FedRAMP PMO’s progress. Specifically, these objective statements were not defined in specific and measureable terms.SpecificityDefine objectives in specific terms_f188fea0-5613-11e9-bf41-a57b291b279e2.1Objectives Not Defined in Specific Terms -- We determined that some of the FedRAMP PMO’s objective statements may not be easily understood by stakeholders and the public. For example, the FedRAMP PMO has two objectives that include similar language – “Increase cloud services working with FedRAMP” and “Increase authorized cloud services.” These two objectives, as written, do not contain adequate detail todifferentiate between them. While the FedRAMP PMO management team reviews its objectives internally on a routine basis, the difference between these objectives may not be evident or understood by those externally without further explanation. Vague objective statements may limit the understanding and clear communication of what the FedRAMP PMO strives to achieve.MetricsDefine objectives in measurable terms_f188ff5e-5613-11e9-bf41-a57b291b279e2.2Objectives Not Defined in Measurable Terms -- We also found that some of the FedRAMP PMO’s objective statements do not allow for a clear assessment of its performance. Many of the objectives do not explicitly identify a specific outcome, which would clearly outline what is to be achieved. For instance, the objective, “Promote better understanding” does not state what is to be understood or define how better understanding will be achieved. Similarly, “Establish FedRAMP Readiness as a viable readiness mechanism for CSPs” does not provide for clear measurement. The terms “viable” and “readiness” could be viewed as subjective and it is not clear from the objective how success will be measured. While the FedRAMP PMO was able to provide us with some targets related to these objectives, the targets were not specifically outlined as part of its objectives. Objectives not defined in measurable terms negatively affect the FedRAMP PMO’s ability to effectively communicate progress. Accordingly, the PMO should create objectives that are specific and measurable.Strategic AlignmentReview the FedRAMP PMO’s mission, goals, and objectives to ensure they align in a cohesive manner._f1890008-5613-11e9-bf41-a57b291b279e3The FedRAMP PMO’s mission, goals, and objectives do not align in a manner where its goals and objectives clearly support the mission. The lack of clear alignment of these elements inhibits the FedRAMP PMO’s ability to assess its effectiveness in achieving its mission. OMB Circular No. A-11 outlines the importance of aligning an organization’s mission, goals, and objectives. An organization’s mission directly supported and aligned with strategic goals, which are each supported and aligned with one or more strategic objectives, clearly outlines how each element is linked to the program’s intended mission and purpose. When the FedRAMP PMO provided us with its goals and objectives, it clearly outlined how they were aligned; however, it did not include a connection to the specific mission statement segment. To evaluate the FedRAMP PMO’s overall structure, we requested a document showing the alignment including the mission. The FedRAMP PMO provided us with this document, which aligned the mission, reported by mission segment, to the goals. From that document, we noted: (1) misalignments between mission segments and goals, and (2) mission segments and a goal without corresponding objectives. GoalsEnsure that strategic goals are appropriately aligned with mission segments._f18900bc-5613-11e9-bf41-a57b291b279e3.1Mission Segments and Goals Are Misaligned -- Some of the FedRAMP PMO’s mission segments and goals were misaligned. An example is depicted in Figure 2 ...The mission segment “Introduce an innovative policy approach to developing trusted relationships between executive branch departments and agencies and cloud service providers” is aligned with the goal of “Transform security authorizations.” This mission segment specifically focuses on relationships between government entities and CSPs. However, the goal of transforming security authorizations focuses more on increased efficiency of the process, not necessarily relationship building. To further demonstrate this, two of the three supporting objectives listed under this mission segment focus on increasing authorization efficiency rather than relationships between government entities and CSPs. Therefore, we determined that this mission segment and its corresponding goal are misaligned.ObjectivesEnsure that each longer-term goal is supported by near-term objectives with SMART metrics._f18901b6-5613-11e9-bf41-a57b291b279e3.2Mission Segments and Goal with No Supporting Objectives -- We also noted that two of the FedRAMP PMO’s mission segments and one goal do not have supporting objectives as shown below in Figure 3...OMB Circular No. A-11 states that objectives should generally encompass the agency’s mission and scope of responsibilities. Goals and objectives should directly support the program’s mission. The lack of appropriate alignment among the FedRAMP PMO’s goals, objectives, and mission inhibits the PMO’s ability to assess its effectiveness in achieving its mission. Accordingly, the FedRAMP PMO should review its mission, goals, and objectives to ensure that they are aligned in a cohesive manner.2019-03-212019-04-03https://www.oversight.gov/sites/default/files/oig-reports/A170023_1.pdfOwenAmburOwen.Ambur@verizon.net