Components of AI in ControlKPMG’s AI in Control ~ KPMG developed the AI in Control framework to help organizations drive greater confidence and transparency through tested AI governance constructs, as well as methods and tooling along the AI lifecycle, from strategy through evolution. By design, this framework addresses the inherent risks outlined in the sections above and it includes some of the key recommendations and leading practices for establishing AI governance, performing AI assessments, and building continuous AI monitoring and visualizations.Controlling AI ~ An effective framework will help organizations gain confidence in their AI technology. Such approach should dig deep into AI at the enterprise and individual model to help ensure that key trust imperatives are integrated and controlled throughout. It should continuously assess and maintain control over sophisticated, evolving algorithms by putting in place methods, controls, and tooling that secure the trust anchors along the lifecycle, from strategy through evolution. It should also provide clear guidance for the organizations—stakeholders across various management and oversight functions.KPMG_c3ceb4c2-41c1-11ec-bf91-61161083ea00LeadersThis report is for leaders involved in the world of Artificial Intelligence and Machine Learning algorithms. The business and compliance imperative to understand and be confident in AI technologies has reached critical mass. This paper explains the urgency and describes methods and tools that can help leaders govern their AI programs.OrganizationsUncovering the full potential of your AI ~ Today’s organizations rely heavily on algorithm-based applications to make critical business decisions. While this unlocks opportunities, it also raises questions about trustworthiness. As we enter an age of governance by algorithms, organizations must think about the governance of algorithms to build trust in outcomes and achieve the full potential of artificial intelligence.Martin SokalskiPrincipal, Advisory, Digital Lighthouse, KPMG USSwami ChandrasekaranHead of Digital Solutions Architecture, KPMG USSander KlousPartner, Data & Analytics, KPMG NetherlandsAchievement of the full potential of artificial intelligence_c3ceb652-41c1-11ec-bf91-61161083ea00To build trust in outcomes resulting from the application of algorithms_c3ceb742-41c1-11ec-bf91-61161083ea00TrustAlgorithms can be destructive when they produce inaccurate or biased results, an inherent concern amplified by the black box facing any leader who wants to be confident about their use. That is why there is hesitancy in handing over decisions to machines without being confident in how decisions are made and whether they’re fair and accurate. This is an AI trust gap.FairnessAccuracyConfidenceGaining confidence with your AIExplainabilityThe benefits of AI will only fully emerge when algorithms become explainable (and, hence, understandable) in simple language, to anyone.TransparencyThe trust gap exists because there is no transparency of AI; instead, there is an inherent fear of the unknown surrounding this technology. Gaining trust also involves understanding the lineage of the AI models and protecting them (and data that forms them) from different types of adversarial attacks and unauthorized use. Critical business decisions made by AI affect the brand—and consumer trust in the brand—and they can have an enormous impact on the well-being or safety of consumers and citizens. No one wants to say “because the machine said so.” No one wants to get AI wrong.GovernanceGovern AI_c3ceb86e-41c1-11ec-bf91-61161083ea001Design & ControlsDevelop AI design criteria and establish controls in and environment that fosters innovation and flexibility._c3ceb954-41c1-11ec-bf91-61161083ea001.1Governance & GapsAssess current AI-related governance framework and perform gap analysis to identify opportunities and areas that need to be updated._c3ceba44-41c1-11ec-bf91-61161083ea001.2Risk ManagementIntegrate risk management framework to identify and prioritize business-critical algorithms and incorporate an agile risk mitigation strategy to address cybersecurity, integrity, fairness, and resiliency considerations during design and operation._c3cebb48-41c1-11ec-bf91-61161083ea001.3LifecyclecDesign and implement an end-to-end AI governance and an operating model across the entire lifecycle: strategy, building, training, evaluating, deploying, operating, and monitoring AI._c3cebc4c-41c1-11ec-bf91-61161083ea001.4Solutions & InnovationDesign a governance framework that delivers AI solutions and innovation through guidelines, templates, tooling, and accelerators to quickly, yet responsibly, deliver AI solutions._c3cebd64-41c1-11ec-bf91-61161083ea001.5CriteriaDesign and set up criteria to maintain continuous control over algorithms without stifling innovation and flexibility._c3cebe54-41c1-11ec-bf91-61161083ea001.6AssessmentAssess AI_c3cebf30-41c1-11ec-bf91-61161083ea002Diagnostic ReviewsConduct a diagnostic review of an enterprise AI program and governance to evaluate the current state and applicability of existing governance elements to AI as well as current operating model and readiness for AI at scale._c3cebff8-41c1-11ec-bf91-61161083ea002.1This will include a capability and maturity assessment, as well as roadmap and recommendations for achieving target state.AlgorithmsConduct assessment of individual AI and ML algorithms: testing of controls, evaluation of design, implementation and operation of the algorithm based on four trust anchors—integrity, explainability, fairness and resilience._c3cec0c0-41c1-11ec-bf91-61161083ea002.2MonitoringContinuously monitor AI in dashboards_c3cec19c-41c1-11ec-bf91-61161083ea003MetricsCreate full visibility into metrics related to the trust imperatives including key performance and risk indicators including Board, Executive, and Program level reporting focused on key relevant AI KPIs and KRIs._c3cec264-41c1-11ec-bf91-61161083ea003.1MonitoringEnable continuous monitoring of key controls and metrics—what is working (or not) across your AI/ML models._c3cec340-41c1-11ec-bf91-61161083ea003.2TrendsProvide view of the upward/downward trend over a time period, based on controls and testing._c3cec444-41c1-11ec-bf91-61161083ea003.3Responses & CorrectionsHave ability to respond and correct issues as they arise._c3cec55c-41c1-11ec-bf91-61161083ea003.4For example, bias is introduced in the learning model, or prohibited features are being used in decision making, etc.Models & ProgramsConduct an assessment of your AI model(s) or a health check of your broader enterprise AI program._c3cec66a-41c1-11ec-bf91-61161083ea003.52021-11-09https://advisory.kpmg.us/articles/2019/controlling-ai.htmlOwenAmburOwen.Ambur@verizon.net