An Intelligence in Our Image: The Risks of Bias and Errors in Artificial IntelligenceThis report illustrates some of the shortcomings of algorithmic decisionmaking, identifies key themes around the problem of algorithmic errors and bias, and examines some approaches for combating these problems. This report highlights the added risks and complexities inherent in the use of algorithmic decisionmaking in public policy. The report ends with a survey of approaches for combating these problems [which are documented as goals and objectives in this StratML rendition]Machine learning algorithms and artificial intelligence systems influence many aspects of people's lives: news articles, movies to watch, people to spend time with, access to credit, and even the investment of capital. Algorithms have been empowered to make such decisions and take actions for the sake of efficiency and speed. Despite these gains, there are concerns about the rapid automation of jobs (even such jobs as journalism and radiology). A better understanding of attitudes toward and interactions with algorithms is essential precisely because of the aura of objectivity and infallibility cultures tend to ascribe to them.RAND CorporationRAND_7b60f2d8-1ba4-11e7-afde-ecb26551e223Osonde A. OsobaAssociate Engineer; Professor, Pardee RAND Graduate School -- Osonde Osoba (pronounced "oh-shOwn-day aw-shAw-bah") is an associate engineer at the RAND Corporation and a professor at the Pardee RAND Graduate School. He has a background in the design and optimization of machine learning algorithms. He has applied his expertise to diverse policy topics such as epidemiology, defense acquisition & science and technology policy. His more recent focus has been on data privacy and accountability in algorithmic systems and artificial intelligence.William Welser IVDirector, Engineering and Applied Sciences Department; Senior Management Scientist; Faculty Member, Pardee RAND Graduate School; Co-Director of RAND's Impact Lab -- William (Bill) Welser IV is the director of the Engineering and Applied Sciences (EAS) Research Department at the RAND Corporation, a professor at Pardee RAND Graduate School, and co-director of RAND's Impact Lab. As director of the EAS Research Department, he is responsible for roughly 200 world-class professional research staff.Better understanding of attitudes toward and interactions with algorithms._7b60f3e6-1ba4-11e7-afde-ecb26551e223To address risks and complexities inherent in the use of algorithmic decisionmaking in public policy. _7b60f472-1ba4-11e7-afde-ecb26551e223Critical Services & SubsystemsIdentify critical services and subsystems that require "human-in-the-loop" decisionmaking._7b60f4ea-1ba4-11e7-afde-ecb26551e2231Selection criteria may include high-risk systems or systems that require special accountability.ConstraintsLimit the role of artificial agents in these systems to a strictly advisory capacity._7b60f56c-1ba4-11e7-afde-ecb26551e2231.1AuditingEmphasize the need for the ability to audit the results of these advisory artificial agents._7b60f5e4-1ba4-11e7-afde-ecb26551e2231.2AuditsEstablish best practices for auditing algorithmic decisionmaking aids designed for use in government services and policy domains._7b60f65c-1ba4-11e7-afde-ecb26551e2232Establish best practices for auditing algorithmic decisionmaking aids designed for use in government services and policy domains (e.g., the criminal justice system and social services administration).Unaccredited AlgorithmsAvoid usage of unaccredited third-party black-box algorithmic solutions._7b60fdaa-1ba4-11e7-afde-ecb26551e2232.1This should include specific guidance discouraging the use of unaccredited third-party black-box algorithmic solutions.DisparitiesAddress disparate impacts._7b60fea4-1ba4-11e7-afde-ecb26551e2232.2Audit procedures should also address questions of disparate impact.Disclosure_7b60fea5-1ba4-11e7-afde-ecb26551e2233NotificationsAdopt standardized disclosure practices to inform stakeholders when decisions affecting them are algorithmically generated._7b60fea6-1ba4-11e7-afde-ecb26551e2233.1Reviews & AppealsInstitute standard procedures for appealing or reviewing such decisions._7b60fea7-1ba4-11e7-afde-ecb26551e2233.2Disparate ImpactInvest science research funds in research on algorithmic disparate impact. _7b60fea8-1ba4-11e7-afde-ecb26551e2234Best PracticesEngage with the commercial artificial intelligence community to share best practices._7b60fea9-1ba4-11e7-afde-ecb26551e2234.1Commercial Artificial Intelligence CommunityDiversityAddress diversity issues in the science, technology, engineering, and math educational pipeline._7b60feaa-1ba4-11e7-afde-ecb26551e2235TrainingUpdate accreditation guidelines for engineering schools to include more training on the effects of technology on society and sociotechnical systems more generally._7b60feab-1ba4-11e7-afde-ecb26551e2235.1Engineering Schools2017-04-07https://www.rand.org/pubs/research_reports/RR1744.htmlOwenAmburOwen.Ambur@verizon.net