How to Approach Analytics Project DevelopmentAnalytics in City Government: How the Civic Analytics Network Cities Are Using Data to Support Public Safety, Housing, Public Health, and Transportation -- If a department, agency, or city is considering using analytics, there are a few ways to get started. While some organizations follow general standards of practice to provide a step-by-step guide on the key phases of project development, others follow a technical framework to identify the degree to which organizational and/or data resources will support a given project. Drawing on the various approaches, process guides, and methodologies developed by Civic Analytics Network cities and partners, this report highlights five key steps that cities can replicate to develop their own analytics projects: (1) identify the problem; (2) assess data readiness; (3) scope the project; (4) pilot the project; and (5) implement and scale the model.The first step the Civic Analytics Network recommends -- before considering what data an organization has available -- is establishing a clear understanding of the problem to be addressed by a given analytics project. Determining data readiness or maturity is critical, but before an analytics project can even be scoped, it is important to ensure that the project’s objective is core to the performance or needs of the implementing organization; data-driven policymaking is not data use for the sake of data use. After working with departments to identify a mission-critical problem, analytics experts or data scientists can proceed with identifying data readiness, scoping and piloting the project, and so on. From forecasting future needs to overcoming staffing or resource shortages, to condensing vast and disparate information into actionable insights, analytics can be a powerful tool in improving city governance, but it is most effective when used to enhance and support the efforts and priorities of city personnel. There is no better way to ensure that than by positioning data scientists within government to work with departments on tackling key issues collaboratively. It is important to note that many analytics teams are small, nascent offices with restrictive funding resources for data experts to introduce the value of data analytics to their cities’ bureaucracy; many new analytics teams or hires are established thanks to the support of the bully pulpit, and the mayor can play an important role in prioritizing data use to address policy needs in local communities.Civic Analytics NetworkCAN_b8e28808-9671-11e8-ab24-f9edabe6e336Based at the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School and funded by the Laura and John Arnold Foundation, the Civic Analytics Network is an affiliation of chief data officers from the largest and most innovative municipalities in United States. They are open data stewards, internal consultants, and performance managers. The network seeks to advance the use of data and analytics in municipal governance through facilitation of in-person meetings among members and production of research and documented best practices. For more on the Civic Analytics Network, please visit civicanalyticsnetwork.org.Jessica A. GoverAuthor -- Jessica A. Gover is a research assistant and writer at the Harvard Ash Center supporting the Civic Analytics Network. Prior to joining the Ash Center, she conducted research on new approaches to data- and tech-enabled policy in the U.S. federal government, innovative public-private partnerships in New Zealand, and civic participation in the American jury system. She holds a Master’s degree from the University of Chicago and received her B.A. at Trinity College. Ash Center for Democratic Governance and InnovationThe Roy and Lila Ash Center for Democratic Governance and Innovation advances excellence and innovation in governance and public policy through research, education, and public discussion. The Ford Foundation is a founding donor of the Center. Three major programs support the Center's mission: The Program on Democratic Governance, the Innovations in Government Program, and the Rajawali Foundation Institute for Asia.Harvard Kennedy SchoolLaura and John Arnold FoundationThe Civic Analytics Network project, including this paper, is generously funded by the Laura and John Arnold Foundation. This report is an independent work product and the views expressed are those of the author and do not necessarily represent those of the funder.Stephen GoldsmithThe Civic Analytics Network is led by Professor Stephen Goldsmith, director of the Innovations in American Government Program at the Ash Center. Professor Goldsmith provided crucial insights for the development of the policy recommendations in this report.Ash Center StaffA sincere thank you to the following Ash Center staff members and Civic Analytics Network partners for their feedback and contributions to this report:Katherine Hillenbrandproject manager, Data-Smart City Solutions, Ash Center at the Harvard Kennedy SchoolLauren Haynesassociate director, Center for Data Science and Public Policy at the University of ChicagoSean ThorntonCivic Analytics Network program advisor, Ash Center at the Harvard Kennedy SchoolJoseph Walshdata scientist, Center for Data Science and Public Policy at the University of ChicagoBetter governance_b8e28b3c-9671-11e8-ab24-f9edabe6e336To spur and guide the effective development of analytics projects in cities across the U.S. _b8e28c18-9671-11e8-ab24-f9edabe6e336ProblemsIdentify the Problem_b8e28cf4-9671-11e8-ab24-f9edabe6e3361Data ScientistsDeveloping an analytics project typically places data scientists in an internal consultant role; by working with a department or agency to identify their key issues or problems, data scientists can support mission-critical needs.City of ChicagoFor example, the City of Chicago's Applied Analytics Guide prioritizes working in partnership with departments and agencies across government as part of its ten-step process to develop advanced predictive analytics projects. Chicago was an early leader in city-level data-driven policy as one of the first cities to create a chief data officer (CDO) position, which is housed within the city's Department of Innovation and Technology (DoIT). In Chicago, direct engagement with departments across city government helped DoIT build relationships that created an environment in which department leaders see the DoIT team as a resource to help them explore new solutions to key problems. DoIT does not prescribe data solutions, but rather supports the priorities of department leaders to optimize performance and/or service delivery in areas that the leaders identify as areas of need. Sometimes, however, public exposure of an underperforming service area can spur an analytics “intervention.” For example, following media coverage on restaurant inspections, DoIT helped craft an analytics model to help optimize restaurant inspections (see "Public Health" on page 21 for further detail).CitiesOther cities similarly prioritize this problem identification phase as a way of developing meaningful partnerships with government agencies.Analytics TeamsMany Civic Analytics Network analytics teams are situated in their respective city governments as discrete groups tasked with supporting and collaborating with other departments or agencies by bringing data-driven insights and expertise to bear on key issues or areas of need. Identifying a critical problem that can be supported or alleviated by analytics is challenging, but it is an important first step in structuring a successful analytics project. While data may abound, matching an area of need with the right data resources within an organization is vital._b8e28f74-9671-11e8-ab24-f9edabe6e336ReadinessAssess Data Readiness_b8e29050-9671-11e8-ab24-f9edabe6e3362Center for Data Science and Public Policy (DSaPP)The University of Chicago's Center for Data Science and Public Policy (DSaPP), a Civic Analytics Network partner, created a "Data Maturity Framework" to help prepare prospective project leaders for the development process. The framework provides an effective structure to help determine data readiness for an organization considering a new analytics project.Project ManagersDrawing on its team's experience sourcing project proposals, DSaPP observed a trend: most prospective project managers referred to a desire to make use of tons of unused data. While having ready-made or already assembled data is a good start for any analytics project, DSaPP needed to help project managers understand that successful analytics projects begin with the identification not of unused data, but of critical issues in need of data-driven solutions.Data ScientistsOnce that issue area or policy need was identified, then data scientists could help project managers assess data readiness and begin structuring and scoping an analytics model.Determining data readiness is a key facet of Civic Analytics Network cities' approaches to analytics and a critical precondition to scoping any project. The success of an analytics project depends not only upon whether there is a need for data analytics, but also, and more importantly, on having the right personnel, data collection and storage practices, and stakeholder buy-in within and outside of the department or agency... DSaPP’s Data Maturity Framework consists of a questionnaire and scorecards to identify the technology, data, and organizational readiness within a department. The framework consists of a questionnaire and survey to assess readiness and three scorecard matrices on: (1) problem definition, (2) data and technology readiness, and (3) organizational readiness. Each scorecard helps organizations identify where they fall on a spectrum of four categories ranging from leading to lagging in terms of data readiness. Scorecard categories include: how data is stored; what is collected; privacy and documentation practices; personnel; data use policy; and buy-in from staff, data collectors, leadership, intervener, and funder. Assessing data maturity can also be approached from the macro level -- for a mayor or chief data officer to assess the enterprise-wide maturity of municipal data, it is important to consider broad-scale questions such as how a government consumes data and how leadership uses data to make policy decisions. Determining the data maturity of city-wide practices is challenging, but cities across the Civic Analytics Network are leading by example through data-driven governance. ProblemsDefine the problem(s)_b8e29140-9671-11e8-ab24-f9edabe6e3362.1Data & TechnologyAssess data and technology readiness_d1e6e15c-968d-11e8-a1ea-1996a2e6e3362.2OrganizationsAssess organizational readiness_d1e6e3d2-968d-11e8-a1ea-1996a2e6e3362.3ProjectsScope the Project_b8e29230-9671-11e8-ab24-f9edabe6e3363ChicagoChicago uses the Research Question Evaluation Criteria for scoping its predictive analytics projects and determining which projects are best suited for development. Similar to DSaPP's project scoping steps, Chicago’s evaluation criteria guides project managers through key issues to better situate a prospective analytics project for success.New OrleansThere is no one "right" approach to project scoping: in New Orleans, the city's performance and data expertise team developed a project criteria framework to help scope projects;DoITin the City of Chicago DoIT prioritizes prospective projects by using a framework of research question evaluation criteria.DSaPPDSaPP's Data Science Project Scoping Guide has been a particularly successful model for approaching project development and was featured as a pre-conference workshop at the Ash Center’s Inaugural Summit on Data-Smart Government in November, 2017. DSaPP’s Data Science Project Scoping Guide was developed for prospective "Data for Social Good" fellowship projects (a program that supports aspiring data scientists by connecting them with real-world problems) in order to facilitate a continuous pool of well-developed projects that could be successfully deployed within DSaPP's program cycle.Once a department's data readiness is assessed, it is time to scope the project... Because each project needs to be scoped thoroughly enough for a data-use agreement, the project scoping steps help expedite project development by providing a concise framework for prospective projects. This project scoping approach helps managers focus on understanding what data is available, who the key stakeholders are for providing and using that data, and how that data is being considered to provide insights into a city governance problem. DSaPP's four steps for project scoping are:GoalsDefine the goal(s) of the project._b8e29320-9671-11e8-ab24-f9edabe6e3363.1ActionsIdentify the actions/interventions the project will inform._b8e29410-9671-11e8-ab24-f9edabe6e3363.2DataDetermine the data requirements._b8e29500-9671-11e8-ab24-f9edabe6e3363.3What data do you have access to internally? What data do you need? What can you augment from external and/or public sources?AnalysisDetermine what analysis needs to be done._b8e29604-9671-11e8-ab24-f9edabe6e3363.4Does it involve description, detection, prediction, or behavior change? How will the analysis be validated?PilotingPilot the Project_b8e296e0-9671-11e8-ab24-f9edabe6e3364Analytics TeamsNo matter how well prepared an analytics team is, sometimes -- whether the problem lies in a key variable, an assumption built into the algorithm, or the project’s general approach -- the pilot just does not perform as expected.Civic Analytics NetworkWhile information on how best to pilot a municipal analytics project is limited, the frameworks, criteria, and guidelines developed by Civic Analytics Network cities and partners can serve as helpful resources and provide useful examples of how to approach and, ultimately, scale an analytics project.Public SectorPiloting an analytics projects, like any effort to innovate in the public sector, is somewhat at odds with the bureaucratic preference for consistency and risk avoidance, but it is a critical phase that can yield important insights for improving performance when it is time for implementation on a larger scale.UK GovernmentMoreover, starting with small-scale pilots can help limit risk and demonstrate clear results. In an assessment of its pilot practices, the UK Government notes that "Once embarked upon, a pilot must be allowed to run its course. Notwithstanding the familiar pressures of government timetables, the full benefits of a policy pilot will not be realized if the policy is rolled out before the results of the pilot have been absorbed and acted upon." The report goes on to argue that "pilots should be regarded less as ad hoc evaluations than as early stages in a continuing process of accumulating policy-relevant evidence."Piloting an analytics project is "the stuff of innovation." This is where the trial and error of testing a new project happens... Piloting also allows for much needed course corrections to help better transition efforts in project-scoping to implementation; adjusting project parameters during the pilot phase can increase the likelihood of success at implementation and beyond. _b8e297ee-9671-11e8-ab24-f9edabe6e336Implementation & ScalingImplement and Scale the Model_b8e298f2-9671-11e8-ab24-f9edabe6e3365Data-Smart City SolutionsThe Ash Center's Data-Smart City Solutions and the Civic Analytics Network have begun to capture early lessons from city-level analytics projects, and as more data-driven decision-making projects mature and are replicated, insights into how to improve and scale these leading projects will grow.Civic Analytics NetworkCurrently, Civic Analytics Network cities are working to replicate models implemented by other member cities, and while some of those leading examples are highlighted in this report, further research and use cases on those replicated projects are forthcoming.Research and literature on implementing and scaling analytics projects remain limited, and given the variability of structures, budgets, and objectives for analytics projects, identifying generalizable practices for scaling these projects is challenging._b8e299e2-9671-11e8-ab24-f9edabe6e3362018-08-02https://ash.harvard.edu/files/ash/files/281995_hvd_ash_paper_v4.pdfOwenAmburOwen.Ambur@verizon.net