The Art & Craft of being A Chief Data OfficerBecause principles outlive tactics, I offer the following [documented as goals and objectives in this StratML rendition]Michael J. ConlinMJC_64dff8e8-7146-11e9-860d-6349ad9d818fChief Data Officer, U.S. Department of DefenseU.S. Department of Defense_64dffa32-7146-11e9-860d-6349ad9d818fTo outline principles to guide chief data officers_64dffb22-7146-11e9-860d-6349ad9d818fSponsorsIdentify executive sponsors who want insights to drive improvement._64dffc80-7146-11e9-860d-6349ad9d818f1Executive SponsorsReady, willing & able -- The number one criteria for launching a Data Science initiative is that you know the name and title of the executive sponsor who wants to use your insights so they can drive improvement. If you don't know that, you're not ready to take action. Whoever owns the organization's results has accountability for value capture in Data Science.AgendasAlign with their agendas._64dffd52-7146-11e9-860d-6349ad9d818f1.1Find them. Talk to them.Sweet SpotsPrioritize and sequence opportunities by value._64dffe1a-7146-11e9-860d-6349ad9d818f2Target the sweet spot -- There are always new opportunities popping up and tempting you. How do you choose the best things to work on? In any organization, the sweet spot is in the overlap of quantifiable value for the customer (for DoD that’s the taxpayer), quantifiable value for the employee (for DoD that’s the warfighter), quantifiable value for the stakeholders (for DoD that's the President & Congress) and quantifiable value for our own executives (see Ready, Willing and Able). There will be a limited number of opportunities that satisfy all requirements, and these can be prioritized / sequenced by value.CustomersQuantify value for customers._64dffeec-7146-11e9-860d-6349ad9d818f2.1TaxpayersEmployeesQuantify value for employees._64dfffbe-7146-11e9-860d-6349ad9d818f2.2WarfightersStakeholdersQuantify value for stakeholders._64e00086-7146-11e9-860d-6349ad9d818f2.3PresidentCongressExecutivesQuantify value for executives._64e0018a-7146-11e9-860d-6349ad9d818f2.4DoD ExecutivesDestinationsFavor direction over speed._64e0025c-7146-11e9-860d-6349ad9d818f3Direction > speed -- Direction is more important than speed. If you're pointed in the wrong direction, it doesn't matter how fast you're traveling. Inversely, if you're locked on to your desired destination, all progress is positive, no matter how slow you're going.DirectionChoose the direction._64e0032e-7146-11e9-860d-6349ad9d818f3.1Choosing direction over speed is a simple concept, but in practice it can be difficult to follow (see Target the sweet spot).GoalsSeek marginal improvements while bearing the ultimate goal in mind._64e00414-7146-11e9-860d-6349ad9d818f4Better is better -- You may have an ultimate goal in mind for each Data Science exercise, but any improvement is better than no improvement. Better is better.ChunkingBreak work into chunks._64e004f0-7146-11e9-860d-6349ad9d818f4.1Adopt a DevSecOps mindset, with appropriate techniques and tooling. Breaking work into chunks, and then build up the big picture.Minimum ProductsProduce minimum-viable-products, lots of them._64e005c2-7146-11e9-860d-6349ad9d818f4.2The cost of a prediction is already $0.IterationIterate your way to the ultimate goal. _64e006a8-7146-11e9-860d-6349ad9d818f4.3LearningLearn by doing._64e00784-7146-11e9-860d-6349ad9d818f5Learn by doing -- When it comes to Data Science, we're all winging it. Even the experts. Especially the experts; we're dealing with cutting edge problems and there's no playbook. This is applied Research & Development. We're figuring it out as we go.Risk TakingTake risks to increase knowledge and skills. _64e00856-7146-11e9-860d-6349ad9d818f5.1If a material percentage of your Data Science projects aren't failing, you're not taking the risks you need to take in order to increase your knowledge and skills.TestingFoster a data-driven, test-and-learn culture._64e00978-7146-11e9-860d-6349ad9d818f5.2Let the data speak to the situation.CommunicationCommunicate bad news quickly and without shame. _64e00a5e-7146-11e9-860d-6349ad9d818f5.3What we learn along the way, and how widely we share that learning, is as important as the immediate outcome.EngagementActively engage in the broader Data Science community to leverage best-practices and learning from completely different fields._64e00b3a-7146-11e9-860d-6349ad9d818f5.4Data Science CommunityCelebrationCelebrate both the outcome and the learning._64e00d60-7146-11e9-860d-6349ad9d818f5.5Next StepsTry to figure out what you should do next, not what the right answer is._64e00e3c-7146-11e9-860d-6349ad9d818f5.6UncertaintyDiscover the path for your organization to navigate through uncertainty._64e00f04-7146-11e9-860d-6349ad9d818f6Walk toward the uncertainty -- While the general direction for Data Science is clear, the exact path needed to navigate through uncertainty needs to be discovered organization by organization. No one is smart enough to see clearly through the opportunities and challenges, you have to navigate your way through.Value ChainWork through the organization's full value chain._64e00fe0-7146-11e9-860d-6349ad9d818f6.1You need to work through the organization's full value chain, use case by use case and data story by data story, to get to value from Data Science.EvidenceProvide executives with evidence to make difficult tradeoffs between nuanced alternatives._64e010bc-7146-11e9-860d-6349ad9d818f6.2ExecutivesYour executive sponsors don’t need help with obvious decisions, they need help making difficult tradeoffs between nuanced alternatives. They want evidenced-based help navigating uncertainty.Change ManagementProvide clear recommendations on the next steps to take._64e01184-7146-11e9-860d-6349ad9d818f6.3Change management is critical to the success of many data science initiatives. Give people clear recommendations of the next steps to take.ProblemsStart with the problems to be solved._64e01260-7146-11e9-860d-6349ad9d818f7There is no truth in data -- Don't start with the data you've collected; start with the problem you want to solve.AdequacyDetermine when the available data is 'good enough' upon which to proceed._64e014cc-7146-11e9-860d-6349ad9d818f7.1Learn to identify when you’ve got 'good enough' data to proceed, and how to manage the risks of acting on it.RiskManage the risks of acting on the available data._64e015a8-7146-11e9-860d-6349ad9d818f7.2You have less data than you think. You certainly don't have any data for things we haven't seen before. Ask yourself, "Does the historical data you’ve got represent the future you want to create?" It probably doesn’t. The old adage 'garbage-in → garbage-out" applies to Data Science.BiasLeverage diversity to guard against hypothesis bias._64e016b6-7146-11e9-860d-6349ad9d818f7.3Diversity is critical -- diversity of data, people, mindsets, backgrounds, models. Diversity helps protect you against hypothesis bias, which often leads you to shape data to an existing agenda.InsightStrive to gather more of the right data to gain better insight._64e01788-7146-11e9-860d-6349ad9d818f8Data > algorithms -- Additional data is more useful than a better algorithm, (Pro-tip - make sure it’s the right data). AI/ML algorithms are widely open-sourced, so they're not a source of relative advantage. More data, when carefully chosen, typically leads to better insights.AlgorithmsTreat algorithms as processes and test them._64e01864-7146-11e9-860d-6349ad9d818f8.1While I'm on the topic of algorithms, they're not products, they're processes. We will never be sure what a process does until we run it. IncentivesEnsure the incentives for AI/ML algorithms are properly aligned with the objectives._64e01954-7146-11e9-860d-6349ad9d818f8.2The biggest problem is always misaligned incentives; AI/ML algorithms are incentive-seeking machines.LeadershipLead by example._64e01a44-7146-11e9-860d-6349ad9d818f9Buoys not boundaries -- You don’t want to leave tool choices wide open but there's no value in publishing guidelines you cannot enforce. Forget all about policies, memos, directives, standards, reference architectures. The office building you work in is already wallpapered with those artifacts, and nobody is paying any attention to them. Don’t add to the wallpaper. Lead by example instead.Utilities & Shared ServicesDefine and implement process models, technical architectures, data formatting standards, and policy statements and make them widely available as utilities or shared services._64e01b20-7146-11e9-860d-6349ad9d818f9.1Rather than publishing prescriptions -- defining a particular process model or a technical architecture for tooling pipeline or a data formatting standard or a policy statement -- implement them and make them widely available as a utility or shared service. Speed thrills and people will use your shared service if it's instantly available rather than having to go through a procurement process.Data PipelinesEstablish a series of vetted pipelines that are very well tested and very easy to consume._64e01c42-7146-11e9-860d-6349ad9d818f9.2Data Science TeamsIf a Data Science team doesn't know what they want to do in order to create a pipeline, they can leverage the central service very easily. This has the added benefits of creating: freedom of maneuver for data talent (they can go anywhere in the organization without retraining); creating freedom of maneuver for proven data workloads (they can be reused anywhere in the organization without retooling); and bargaining leverage with tools suppliers.The concept is called "buoys, not boundaries". You establish a series of vetted pipelines that are very well tested and very easy to consume.AlternativesEmpower data scientists to explore alternatives in their own environment._64e01d32-7146-11e9-860d-6349ad9d818f9.3On the other hand, if people do want to explore alternatives, they can go explore in their own environment. That's critical in the Data Science world, because it is changing constantly. You don't want to be a barrier to innovation. You don't want to apply your central team’s scarce resources to assessing a queue of tools or techniques. By the time each request makes it to the front of the queue some alternate will already be available in the market. Give your organization the ability to explore and innovate.AI/MLUse AI/ML to assist people do their work better and faster._06018e64-71ae-11e9-b009-6a70af9d818f10Help Wanted -- First, let's assuage a concern. Neither Artificial Intelligence nor Machine Learning (AI/ML) will replace people where you work. People who use AI/ML where you work with replace people who do not use AI/ML where you work. AI/ML will assist those that remain do their work better and faster.SkillsAttract, develop and retain people with the necessary skills._06019012-71ae-11e9-b009-6a70af9d818f10.1In order to realize the promise of AI/ML, you’re going to need a number of quintessentially human skills. The biggest challenge in AI is knowing if you have the right data. That's a quintessentially human skill. Knowing when the 'right data' shows the wrong things is a quintessentially human skill. Finding 'good enough' data, for AI, is a quintessentially human skill. Generating synthetic data, for the future you want to create, is a quintessentially human skill. Assessing which trained AI model is the most useful and explaining why, these are quintessentially human skills. You need to attract, develop and retain people with these skills.RequirementsUnderstand what data is required._06019148-71ae-11e9-b009-6a70af9d818f10.2InterpretationUnderstand when data is misleading._06019422-71ae-11e9-b009-6a70af9d818f10.3SufficiencyUnderstand when the data is 'good enough' for decision-making._060195a8-71ae-11e9-b009-6a70af9d818f10.4SynthesisGenerate synthetic data, for the future to be created._060196ca-71ae-11e9-b009-6a70af9d818f10.5UtilityAssess and explain why AI models are useful._06019a1c-71ae-11e9-b009-6a70af9d818f10.6Data QualityImprove data quality._06019b66-71ae-11e9-b009-6a70af9d818f11Pareto was right -- Data Science is the easy part. Getting the right data, and getting it ready for analysis, is much more difficult. Most of the costs go to getting data ready for use, not getting value out of the data. Across all industry sectors, CDOs report spending 80% of their resources (people, time, $s, energy) wrangling the data. It's critical work, organizational hygiene, but not high value-adding work. Don’t let it take over your team's focus.MLLeverage the machine learning._06019c88-71ae-11e9-b009-6a70af9d818f11.1Leverage the machine learning that comes embedded in your tools. It'll tell you the best ways to visualize the data for improved understanding. It’ll tell you how and where to improve data quality.IntegrationImprove data quality at the point of integration._06019e22-71ae-11e9-b009-6a70af9d818f11.2The best time to improve data quality -- whether with tagging or formatting or anything else -- is at the point of integration, when the data crosses the boundary into your possession. You don't want to depend on anyone else for the quality of your data. AutomationAutomate everything._06019f4e-71ae-11e9-b009-6a70af9d818f12If a task is worth doing, it’s worth automating. This is especially applicable to Data Science. You don't have enough people, time or energy to do everything manually. It doesn't scale. What's worse, manual effort decreases quality. The engineering console is not your friend, YAML is.TasksAutomate tasks._0601a07a-71ae-11e9-b009-6a70af9d818f12.1Automate tasks the first time you do them.SharingShare automation in shared repositories._0601a1ba-71ae-11e9-b009-6a70af9d818f12.2And share the automation by posting it in the shared repository.COIsFoster communities of interest._0601a340-71ae-11e9-b009-6a70af9d818f12.3We've established a central capability to capitalize on economies of scale for hard assets, share best practices, and improve talent retention by fostering communities of interest.ProcessesFix production processes._0601c05a-71ae-11e9-b009-6a70af9d818f13Fix the production process, not its products -- There's a technical term for fixing quality defects in your products. That term is ‘rework’. As any quality expert will tell you, rework means unnecessary delays and costs. CausesAnalyze root causes of defects._0601c276-71ae-11e9-b009-6a70af9d818f13.1The accepted best practice is when you see quality defects, to perform root cause analysis then fix the reason for the defect.RefinementRefine and perfect data pipelines._0601c3ca-71ae-11e9-b009-6a70af9d818f13.2This approach applies to the Data Science pipeline as well. We'll refine and perfect the pipeline, then let the pipeline refine and perfect our data products.2019-05-07https://www.linkedin.com/feed/update/urn:li:ugcPost:6527524123990585344OwenAmburOwen.Ambur@verizon.net