All News & Insights

The Case of the Missing Insights

Manufacturers can build on ideas from Sherlock Holmes to derive value from data using artificial intelligence and machine learning technologies. By Sath Rao

Enterprise transformation depends on the successful application of technology and change management to both solve business challenges and exploit new opportunities. The power of transformative technology can be exponential when an integrated approach towards humans, machines, and data is adopted. This concept can be applied to digital transformation, to addressing manufacturing inefficiencies, and to enabling next-generation business model transformation — all components of the fourth industrial revolution, Manufacturing 4.0.

As companies attempt to optimize and disrupt business via new technologies such as artificial intelligence (AI), machine learning (ML) and the internet of things (IoT) — and grapple with proofs of concept — the challenge has often been the focus on the wrong problem. Prioritizing the right problems in the run-transform cycle and focusing on business value will help firms derive value from the power of these technologies. In the end, focusing on return on data will ensure rich rewards.

What Are the Right Problems?  

Data science is about achieving positive business outcomes by bringing together the best analytical framework with domain know-how — or is it? Domain know-how is based on known answers to known problems. What if your business outcomes change constantly, but your decision-makers are still applying approaches that worked with previous problems? Do you need to be more intelligent about embarking on this transformation journey? And, equally important, what problems should you try to solve first?

The answers to these questions are in the data itself and in what lies hidden in that data. Given the complexity and volume of potential questions and answers faced by today’s real-world businesses, the probability of solving any particular problem increases greatly when advanced analytics, AI, and ML are added to the mix.

The foundation for this new approach to problems and solutions is illustrated in the following four-quadrant graphic. The quadrants progress from applying known answers to known questions (known-knowns), to applying new answers to known questions, new questions to known answers, and, finally, new answers to new questions (unknown-unknowns). This visualization makes it easy to compartmentalize problems and then apply solutions appropriately to each quadrant.

Compartmentalizing vs. Sense-Making 

Problem-solving approaches are usually post facto, meaning a problem already occurred and now it needs to be solved. The traditional attempt to problem-solving is to adopt solution approaches based on the type of the problem. Incremental gain seekers tend to focus on problems in the known-known quadrant. Businesses today must decide whether to start with known questions and answers or questions and answers they know nothing about.

When it comes to new answers to new questions (unknown-unknowns), the typical approach is to try to solve the problem first, and then attempt to figure out what caused it in the first place. Unlike human problem-solvers, AI can leverage big data to go beyond the obvious (such as trying to use existing, known-known answers) by deriving its own conclusions based on both structured and unstructured data. In the categorization models, frameworks precede data and in sense-making models data precedes frameworks. In the classic HBR article, “A Leader’s Framework for Decision Making”, there is emphasis on the need for agility in decision-making styles to match changing environments1.

Machine-learning algorithms such as reinforcement learning game theory can be used to develop new theories on what works and what doesn’t. By using these algorithms, businesses can generate new insights and simulate outcomes through hypothesis testing. Further, the ability to identify new problems based on the data and then develop answers helps scale economies of learning. This process takes human assumptions and traditional rules-based approaches out of the loop and enables autonomous insights. Organizations focusing on economies of learning can also leverage the asset-light model and can quickly challenge incumbents focused on economies of scale (think Uber or Tesla versus the automotive majors).

As more businesses apply AI to defining and solving problems for business outcomes, the challenge is to let go of past practices. Domain expertise is very important to guide outcomes especially when controlling critical situations. The key deficit area now is trust. Explainable AI (XAI) is an emerging area where the aim is to help demystify the black-box approach of arriving at decisions by progressively building trust in the system. Decision making can then be split into tiered layers with standalone systems that can also connect to a system of systems that help with prediction accuracy, decision understanding, and inspections and traceability.



“Machine-learning algorithms such as reinforcement learning can be used to develop new theories on what works and what doesn’t.”

A System of Systems Approach  

An evolving paradigm is the horizontal approach that determines where computing will be done to solve problems based on the complexity of the problem, the tolerance for connectivity-oriented latency, and decision latency, as shown in the illustration. Note that there is still a place for steady state point-of-use computing based on the time needed for the decision.

In some instances, computing for problem solving must happen locally. For example, a self-driving car needs to use onboard algorithms while it’s driving so that it can make moment-by-moment decisions for braking, changing lanes, and so on, without communicating with cloud databases. It can still communicate with the cloud to gain broader knowledge that isn’t critical in the moment, such as traffic conditions and alternate routes. Based on permission, if a traffic congestion situation can prompt special incentives for a detour to a favorite eatery, and reserves a table, too, it would be a whole new level of customer experience! Reinforcement learning could actually deliver a combination of surprises that are pleasant and unexpected.

In a manufacturing environment, the ability to drill down to root-cause, examine incoming supplier records, and use video-based analytics for inventory backlog analysis can trigger optimization opportunities in the manufacturing process and drive forecasting insights.

Sourcing Data from the Edge to the Cloud 

Data today is flowing in at an unprecedented rate, from sources on the edge, in the cloud, and everywhere in between. This data provides a wealth of answers never before attainable, but, at the same time, it produces an overwhelming number of new questions. The sheer volume of data being generated is no longer possible for humans to decipher. Addressing this challenge is another aspect of the new paradigm, an approach that moves from the edge to the cloud and from storage to enrichment.

In manufacturing, the horizontal approach might translate to computing at the edge to monitor processes within a single machine, while communicating with the cloud to coordinate with other machines or processes and lines, or even across the supply chain to optimize the overall manufacturing process. This is a key component of Manufacturing 4.0, which relies on data-driven insights to reframe assumptions and look beyond past practices.

Applying the New Paradigm 

Making AI part of your approach to Manufacturing 4.0 requires more than just the technology; it requires a new way of thinking about that technology. Manufacturing 4.0’s core promise of cyber-physical systems — the coming together of physical and cyber functionality — enables the modeling of digital twins for processes and products, and it provides the ability to predict failures and initiate remediation ahead of time. When you have AI integrated into your journey to Manufacturing 4.0, you can decide which steps to take with greater confidence, can look at the business context before making decisions, can predict bottlenecks, and much, much more.

In the customer-centric economy, the real value is in the outcome for the customer and the ability to differentiate based on experience and personalization. The ability to generate insights that can actively empower customers is going to drive the next wave of competitive differentiation. Gartner notes in a research outline, “The CX Pyramid: A Framework for Powerful Experiences”, that the focus of differentiation ranges from solving problems to proactively making customers feel better and empowered with answers. 2

Manufacturing 4.0 will empower progressive manufacturers to provide this visibility to their customers both in a B2B and B2B2C context. While the current focus of Manufacturing 4.0 has been on digital twins of products and processes, increasingly, AI/ML will help drive a vision of the digital twin of the entire enterprise, and, based on the customer’s digital-entity, transform their experiences.

The Importance of DataOps  

To realize the full benefit of AI technologies, you need a vision, and you need to figure out how to derive value from your data. The overall goal is to derive benefit from people, processes, and technology, but the key is to start with people. The human in the loop is always the most important element. Manufacturing 4.0 is as much about cultural change as it is about moving people away from their original paradigms — and DataOps can help you get there.

IT and operational technology (OT) skill sets are converging to drive business value. Organizations can support this by implementing DataOps to improve data analytics, layering on lean manufacturing practices with AI and ML to increase efficiencies and improve quality. DataOps is data management for the AI era — getting the right data at the right time, to the right person to reduce errors, improve quality, and drive insights, and do it all in a faster and secure environment.

A robust DataOps approach allows your organization to use data in innovative ways, gain a competitive advantage, and, ultimately, monetize your data. Rather than simply hiring data scientists to solve every issue, focus on aligning skill sets to deliver the four Cs of DataOps: connected, curated, contextualized, and cyber-confidential (see the illustration ). 3

These four Cs represent how to manage data and make it available for analytics processing that enables value extraction. Competing on analytics will require challenging past paradigms and encouraging a cultural shift.

Lessons for the Disruptive Decades Ahead 

Central to success with transformative technologies will be your ability to make three critical paradigm shifts — all of which DataOps makes possible:

  • Shifting from a walk-and-look approach to a look-and-walk approach. In the old paradigm, you would walk the shop floor or the top floor and look for the problems that needed to be fixed. In the new paradigm, insights from data analysis trigger investigation opportunities and predict anomalies. Only then do you walk the floor to deal with the problem.
  • Shifting from fixing the broken to brokering the fix. The earlier approach was production or operations was paramount and reaction was to failures. When something failed, then you could fix it. Now, advanced analytics allow you to look through data silos to learn what might fail and what preventative actions you might need to initiate to broker the fix. Busting data silos is what gets the maximum return on data.
  • Shifting from ring-fencing issues to wringing value from your data. In the old approach, you would leverage data within a silo, ring-fence issues, and solve them. In the new paradigm, your reach extends much further as you wring value from the data across the organization and the supply-chain to impact the final frontier – the customer experience. It’s the customer- and outcome-centric world that we need to prepare for!

The Right Growth Mindset 

The secret to Manufacturing 4.0 will be the ability to apply AI and ML approaches to solve problems your team doesn’t even know about. The good news is that the approach to solving the mystery is in plain sight. It just takes the right growth mindset to identify it.

There is an explosion of data around the use and delivery of products and services. This is the precursor to the formation of the outcome-based economy. To succeed in the coming years, businesses will need an acute focus on their return on data — and that requires agility. Your future depends on your ability to focus on infrastructure agility, data agility, and, ultimately, business agility.

Build your organizations DataOps capability to relentlessly define and prove business value. This is your ultimate goal: to work with everyone, break down silos, and leverage AI and ML to make the most of the wealth of data at your disposal. M

1 David J. Snowden Mary E. Boone, A Leader’s Framework for Decision Making, Harvard Business Review, November 2007,
2 Chris Pemberton, “Create Powerful Customer Experience,” Gartner, May 19, 2019,
3 Sath Rao, Manufacturing 4.0 – Time for the DataOps Revolution,” Manufacturing Leadership Journal, June 6, 2019,

View More