MxD’s 22,000 square-foot manufacturing floor

ML Journal

ML Journal

Sensible Advice for the New Wave in Decision Systems

Preparing for manufacturing’s AI revolution requires security, teamwork and data literacy to navigate new technological landscapes 

 

TAKEAWAYS:
AI Integration and Security: The deployment of AI in manufacturing requires careful consideration of security measures, including secrets management.
Empowering Workers with Data Literacy: Successful AI implementation hinges on involving daily operators, developing security champions and promoting data literacy among workers.
Infrastructure and Network Adjustments: The shift towards AI-driven manufacturing necessitates significant changes in infrastructure, including increased internet connectivity and changes to network security architecture.  

 

As artificial intelligence arrives for manufacturing but it does not require hasty sacrifices in operational security. The technology spans a variety of use cases and personas, including predictive maintenance, supply management, and customer service. Expect AI to come with major changes to architecture, such as demand for greater public internet connectivity, leading architects to revisit security architecture. With such sweeping change the challenge becomes how to accept these new opportunities to improve the business, while managing the risks without giving up the whole game. For example, one can choose OT monitoring systems to observe the internet-connected sensors that drive many digital twin projects. Beyond the quick wins, leaders should offer autonomy and support to their plants’ operations experts, to team with these new tools, and give them the resources to evaluate their estate using a data-driven approach that coheres to the AI program’s assumptions.

AI denotes the use of computerized decision systems to drive tasks that would otherwise use human intellect1. Sensors for these systems can span a range of inputs and would be positioned throughout the factory floor2. Ranging from heuristics to statistical learners and large language models3 — the factors in developing an optimal decision system often depend on system specifications and learning objectives. Most systems are designed explicitly for teaming between subject matter experts and the underlying learners, so the interfaces of these systems depend on their technical requirements4.

On the factory floor, there are three prospects for transformation. First, is the notion of digital twins5 and intelligent factories, which uplifts product lifecycle management into a fully computerized realm. Practically, this implies wide scale sensor deployments and ubiquitous availability of engineering computers, handheld, and personal devices. Second, is the increasing use of analytics to control supply chains and inventory. The state of the art has grown from ABCD analysis in the context of lean manufacturing, and into forecasted reorder systems and predictive maintenance regimes6. Automated support equipment is more likely to produce metadata, implying connectivity to processing grids designed for analytics7. Third, is the increasing use of customer feedback to optimize field operations. Call lines are increasingly centralized into virtualized equivalents of physical systems, with calls and e-mails being transcribed and feeding Voice of the Customer systems8. Data that was typically solely reserved for the back office is now available everywhere.

Regard the deployment of AI in your manufacturing environment as an opportunity to serve your daily operators as customers who require buy-in to execute your roadmap

 

To meet the challenge of transformation, infrastructure changes will come to the factory floor. Many sensors will need internet access to send communications to cloud-hosted brokers. This traffic can be proxied via self-hosted infrastructure, broadening access previously dedicated to the plant’s workload. Operations will also have to abide by a broadening raft of data sovereignty rules9. As the data bandwidth increases, companies will also find themselves racing to keep their key contributors appraised with the new systems. Engineers will have to become familiar with new protocols, cloud services, and most challenging of all, the analytical assumptions for these new decision systems10.

There are huge security trade-offs with the revolutionary changes discussed above. Enterprises should be prepared to proxy internet enabled devices. Certain applications need direct internet access and may cease functioning when routed through a proxy. Operators should be aware that those same devices will demand far more network bandwidth than prior. Network perimeter audits will have to scale with the added traffic and expanded blast radius. Bad actors may find additional resources for lateral movement where networks have not been adequately segmented. The use of additional systems of record also increases the risks posed by shared secrets, a challenge illustrated by the unprecedented data leak Snowflake customers face today11. AI systems have complex software dependencies that compound hardware and software BOM data collection. This creates a stark paradox given that the nature of recent high severity vulnerabilities, such as with LangChain12, are simple and can be exploited with little novelty on an adversary’s part. Finally, AI systems operate based upon a variety of approaches to data analysis, which can vary from simple decision trees and logistic regression models to large language models and other so-called overparameterized models. Regardless of these systems’ methods, retaining personnel capable of interpreting their output is crucial for good outcomes.

There are ways for operators to be prepared for the challenges of direct access, greater data egress, expanding vulnerabilities, and added areas of special expertise needed to operate. Be it through architectural adjustments or unlocking new areas of technical expertise, the following advice draws from primary sources that are indispensable for manufacturers. These are the SANS Five ICS Critical Controls whitepaper as well as the NIST guidance for manufacturing (most notably, SP 1800-10 and the manufacturing framework profile)13.

The crucial resource is humans, and if you intend to fund AI solutions on the factory floor, be sure to also invest in your workers’ data literacy

 

A finding in common with SANS and NIST publications is that defensible architecture depends on definitive logging on networks. This can mean further transport and analysis of firewall logs, but also recognizes that perimeter firewalls have little insight into internal networks, including remote access sessions or resulting traffic and actions within the OT environments. Fortunately, there are many first- and third-party solutions that can help detect threats on the perimeter, particularly in secure access areas14. The challenge becomes detecting threats that have made it beyond the perimeter. Secrets management programs can also help manage the blast radius by reducing secret reuse. Finally, one should invest in an asset monitoring program that tracks vulnerabilities, rewards patching discipline, and concentrates on risk mitigation for systems where patching is not possible. OT monitoring systems such as the Dragos Platform can build a bridge from ingress logs to a plant’s historians and other internal measurements, adding context in the case of a breach and maximize ICS network visibility. Having a registry of OT assets also allows one to focus their attention on the systems with the greatest effect on plant operations. Such a registry is typically a deliverable from a crown jewels analysis, a systematic, OT-focused threat analysis workshop15. In addition, canary token programs, where one issues a token that is tracked but does not provide deep access, is a useful approach to find areas of exposure16.

Just as creating shared ownership is a cornerstone to operating a security culture, growing shared technological expertise is the strongest means of hardening an environment subject to uplift by AI. The major theme for achieving this is to regard the deployment of AI in your manufacturing environment as an opportunity to serve your daily operators as customers who require buy-in to execute your roadmap. While it is typical for consultants or a center of excellence to develop AI systems, it is crucial that ideation includes these operators at all steps of the requirement gathering process. These operator roles can span from technicians and fabricators who assemble parts, the designers and engineers who specify drawings, all the way to inspectors and process analysts who keep the plant safe and thrifty. From these ranks one should identify champions as those who enjoy systems and/or security thinking, are empathetic for the conditions of their teammates, and are motivated to teach their findings.

Developing security champions for AI should not be treated as any other milestone. Getting buy-in from contributors will imply investing in education, not just for the AI system you are running, but also general data literacy and context for security architecture. The intuition for teaming with your individual contributors should be that AI is most effective in construction as a human-in-the-loop system17. The individual contributor should become more introspective about their operations such that it expresses growing data literacy. Leaders should reward examples of data literacy every day. For example, it is common for operations staff to know what typical levels look like, sometimes with greater detail than the equipment’s operating parameters. Challenge these experts to define these parameters formally via control charts, to impart this additional precision systematically18. Such a study applies just as effectively for AI-driven systems as well — one may consider this as calibration for the system. Finally, these findings have security implications too. A well-quantified baseline for normal activity in a plant is crucial for timing security response as well.

The winds of change in manufacturing tend to be steady and strong rather than sudden and violent, and this extends to AI. There will be deployments of troves of sensors, and greater networking of extent devices, which on the security front, means increased risk. More effort should be spent auditing traffic to the public internet and isolating important data and credentials. Security platforms, solutions, and services can be utilized for this effort, and OT-focused offerings such as the Dragos Platform and crown jewels analysis will be most in tune with your plant’s specific needs. The crucial resource is humans, and if you intend to fund AI solutions on the factory floor, be sure to also invest in your workers’ data literacy. Common awareness of the new operational norms and security architecture of your AI solutions will go a long way into hardening your environment as it becomes more efficient.  M

Footnotes: 202408_MLJ_Dragos_Footnotes

About the author

 

Jonathan Reiter is a Principal Engineer at Dragos Inc, where he has been since 2020. In his career he has contributed to router firmware, antivirus telemetry systems, industrial demand forecasting modules, and tagging systems for 737 fuselages for use on the Boeing Renton Factory floor. 

 

View More