DATA SCIENCE AT THE EDGE: How manufacturers can maximise their IIoT investmentUsing accelerated GPUs at the edge to spot patterns in the blizzard of data that the Industrial Internet of Things (IIoT) generates can seriously boost a firm’s productivity.

he burgeoning volumes of data being generated by the many millions of devices that comprise the Industrial Internet of Things (IIoT) lend manufacturers some powerful opportunities to advance their business prospects. This is because they provide firms with ultra-clear views of how their manufacturing processes are really performing – with insights from machine learning being used to optimise efficiency and productivity.

Escalating data volumes will mean that, by 2025, data scientists will be injecting datasets 100 times larger than they are today into their machine learning systems, according to market research conducted by IT

industry analyst IDC.1 This, in turn, will drive computer processing requirements higher and software complexity will grow tenfold in this period, making still higher compute demands of the IT systems.

Growth will be strong in hard cash terms, too, say analysts at Million Insights in Felton, California: they estimate that the global market2 for IIoT processors, sensors and connected systems will grow at a rate of around 28% per annum up to 2025, reaching a value of almost $1 trillion ($934 bn) by the middle of the decade.

But for this technology to deliver on its promises to manufacturers, it’s vital that the IIoT data generated is processed as physically close as possible to where analytics results are needed, and as quickly as possible, if it’s to have the maximum impact on the bottom line. And that means businesses may have to reconsider (a) their use of cloud resources and (b) the types of computer systems on which they are running their machine learning.

THow manufacturers can maximise their IIoT investment

PA G E 2

How manufacturers can maximise their IIoT investment

PA G E 3

DATA SCIENCE: Creating evidence-based industries for the first timeThe idea behind data science is that extensive, cleaned-up, de-duped industrial datasets are fed into neural networks running machine learning algorithms. These algorithms then tease out previously unseen patterns in the dataset, allowing data scientists to infer what combinations of inputs caused those behaviours. These insights let the scientists decide whether they need to alter the way a machine/system works in response – to either eradicate unwanted behaviours or amplify good ones.

So what is analytics allowing manufacturers to do right now? Take one of the firms that has been using it for decades: Siemens of Germany. Siemens has been using neural networks to improve the efficiency of its steel manufacturing operations3 since the 1990s. Now, with some 200 data scientists in its employ, Siemens is also using machine learning to:

• Automatically optimise gas turbine operation so they always run at optimum efficiency

• Improve the monitoring of energy distribution in smart power grids

• Optimise wind farm operations to obtain peak power• Predict when parts will fail in industrial facilities so they

can be replaced before any expensive maintenance downtime is necessary

But AI does not need to be aimed solely at such large-scale, complex tasks: its ability to perform automated pattern recognition is useful in simpler roles, too – and once firms have become used to it, they can expand their AI applications to other processes.

As an example, Jared Dame, Director of Data Science and AI at Z by HP Inc. in Fort Collins, Colorado, describes the production line of a China-based circuit board maker he’s familiar with. The factory was almost entirely staffed by people, but semi-automated its processes to look for quality. It then gradually added more AI processes, using edge sensors and cameras that can completely manage the quality control and, for example, reject bad boards in seconds with significantly lower error rates.

Now, three years after that tentative introduction of AI, 90% of that very same factory is AI driven, says Dame. “One of the biggest shocks is how fast AI has been adopted in manufacturing – and how the human role has changed to being more like that of a caretaker today.”

PA G E 3

How manufacturers can maximise their IIoT investment

PA G E 4

CLOUD ISSUES: Damaging implications of low speed and high costThe cloud has been a powerful enabler of machine learning across industries, thanks to the fact that it is always on, always available and always security patched. Firms know that IIoT data can reach them with minimal security risks. But as cloud adoption has mushroomed, issues over latency and storage costs have reared their heads – and these need to be addressed if the analytics revolution is to continue improving manufacturers’ prospects.

Anything that hinders genuine real-time data analysis can have a significant impact on manufacturing. Dame recalls an oil and gas plant in Arizona where the operator decided to run a pilot programme designed to test the efficacy of predictive analytics. The pilot did not run in real time: instead, every Friday, operatives would visit the plant and download information from a data logger monitoring the boilers, in order to check how performance predictions tallied with reality.

Late one night, however, one of those boilers exploded – and although no one was hurt because the boilers were unattended at the time, the incident caused millions of dollars in damage. Forensic analysis of the contents of the scorched data logger showed the explosion would have been predicted – and avoided – if the data had been analysed in real time.

With real-time analysis, says Dame, “we could have predicted that boiler pump failure days prior to it actually failing. We could have shut it down and lost only a couple hours of production by fixing that pump valve, instead of losing months’ worth of production.” And it’s about more than hard cash: “Had it gone up when people were present it could have killed someone.”

The data analytics revolution in general is hitting some serious hurdles because speed and cost issues associated with cloud computing and storage are causing complications. For instance, to generate actionable insights, data must be sent on round trips to the cloud for processing by machine learning algorithms. But that is becoming a less and less viable proposition because that journey simply takes too long.

Indeed, the time and expense involved in moving a petabyte (one thousand terabytes) or an exabyte (one million terabytes) of data is becoming prohibitive, as Amazon Web Services CEO Andy Jassy has calculated. “Moving an exabyte of data would take 26 years with a 10 GB-per-second connection,” he says.4

But it is not just the round-trip delays that are causing issues: as dependency on the cloud for storing datasets has risen inexorably over the last decade, so, too, have companies’ cloud costs. “Some firms have seen their cloud bills grow astronomically. They sign up for three years at a cut-price processing rate, sometimes with free storage to start with, and all that makes a lot of sense at first,” says Dame.

“But then, all of a sudden, the bills start coming in. And a lot of companies will end up spending more on the cloud than if they had hired 10 IT staff to manage those datasets in house.”

The global market for IIoT processors, sensors and connected systems will grow at 28% per annum until 2025 – Million Insights

How manufacturers can maximise their IIoT investment

PA G E 5

EDGE COMPUTING: Crunching data locally for the fastest deliveryThese speed and cost issues are driving manufacturers away from the cloud, for some tasks at least, to another position on the network entirely. Known as “the edge”, this is generally defined as either an endpoint device that generates data, or a compute resource that’s no more than one network hop away from that device.

Instead of shipping all data off to the cloud, and incurring that round-trip time penalty, some edge data is processed in situ, far more locally to the factory floor IT systems. Because the data does not necessarily need storing if it’s processed immediately, this improves turnaround time and cuts cloud costs.

It’s an idea that is taking off big time, with a slew of content delivery networks jumping on the bandwagon.5 “So much data can now be produced at the edge that it couldn’t all be streamed to the cloud fast enough in any case,” says Dame. “So it’s vital devices process at the edge to generate timely insights.”

But that’s not to say all data stays on the edge. “There is a balance that needs to be struck between cloud and edge,” he says – with real-time tasks needing edge processing the most to quash latency issues.

WHAT IS EDGE COMPUTING? The proliferation of data, especially through the Industrial Internet of Things (IIoT), means that for reasons of latency and network bandwidth cost, it often makes more sense to process data closer to source rather than sending it back up to the cloud.

DOES THIS MEAN A SWING AWAY FROM THE CLOUD?Not necessarily. Sometimes it is better for data to be processed in the cloud and sometimes it is better for that processing to be done on the spot. The availability of edge processing can also help maximise the efficiency of cloud use, as only the best stuff is sent “upstairs” for storage, while the weaker stuff can be processed and potentially discarded on site.

WHAT DOES THIS MEAN FOR INDUSTRY?It provides more choice. And it ultimately means everything can be done much more quickly.

EDGE COMPUTING IN MANUFACTURING – WHAT YOU NEED TO KNOW

PA G E 5

PA G E 6

GRAPHICAL PROCESSORS: Maximising speed and improving sustainability The machine learning data load is growing fast – a hundredfold by 2025 – and energy-efficient, novel processor architectures are needed to cope with it. The associated rise in data-intensive workloads at the edge will drive the need for a new generation of energy-efficient architectures – and a new class of processor called a Machine Learning Accelerator (MLA) promises great advances in energy-efficient performance.

Driving the need for these go-faster AI devices is the fact that the rate of CPU efficiency gains has been slowing. For over 30 years, CPUs exponentially increased in performance as Moore’s Law saw each semiconductor generation roughly double chip transistor density every 18 to 24 months. But it is now clear that we are getting close to the limits of miniaturisation.

High-performance GPU architectures, however, have thousands of processing cores and so are managing to stay slightly ahead of the Moore’s Law slowdown. But because of the data explosion at the edge and increasing task complexity, compute demand is outpacing the power-performance growth of even a GPU.

With manufacturers, like all businesses, aware of the need to cut carbon emissions, and be seen to be doing so, it is not enough to increase compute capacity to meet the demand: this must be accomplished without significantly increasing power drain, by using ultra-energy-efficient computing techniques. Hyper-efficient, next-generation GPUs are likely to be the dominant machine learning accelerators in the near future, with versions optimised for specific markets expected to become available.

Manufacturing, for instance, will have its own AI hardware and software combination that streamlines its particular analytic workflows. So will other sectors. “For instance, biomedical sciences, security, image recognition and natural language processing applications will have slightly different ones,” Dame predicts.

“GPU acceleration is going to be an absolute game changer, because it allows you, especially on edge devices like a workstation, to be able to inject four to seven billion rows and columns of data, which would usually require a high-performance compute centre or cloud environment, which would cost a lot more and create task scheduling issues.”

WHY GPUS? We’re reaching the limits of CPU miniaturisation and GPUs are slightly ahead of the Moore’s Law slowdown. This is because GPUs have hundreds (or even thousands) of arithmetic cores to crunch through data simultaneously, unlike CPUs, which typically have less than 10 cores. They are also more energy efficient, which is better for the environment.

HOW DOES THIS HELP AI? GPUs were originally used to speedily render video frames in graphics-heavy imaging applications, such as video games. Now, just as video comprises massive arrays of pixel data, AI problems involve vast matrices of data that mathematical operations must be performed on. GPUs turn out to be just as good at analysing such matrices as they are at processing images.

WHAT DOES THIS MEAN FOR INDUSTRY?It means firms can seek out previously unseen patterns in machine and shop-floor data – and solve problems before they arise, through anomaly detection which might indicate a part is about to fail.

WHERE IS ALL THIS HEADING?Eventually the application of accelerated AI should enable industrial processes to be self-optimising, maximising both process and energy efficiency.

ACCELERATED GPUS – WHAT YOU NEED TO KNOW

How manufacturers can maximise their IIoT investment

SOFTWARE 2.0: Providing a sea change in software developmentThe machine learning revolution is not just a hardware play: it is also being driven by major-league changes in the way software is developed. Hand in hand with the rise of new data-rich workloads and new hardware architectures, a new paradigm in software development has emerged: Software 2.0. This enables faster development and greater flexibility for solving traditional problems and tackling complex new ones. It can speed products to market, as well as reduce development and maintenance costs.

Software 2.0 is a differentiator because, in traditional/classic software development, the goal is to write lines of code that are treated as instructions to the computing environment. A typical project for a software team may have many millions of lines of code, some of which will come from outside development teams, and some from the team itself.

But Software 2.0 represents a new way of thinking. In it, value creation comes not from code writing, but from data curation. Users collect data from their IIoT sensors and devices, curate that data (that is, select relevant datasets, verify and label them) and use them to train machine learning models. They then distribute those models and “run” them, instead of creating algorithms and writing code. “It is a very different way of programming indeed,” says Dame.

The result of all this is that no software engineer has to laboriously write code in the old way. Software 2.0 removes the need for expensive hand-coding of potentially fragile defect-spotting algorithms, creating instead far smarter ones – like those used in China to spot defective solder joints in the earlier example. And the process of selecting data and training the machine learning system is fast, from a few hours to days for typical training.

WHAT IS SOFTWARE 2.0? In regular software development, programming teams write thousands of lines of code in a protracted process in which they attempt to cajole their software into recognising, and acting upon, certain conditions – such as recognising images of defective microchips, or spotting anomalies indicative of faults. Software 2.0 avoids much of the grunt work this process usually involves.

HOW DOES SOFTWARE 2.0 WORK?By setting up machine learning neural networks so that they “learn” the conditions (the look of a microchip, say, or the shape of a fault waveform) they need to recognise, instead of having to be explicitly programmed to recognise them.

HOW IS THIS DIFFERENT TO HARD CODING? The neural network is trained by showing it the data it is meant to recognise. This creates a distinct array of signals across the network, called weights, and these appear every time that data is input to the system.

WHAT DOES THIS MEAN FOR INDUSTRY? Software 2.0 lets data scientists curate datasets and train systems up more quickly. This way, their technology can quickly cater for new products and systems, or perhaps defend against novel industrial cyberattacks.

SOFTWARE 2.0 – WHAT YOU NEED TO KNOW

PA G E 7

How manufacturers can maximise their IIoT investment

PA G E 8

DIGITAL TWINS: Building test-to-destruction doppelgangersBut data scientists are not finished yet. The volumes of data machine-generated by the IIoT and the processing power available at the edge allow data scientists to play one more very cool trick: the construction of virtual machines, or digital twins. These are hyper-detailed digital representations of physical products – such as a jet engine, a wind turbine or a large electric motor – and they only exist in the memory of a fast workstation.

The aim? To let firms do as much testing of the machine as possible without actually having to build it. This way, much of the lifecycle of a product can be explored digitally at very low cost – testing to virtual destruction if need be. These multi-layered representations of the physical system model features such as the software, electrical circuitry, aerodynamics and the mechanics of a machine – providing as many of the functional attributes needed to mimic its operation as possible.

The benefits turn out to be quite profound, with digital twins seriously reducing the time it takes to develop a product – cutting it by as much as 30% in the case of the luxury car maker Maserati, for instance.6 And factory-scale simulations of deployment environments, such as additive manufacturing (industrial-quality 3D printing) operations, can help customers predict the economics of the technology based on various workflows and service-level agreements. HP Inc. and Siemens are working on just such an additive manufacturing venture.7

“GPU acceleration is going to be an absolute game changer because it allows you to be able to inject four to seven billion rows and columns of data.” – Jared Dame, Director of Data Science and AI at Z by HP Inc.

How manufacturers can maximise their IIoT investment

MANUFACTURING TODAY AND TOMORROW: Maximising IIOT investmentIndustry is now awash with data from the Industrial Internet of Things. And the challenge for manufacturers is to leverage that data – using AI to optimise their operations to peak efficiency and to predict expensive maintenance issues well before they happen. Only that way can they stay ahead of the competition.

Improved ways to maximise hardware and deliver software are already providing answers via faster, more efficient processing. And the promise, further down the line, is of specialised, accelerated hardware and software combinations, optimised to deliver the ultimate machine learning experience, at speed.

The trick for many manufacturers will be to ensure they take full advantage now, by looking carefully at their cloud resources and the types of computer systems that are running their machine learning. Making small changes today could be the key to unlocking their IIoT investment in the long term – and securing their operation for tomorrow.

How HP can help HP works with leading manufacturers, co-development partners and strategic partners to transform manufacturing. The power of Z by HP Workstations can unlock the transformative potential of the IIoT, cloud computing, AI, machine learning and big data. Boost innovation and productivity with HP’s VR and 3D printing solutions – and our wide range of managed print and personal systems services, too. Discover how automation and faster access to data-informed business intelligence can deliver on the promise of Industry 4.0.

PA G E 9

“One of the biggest shocks is how fast AI has been adopted in manufacturing – and how the human role has changed to being more like that of a caretaker of the technology today.” – Jared Dame, Director of Data Science and AI at Z by HP Inc.

How manufacturers can maximise their IIoT investment

PA G E 1 0

1 Smart Data Collective, IDC, quoted in Exciting Predictions For Where Big Data Analytics are Headed by 2025, Feb 2019. https://www.smartdatacollective.com/exciting-predictions-for-where-big-data-analytics-are-headed-by-2025/

2 Million Insights, IIoT market research report, 2014-2025, April 2017. https://www.millioninsights.com/industry-reports/industrial-internet-of-things-iiot-market

3 Siemens AG, Artificial Intelligence: Optimizing Industrial Operations, March 2018. https://new.siemens.com/global/en/company/stories/research-technologies/artificial-intelligence/artificial-intelligence-optimizing-industrial-opera-tions.html

4 GeekWire, Amazon reveals AWS Snowmobile, a 45-foot semi-trailer that moves exabytes of data to the cloud, Nov 2016. https://www.geekwire.com/2016/use-amazons-snowball-snowballs-unleashes-45-foot-truck-model/

5 Light Reading, Another CDN Company Jumps Into Edge Computing Market, Aug 2019. https://www.lightreading.com/the-edge/another-cdn-company-jumps-into-edge-computing-market-/d/d-id/753782

6 Deloitte, Expecting digital twins, May 2018. https://www2.deloitte.com/us/en/insights/focus/signals-for-strategists/understanding-digital-twin-technology.html

7 HP, HP and Siemens Deepen Additive Manufacturing Alliance to Advance Digital Manufacturing, May 2019. https://press.ext.hp.com/us/en/press-releases/2019/hp-and-siemens-deepen-additive-manufacturing-alliance.html

Sources

© Copyright 2019 HP Development Company, L.P. The information contained herein is subject to change without notice. c-06488169, October 2019