Blueprint: The Layers of IIoT Analytics Architectures

September 6, 2019 Brandon Lewis

Simple Circuit Board Top View

The ITU defines the “Tactile Internet” as a network that delivers high availability, reliability, and security at very low latency. Modern-day network engineers associate these characteristics with 5G communications. But operational technologists have been deploying Tactile Internet networks for years using various flavors of Industrial Ethernet.

Unlike in the past, networked industrial devices must now interact directly with enterprise applications. Organizations looking for greater insight into factory floor operations are even deploying some of these applications directly onto industrial devices themselves.

Whether this software consists of analytics engines or full-blown AI algorithms, it can quickly compromise the integrity of real-time equipment if not designed properly.

Today’s industrial engineers need a blueprint for integrating IT and OT technologies into a single piece of machinery. These systems must integrate the power of IT networking and processing, while remaining deterministic, safe, and reliable.

Fortunately, Proof of Concept (PoC) system designs that provide an outline for these system architectures are now becoming public. And they’re based on the principles of workload consolidation and Ethernet Time-Sensitive Networking (Ethernet TSN).

An Upgrade for Industrial Ethernet

Industrial engineers have traditionally created deterministic versions of IP networks using protocols such as EtherCAT, PROFINET, and POWERLINK. These Ethernet-based technologies carry IP packets to devices like programmable logic controllers (PLCs) at the network control layer. Beyond that, traditional fieldbus communications protocols are required.

Now, TSN extends Ethernet packets to subsystems like I/O modules, sensors, and actuators with sub-millisecond latency (Figure 1).

TSN delivers Ethernet packets to industrial sensors and actuators—edge computing
Figure 1. Time-Sensitive Networking (TSN) delivers Ethernet packets to industrial sensors and actuators. (Source: Control Engineering)

Ethernet TSN and Workload Consolidation

Ethernet TSN is based on the IEEE 1588 Precision Time Protocol (PTP) standard, which timestamps packets based on the device clocks of network endpoints. One centralized master source is used as a reference to keep the network synchronized.

PTP networks are capable of achieving sub-millisecond latencies. But just as important, PTP packets can be encapsulated in IP. Therefore, operational devices can essentially become a part of enterprise networks. For more about TSN, read Time-Sensitive Networking: Hype vs. Reality.

Meanwhile, industrial devices are being asked to incorporate enterprise functionality like real-time edge analytics.

Known as workload consolidation, industrial edge systems that have traditionally performed fixed functions are now being designed around multicore processors. Virtualization technology allows individual cores to execute processes like data analytics while other cores can be reserved for data acquisition or control tasks.

With more powerful, flexible systems at their disposal, organizations can reduce costs, minimize points of failure, and even use enterprise software upgrades to extend deployment lifecycles. (Read Embedded Virtualization Powers Mixed-Criticality IoT Systems to learn more).

The convergence of workload consolidation and TSN can unlock the true potential of multilayered, real-time data analytics architectures. With deterministic, low-latency enterprise communications and multipurpose compute platforms at each layer of the network, factory operators can deploy AI to oversee various domains and subdomains (Figure 2).

Ethernet TSN and workload consolidation can enable sophisticated tiered data analytics
Figure 2. Ethernet TSN and workload consolidation can enable sophisticated tiered data analytics. (Source: RTInsights)

Decisions are made by AI and analytics engines at each tier based on the response time required and the compute resources available. But the end-to-end information and decision-making hierarchy is connected across a single, unified, time-sensitive communications protocol. Such architectures promote technology reuse, interoperable networking equipment and services, and common data exchange formats.

Bringing Tiered Industrial Data Analytics Architectures to Life

A microcosm of this architecture can be found in a real-time industrial control server PoC from congatec and Real-Time Systems (Figure 3). The PoC is built around:

  • The 35W conga-TS370 Type 6 COM Express module based on a six-core Intel® Xeon® processor
  • An Intel® Arria® 10-based PCIe FPGA card
  • A Basler machine vision camera
  • The RTS Hypervisor
congatec and Real-Time Systems PoC AI vision, motor control, IoT connectivity
Figure 3. congatec and Real-Time Systems PoC consolidates non-deterministic AI vision, motor control, and IoT connectivity workloads onto the same hardware platform. (Source: congatec)

To demonstrate workload consolidation, the RTS Hypervisor partitions Xeon processor cores into multiple virtual machines (VMs).

The first VM occupies three cores and runs a Linux OS. The OS supports a 300 fps Basler industrial camera and Arria 10 FPGA card. AI algorithms developed using the Intel® OpenVINO toolkit are executed on the FPGA card for on-device object detection and recognition.

Two additional VMs host real-time Linux OSs. These control servo motors keep an inverted pendulum balanced upright. And the final VM hosts a third OS type that handles enterprise security and communications back to the cloud.

All of these applications run concurrently on the same hardware platform. But with the hypervisor, non-real-time VMs can be rebooted without interrupting the real-time partitions. This means that software updates can occur while control tasks continue to execute. And security vulnerabilities can be contained to enterprise VMs and not impact embedded systems as well.

A live demonstration of the system can be seen in (Video 1).

 

Video 1. Proof of Concept demonstration from embedded world 2019

 

Another key feature of the conga-TS370 is that it includes an Intel® i219-LM GbE LAN Controller. This discrete networking chip natively supports IEEE 1588 PTP.

In platforms like the congatec/Real-Time Systems PoC, the i219 determines the source and destination of PTP packets and synchronizes them using integrated timers. The controller works in conjunction with an OS to schedule packets during transmission windows that are dedicated to TSN traffic.

Release 4.20 of the Linux kernel added support for the Time-Aware Priority Scheduler (TAPRIO) and Earliest Transmit Time First (ETF). These modules allow developers to configure the system so that Ethernet TSN packets receive priority regardless of their position in the transmission queue.

The RTS Hypervisor also offers a privileged mode that allows guest OSs to access system networking resources in just a few microseconds. TSN packets from guest VMs therefore receive the same low-latency priority scheduling.

IIoT Analytics on the Shoulders of Standards

One characteristic that stands out about the PoC is that all of the core functionality is based on open source technology and industry standards. Linux and OpenVINO are open-source software technologies. IEEE 1588 PTP and the Ethernet TSN umbrella of standards are industry-driven efforts that will keep costs down, interoperability high, and ensure longevity.

COM Express is one of the most popular embedded hardware specifications available that supports server modules, gateway controller, and sensor processing use cases. It provides a stable, affordable, long-lifecycle deployment platform for new tiered industrial data analytics architectures.

What will you do with increased intelligence?

About the Author

Brandon Lewis

Brandon is responsible for Embedded Computing Design’s IoT Design, Automotive Embedded Systems, Security by Design, and Industrial Embedded Systems brands, where he drives content strategy, positioning, and community engagement. He is also Embedded Computing Design’s IoT Insider columnist, and enjoys covering topics that range from development kits and tools to cyber security and technology business models. Brandon received a BA in English Literature from Arizona State University, where he graduated cum laude.

Follow on Twitter Follow on Linkedin More Content by Brandon Lewis
Previous Article
FPGAs & Arduino Drive Motor Control
FPGAs & Arduino Drive Motor Control

High-end motor control often surpasses the capabilities of modern MCUs. FPGAs are an option, but their prog...

Next Article
Safeguarding the Grid with IIoT
Safeguarding the Grid with IIoT

How can energy distributors get ahead of challenges created by climate change? A utility in Finland is tack...

×

First Name
Last Name
Your Company
Phone Number
Country/Region
Subscribe To Intel Updates
Subscribe To Alliance Partner Updates
By submitting a form on this site, you are confirming you are an adult 18 years or older and you agree to Intel and Intel® IoT Solutions Alliance members contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel's web sites and communications are subject to our Privacy Notice and Terms of Use.
I would like to be contacted by: - optional
Your contact request is submitted.
Error - something went wrong!
×

Get the latest updates on IoT for industrial.

Country/Region
Subscribe To Intel Updates
Subscribe To Alliance Partner Updates
By submitting a form on this site, you are confirming you are an adult 18 years or older and you agree to Intel and Intel® IoT Solutions Alliance members contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel's web sites and communications are subject to our Privacy Notice and Terms of Use.
You are following this topic.
Error - something went wrong!