Supporting Deterministic Traffic with Next Generation Ethernet
As we scale to a world of billions of intelligent, connected devices, crucial differences are emerging between the Internet of Things (IoT) and its subset, the Industrial Internet of Things (IIoT), The biggest differences involve time sensitivity. Many IIoT systems deal with mission-critical data that must be transferred and shared within strict bounds of latency and reliability.
Recognizing the inadequacy of most network infrastructure for handling such time-sensitive data, industrial suppliers, IT vendors, and silicon providers are collaborating to update standard Ethernet. Their solution is a next-generation Ethernet standard called Time-Sensitive Networking (TSN).
This article looks at one of the first commercially available industrial controllers that supports deterministic traffic using the new IEEE 802.1 TSN standards. We consider how TSN implementation enables networked synchronization at accuracy better than 100 ns. We explain how a solution based on Intel® processors and Ethernet controllers delivers the tight synchronization and low latency required for industrial applications such as motion control and condition monitoring. We show how devices based on this solution enable the convergence of control traffic and standard IT traffic. We conclude with details of an industry-supported testbed for demonstrating the advantages of TSN-based devices.
The Synchronization Challenge
Standard Ethernet is ideal for openness and interoperability, but it falls short when it comes to IIoT requirements for bounded latency and reliability. Information such as control and fault detection data must be processed, shared, and acted upon immediately, regardless of other network traffic.
To surmount this shortcoming, organizations and applications use nonstandard network infrastructure or isolated networks that limit latency but make devices and data harder to access through standard networks – if accessible at all. Each network layer in these implementations features a different level of latency, bandwidth, and quality of service (QoS).
The few available proprietary Ethernet derivatives typically deliver limited bandwidth. They also require modified hardware, making interoperability and communication of process data to higher level algorithms challenging (Figure 1.)
Figure 1. This diagram compares a standard Ethernet-connected device (left) with one connected with a proprietary Ethernet solution (right) in which extensive and costly modifications are required for handling time-critical data.
To support the many advantages of IIoT-enabled infrastructure, designers and end users alike need reliable, remote, and secure access to smart edge devices without impacting a device's ability to handle time-sensitive data. Network technologies must evolve to advance the way we operate our machines, electrical grids, and transportation systems.
TSN to the Rescue
In a major effort to support these needs, industrial suppliers, IT vendors, and silicon providers are collaborating on TSN within IEEE 802 and the recently formed AVnu Alliance. By updating Ethernet protocols to provide bounded, low-latency data transfer for time-critical data in IIoT and Industry 4.0 applications, they are addressing the shortcomings of existing networks.
An update to standard Ethernet, TSN gives today's open network infrastructure the ability to handle bounded low-latency data transfer alongside non-time critical or best-effort data. By converging time-critical and best-effort data within standard Ethernet, TSN delivers both cost savings and improved interoperability (Figure 2). By being part of the Ethernet standard, TSN can continue to benefit from continuing improvements in the standard's security, bandwidth, and other characteristics.
Figure 2. Converging time-critical and best-effort data within standard Ethernet, TSN improves interoperability and eliminates the need for expensive custom solutions.
The new TSN standard delivers numerous advantages over today's standard and specialty Ethernet protocols. Here we briefly address each of them.
Proprietary Ethernet derivatives commonly used for industrial control are often limited to 100 Mb, so large data sets from advanced sensing applications such as machine vision, 3D scanning, and power analysis put a strain on network bandwidth. TSN overcomes these limitations by embracing standard Ethernet rates (1 Gb, 10 Gb, and 100 Gb) and supporting full-duplex communication.
Latency, Synchronization, and Redundancy
TSN prioritizes the low-latency communication required for fast system response and closed-loop control applications. It can achieve deterministic transfer times on the order of tens of microseconds and time synchronization between nodes down to tens of nanoseconds. To ensure reliable delivery of this time-critical traffic, TSN provides standard configurations for high-reliability data paths where packets are duplicated and merged to provide lossless path redundancy.
Most lower-level field buses implement security through physical isolation (air gapping) and obscurity. However, today's frequent security breaches emphasize the need to fully extend security into the critical lower levels of control infrastructure.
TSN enables adoption of top-tier IT security provisions to protect critical control traffic and adds additional features that can be used to build additional security provisions. TSN's ability to guarantee network delivery and timeliness limits the impact of attack by protecting critical control traffic. In addition, synchronicity between the various devices in the system can form a basis of trust so systems can detect intrusions by checking for synchronicity disruption.
Integration, Manageability, and Cost
The IIoT relies on the universal availability of data, but today's unconnected and proprietary networks significantly limit accessing machine data. Using standard Ethernet components, TSN safely, securely and seamlessly integrates these brownfield applications with standard IT traffic. Coupled with TSN's low latency and high bandwidth, this ease of integration enables new control strategies, increased machine-to-machine (M2M) capabilities, and integration with diagnostics systems.
As an update to standard Ethernet, TSN also includes features such as HTTP interfaces and web services that enhance manageability. These Ethernet features enable the remote diagnostics, visualization, and repair features necessary for IIoT systems. As an added benefit, by leveraging standard Ethernet chipsets, TSN drives down component cost. High-volume, commercial silicon is much less expensive than low-volume specialty Ethernet variants.
TSN comes with a growing ecosystem supported by the AVnu Alliance and member companies such as NI, Intel, Broadcom, and Cisco. The AVnu Alliance plans to operate similarly to the Wi-Fi Alliance, providing certification services for products and devices designed to meet the IEEE TSN standard.
Using Intel® Technology to Create a TSN-equipped Industrial Controller
Intel® Ethernet controller reference designs form the basis of the open-source codebase maintained by the AVnu Alliance. NI, Intel, and other member companies are collaborating to extend this codebase, and are using it to create the first implementations of TSN.
NI is implementing TSN on the NI CompactRIO controller (Figure 3). This unit employs two processing targets: an Intel® Atom™ processor running NI Linux Real-Time as well as a field-programmable gate array (FPGA). The controller also includes industrial I/O and powerful development and runtime software.
Figure 3. The CompactRIO controller integrates TSN capabilities with high-performance processing, industrial I/O, and industry-standard certifications.
An Intel® Ethernet Controller I210 handles time-sensitive and best-effort data. Designed for applications requiring tightly controlled synchronization, the Intel Ethernet Controller I210 provides the ideal blend of Gigabit Ethernet (GbE) performance and precision time stamping.
The Ethernet controller contains four transmit and four receive queues offering error-correcting code (ECC) protection for improved data reliability. Incoming Ethernet traffic is split into two distinct streams: a time-sensitive stream and a best-effort stream that operates as standard Ethernet does today.
Outgoing Ethernet traffic is transmitted according to a transmit schedule to ensure timing requirements are met. The timekeeper on the Intel Ethernet Controller I210 is synchronized to network time and is shared with timekeepers on the processor and FPGA, ensuring timing requirements are met (Figure 4). Combining parallel and pipelined logic architectures optimized for these independent transmit and receive queues, the controller efficiently manages packets based on their time sensitivity.
Figure 4. Traffic on the Intel® Ethernet Controller I210 is split into separate streams and handled according to a transmit schedule and timekeeper.
The design enables NI's TSN-enabled controllers to provide:
- Networked synchronization at accuracy better than 100 ns
- Deterministic data communication at loop rates up to 5 kHz
- Safe, reliable convergence of control traffic with standard IT traffic
Benefits of the NI CompactRIO Controller for TSN
Using the NI CompactRIO controller for TSN implementation reduces risk, boosts system performance, enables faster development, and simplifies system design. Design features include: extreme ruggedness; industry-standard certifications; integrated vision, motion, industrial communication, and human machine interface (HMI) capabilities; and industrial I/O modules.
Inside, the NI CompactRIO controller delivers significant advancements in processing and heterogeneous computing elements by coupling a choice of processors from the Intel® Atom™ processor E3800 product family with an FPGA. The FPGA enables custom I/O timing, control, and processing functions.
The Intel Atom processor E3800 product family benefits from Intel's 22 nm process technology with 3-D Tri-Gate transistors. The processor's microarchitecture incorporates an updated out-of-order execution engine and other improvements to deliver up to three-times higher performance and consume as little as one-fifth the power of the previous generation. With extended temperature support and BGA packaging, these processors enable the NI CompactRIO Controller to reliably perform in rugged industrial applications and harsh environments. In addition, their integrated graphics and support of common display interfaces make them ideal for powering HMIs.
Software for Innovation
The NI CompactRIO controller's development and runtime software lets developers focus on innovation, rather than implementation. It includes constructs to manage timing and memory, built-in drivers and APIs to move data between heterogeneous components, and over 950 available functions to assist with signal processing, analysis, control, and mathematics. Developers can unlock the power of FPGAs without any knowledge of hardware description languages, placing and routing tools, or complex constraint languages and files. The result is a system that can be customized and reconfigured through software – even after deployment.
The Industrial Internet Consortium (IIC) and industry leaders such as NI, Intel, Bosch Rexroth, Cisco, KUKA, Schneider Electric, and TTTech are collaborating to create a TSN testbed. The testbed will showcase the value of TSN in an ecosystem of manufacturing applications (Figure 5).
This testbed, hosted at NI corporate headquarters in Austin, Texas, will combine different critical control traffic such as the Open Platform Communications Unified Architecture (OPC UA) and best-effort traffic on a single resilient network based on the IEEE 802.1 TSN standard. The setup will enable demonstrations of TSN's real-time capabilities and vendor interoperability for applications such as industrial machines, electrical grids, and transportation systems connected on standard converged Ethernet.
Building the IIoT with TSN
As IIoT adoption continues, ever-increasing amounts of data and widely distributed networks will require new standards for sharing and transferring critical information. Just as an ambulance or fire engine receives priority among other traffic during an emergency, the TSN standard as implemented in industrial devices such as the NI CompactRIO controller will ensure that critical, time-sensitive data is delivered on time and cost effectively over standard network infrastructure.