Inside the Development of Autonomous Mobile Robots

Christina Cardoza: Hello and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech, and today we’re going to be talking about autonomous mobile robots, which is an industrial trend I’m happy to finally see gaining more traction in manufacturing space. So, joining me to tell me more and to talk about this topic we have Claire Liu from congatec and Timo Kuehn from Real-Time Systems. So, before we get started, Timo, I’d love to learn a little bit more about yourself and what you do at Real-Time Systems.

Timo Kuehn: Hi, my name is Timo. I’m a Systems Architect and Product Manager at Real-Time Systems. I’ve been part of RTS since day one, which is 17 years ago. Before joining RTS I was a software engineer at the robot manufacturer Kuka, and now I’m happy to share my knowledge and insight with you.

Christina Cardoza: Yeah, absolutely. Wow, 17 years. So I’m sure you’ve seen this space evolve quite a bit over your time at RTS. Claire, please tell me more about yourself and congatec.

Claire Liu: Hello Christina. Thank you for having me. And hello everyone, I’m Claire, the Product Marketing Manager from congatec. And congatec is a company that provides computer modules for embedded applications. My role in the company is to look at congatec’s products and the technology to see how they apply into the robotic market, like in autonomous mobile robots, and see how congatec can help autonomous mobile robot companies to take all the challenges in the competitive environment with computer and module concepts.

Christina Cardoza: Yeah, absolutely. And I’m sure there’s a lot of complexity and challenges in the computer market that, when you’re trying to add these autonomous mobile robots into the factory, that businesses come across. And so I, like I said in my intro, I’m very excited to talk about this topic because I feel like autonomous mobile robots—this is something that we were dreaming about a couple of years ago. You know, something that seems like science fiction, but over the last couple of years or even in the last year or so we’re seeing them become more prevalent in the factory space.

So, Claire, I just want to start this conversation with you talking about, when we say autonomous mobile robots, what are we actually talking about? And why is this, as of lately, taking the industrial space by storm?

Claire Liu: Autonomous mobile robots, robots or robotic systems, they are capable of operating independently without direct human intervention. And those robots are equipped with defense sensors, artificial intelligence algorithms, and a sophisticated control system that enables them to navigate autonomously, to perceive their environment, and to make decisions. And right now why the industry is increasingly interested in autonomous mobile robots is because the benefit, the many benefits they can offer, for example like material-handling tasks.

These material-handling tasks used to be executed manually; workers need to perform those tasks to pick up and deliver the raw material and the in-process product on the production line. And those tasks actually acquire repetitiveness and can post a risk on workers’ health and safety. Right now, by using autonomous mobile robots to automate the material-handling process, to transport the products on the factory floor, workers right now, they don’t have to waste their own production time to do those manual work and tasks. They can focus on highly skilled and more value-added tasks.

So by using autonomous mobile robots in the manufacturing environment, it streamlines the manufacturing process, can increase productivity, can improve operation efficiency, and enhance workers’ safety.

Christina Cardoza: Yeah, that’s great. And you mentioned these are robots that are moving on their own. You mentioned a couple of different components in there, and obviously the benefits are improving worker safety and allowing them to focus on more important tasks that they, instead of doing these repetitive tasks, we can get the robots to do them, but we also see these robots working alongside workers.

And so I’m curious, because we have these robots moving around independently, how do we ensure that they don’t hit other workers, they don’t mess up the production line, they don’t mess up equipment. So how exactly—if you could give us a little bit more insight into how they work. I know you mentioned AI and all of that. So how does this all go into that?

Claire Liu: Yes, of course. Autonomous mobile robots rely on a combination of technologies including sensors, AI algorithms, motion control, wireless communications, and the computing platform. Let’s enable autonomous mobile robots to perform the text autonomously. And autonomous mobile robots utilize various sensors, for example like LiDARs or 2D or 3D cameras to perceive their environments. And that sensor data is processed in real time by the computing platform. And that sensor data actually profiles the information about the environment. And the robot can use this information to create a map to localize and navigate themselves to the destination autonomously and to perform the task within the environment.

And I’ll just explain—autonomous mobile robots are intelligent machines, and when it comes to develop the next future-oriented autonomous mobile robot, Intel’s^® 13th Gen Core^™ processors with congatec’s computer module is an ideal solution. Intel’s 13th Generation Intel Core processors lets—combine the power, efficiency, flexibility, performance, and deliver the boost computing performance prior to previous generations. And it’s a very good solution for a defense robotic computing platform. And MrCoMs right now actually can benefit from this latest Intel processor to run more applications simultaneously and run more workloads and more connected devices.

Christina Cardoza: Yeah, that’s great to hear that you guys are using the 13th Gen Intel processor. I know Intel had just released that. So it’s great to see you guys taking advantage of all the latest technology out there, because, like you mentioned, manufacturers, they have a lot more workloads and they’re very high-intensive for computing, so they want to make sure that they are using the best technology.

Timo, I’m wondering if you can, from Real-Time Systems, if you can also talk about the software architecture that goes into this, and how you help developers and manufacturers meet the real-time demand of these robots.

Timo Kuehn: Yeah, a lot of software goes into AMRs of course; there are various functionalities like perception. The robot has to perceive its environment in order to know what’s going on, localization to find out where it’s situated at the moment; the path planning—the system is autonomous so it needs to find out where to move to. The movement itself, the motion control, is very important; obstacle avoidance of course; interaction with humans, depending on the type of robot and diagnostics.

So a lot of software goes into such a system, and those software functions have to be mapped by the corresponding software modules, and often they have very high requirements or even competing requirements in terms of timing and resource usage. Competing requirements means, for example, if one software module needs a lot of performance, while a different software module needs a deterministic response in a timely manner, you cannot just throw everything in and make it work. It is quite complex.

So typically real-time operating systems are used, or operating systems that have real-time capabilities in order to have party-based scheduling and making sure that deadlines are never missed. Critical tasks like perception or motion control can get higher priority so they don’t get interrupted by lower-priority tasks.

And, especially for motion control, it can be quite challenging. It needs determinism, of course; it needs to react to sensor signals within a predefined time frame. The time frame depends on various factors like: Do we have wheels? Do we have axes? How many axes have to be controlled? What is the speed of the AMR? What precision is needed? Is the device moving in two dimensions or three dimensions? Is the load dynamically added or unloaded? As you can see, there are many different options and it can get quite complex.

Resource allocation and optimization is something that is important and has to be provided by the operating system or the software architecture. And it is necessary to have some moderate design and component-based development for the separation of different functionalities, which makes it easier for independent development, testing, release, and updating of the individual modules. Often third-party code has to be integrated, containers are being used, and not to forget time synchronization between the different modules so we don’t have a lot of overhead or locking so everything works smoothly together.

Christina Cardoza: Great. So it sounds like a lot really goes into all of this. Timo, you mentioned all of these different things to make sure that you have the memory, you have the computing, you have the bandwidth to do all of this. One thing you mentioned that I’m interested in hearing a little bit more about is the different modules that go into this, and taking a modular approach or a concept when developing these next-generation autonomous mobile robots.

So I’m interested in hearing a little bit more about why a modular concept is something that you guys are utilizing. Claire, maybe you can tell me more about it from congatec’s perspective.

Claire Liu: Sure. So, congatec’s computer modules leverage Intel’s processor technology scale seamlessly, from low power to high computing performance, enabling developers to develop their robots to work longer, smarter, and perform complex tasks with higher proficiency and efficiency. Developers right now can actually quickly and easily adapt to the latest Intel processor technologies through a simple module change and add intelligence to their autonomous mobile robot even after years of operation. Additionally, Intel OpenVINO^™ toolkit is offered, which provides comprehensive support for developers and for optimized AI influence models. Intel OpenVINO toolkit simplifies the development of deep learning applications on inter platform.

Christina Cardoza: Yeah, and I think that OpenVINO piece of this is extremely important. You know, like you said, making sure that you can add the intelligence and the deep learning models onto these robots. I know OpenVINO, with its latest releases over the last couple of years, they’ve made it very easy, that they really help developers utilize the hardware that’s available to them in the best possible way.

So, we’ve been talking about the AI toolkit OpenVINO, we talked about Intel processor technologies and the 13th Gen Core. So a lot goes into this to making sure that robots can sense, they can see, they can conduct operations, they can take tasks and orders and things like that. I’m curious, because I’m sure there’s still a lot more that goes into making this possible, Timo, what are the other tools and technologies you’re seeing go into developing autonomous mobile robots? And how can developers take advantage of some of these tools like OpenVINO to really simplify their efforts?

Timo Kuehn: As we just learned, the development of AMRs requires a combination of hardware, software, and connectivity. In terms of hardware, there’s the computing platform, the chassis, the motors, the sensor power systems, and of course what exact sensors and so on is being used depends on the requirements of the application.

On the software side: perception, localization, path planning, motion control, obstacle avoidance, interaction with humans and diagnostics, as we talked about before, play very important roles. And, as you can imagine, integrating and managing all of those functions can get quite complex. So you cannot just add a control unit for each of those functionalities because it has its limitations and of course it hits physical limits very quickly.

AMRs are battery powered, so adding a lot of controllers doesn’t make sense. The controllers need to be connected with each other, that adds weight, increase the size, costs, and complexity. So this is why multiple functions must be consolidated on fewer processors.

And here is where an embedded real-time hypervisor can help a lot, integrating multiple workloads on a single processor, that has many advantages, isolation and security, for example. So, for example, perception, motion control can run in their own virtual machines securely separated from each other, making sure that when one VM needs a lot of load or creates a lot of load, the other one is not affected and can still meet its deadlines.

And this is really crucial. Imagine there’s a signal from a sensor and the reaction from the MR or from the controller comes too late. This can lead to a crash or even to injuries when humans are involved. It helps with performance optimization, load balancing; every VM can get dedicated resources to meet timing and performance requirements.

And the system gets more modular and flexible; you can update and modify various functions individually, especially on Intel processors, because all of the modern Intel processors have virtualization extensions like VTX and VTD. They allow for secure separation of resources and high performance at the same time. And, in addition to that, the Intel time-coordinated computing, the TCC, provides temporal isolation of workloads. This really guarantees the determinism, so you can have both deterministic workloads and performance securely separated in space and time on the same processor.

Christina Cardoza: So, I think we’ve done a great job of laying out for developers and businesses all the tools and technology and components they’re going to need to make AMR development successful. What I would love to hear from you guys next is after they develop an autonomous mobile robot, what it actually looks like in the factory. Where are the biggest opportunities for manufacturers, or what are the biggest use cases they’re using AMRs for today? And where do we still have to go? So, Claire, I’ll start with you on that one.

Claire Liu: Yeah, okay. Autonomous mobile robots have proven to be versatile in various industries, and some use cases, including the material handling I just mentioned in warehousing, in logistics, and, for example, their fulfillment for e-commerce and even collaborative assembling in the manufacturing environment. During the pandemic autonomous mobile robots were utilized for delivering medical supplies and delivering medication and assisting patient care.

Additionally, autonomous mobile robots, right now they are finding more applications in other areas, like in agriculture, in hospitality, and retail. And the possibilities are intensive as the technology continues to evolve. New use cases are consistently emerging.

Christina Cardoza: Yeah, absolutely. And you make a great point: when I think of autonomous mobile robots I think of them in this manufacturing, industrial setting. But, you know, really there are so many other different industries that can take advantage of this technology. Timo, I’d love to hear from you where Real-Time Systems sees the biggest advantages for AMRs.

Timo Kuehn: So, Claire already mentioned optimizing warehouse operations, where AMRs can conduct inventory audits and optimize storage configurations. There are many more use cases, for example in hazardous environments for inspection of, for example, power plants, reducing the risk for human workers. They can be used in public places to provide real-time video feed. Or, for example, in large facilities they can be used in last-mile delivery to transport packages. They can assist in material transportation, also in construction projects. Environmental monitoring is a good use case for AMRs in order to collect data on air quality, water quality, or soil conditions. Hospitality services has been mentioned before, and they can also assist passengers in transportation, guide them, help with mobility challenges, and so on. So there are really a lot of different use cases, and I think it’ll become more in the future.

Christina Cardoza: Yeah, I can expect seeing, in the near future too, more of these AMRs just prevalent in our everyday lives. Just a small example is like in my supermarket there’s a little robot moving around the retail store trying to identify hazards and spills, and I think that’s really the first implementation of it that we’ve been seeing. But it’s only going to get smarter, more intelligent, with all these things we’ve been talking about. And soon we’ll probably see them stocking shelves or helping customers in some other ways.

So, Timo, you mentioned a lot of other different use cases that we can expect or that we’re gearing up for. I’m curious what other advancements or what else do you think we can expect in this field over the next couple of years?

Timo Kuehn: Well, it’s of course hard to predict, but I’m sure there will be many advancements in the near future, especially with Intel processors with integrated AI accelerators. So this will lead to enhanced perception and object recognition, more intelligent path planning and optimization, and, of course, adaptive-learning capabilities. What we can also imagine is improved collaboration between humans and robots. You have things like capability of making complex decisions in real time, for example, to assess situations and execute complicated tasks with only a little human intervention.

So, to summarize: the combination of virtualization technology, real-time capabilities, and integrated AI accelerators has a high potential for completely new types of autonomous mobile robots. They will become more intelligent, adaptable, and capable of performing complex tasks with high precision and efficiency.

Christina Cardoza: Absolutely. Well, we are running out of time. I’m sure we’ve only scratched the surface of this conversation of what these AMRs can do, how we can successfully build them, and what we have to look forward to. But before we go, I just want to throw it back to each of you for any final thoughts or key takeaways you want to leave our listeners with today. Claire, we’ll start with you.

Claire Liu: Okay. As Timo mentioned, we will expect more new and exciting possibilities in the field of AMRs in the near future, and technological development evolving rapidly in the robotic area, with a modular approach to hardware systems and the software-architecture design, autonomous mobile robot companies can adapt to the fast-changing environment and bring their cutting-edge solution to life with great scalability.

Christina Cardoza: Absolutely, Claire. And flexibility—I think flexibility and scalability, like you guys have mentioned, that’s really going to be key, because we are implementing AMRs to meet needs and benefits today. But those—like you said, it’s a fast-changing environment—those needs and those—the technology’s going to advance, those needs are going to change, and we need to make sure that we’re able to develop these systems and future-proof them as we go on.

So I just want to thank you both again for joining us today and for the insightful conversation. I can’t wait to see what else congatec and Real-Time Systems come out with in this space. And I encourage all of our listeners to keep up with them. We’ll put their links in our bio so that you can continue to follow along what’s going on in this space. But, until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel^® trademark information.

This transcript was edited by Erin Noble, copy editor.