AI AT THE EDGE

Christina Cardoza: Hello, and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech, and today we’re going to be talking about the next generation of AI development with Paula Ramos from Intel. But before we get started, let’s get to know Paula. Hi, Paula. Thanks for joining us.

Paula Ramos: Hi, Christina. Thanks for having me here. I’m so excited with this conversation.

Christina Cardoza: Yeah, absolutely. What can you tell us about yourself and what you do at Intel?

Paula Ramos: As you mentioned, my name is Paula Ramos. I am an AI Evangelist at Intel. I made my PhD in Computer Vision and Machine Learning some years ago, but before Intel I was a researcher in applied engineering. I was the tech leader of multiple teams designing, building, and deploying AI technology at the edge. And I have more than 17 years of experience bridging the gap between technology and agriculture. So now in Intel I am a computer vision and AI advocate, and with my knowledge I can also bridge the gap between the technology and developers.

Christina Cardoza: Awesome. Well, I’m very excited to get into the conversation with you today. You know, we’ve done a lot of work outside of the podcast together, so I know you have just a wealth of information in this area. And obviously a AI development, that’s a big general area that we’re talking about, but I wanted to start—the last time we spoke and last time we saw you was at Intel Innovation a couple weeks ago, so I wanted to start there. There was a lot that came out of different companies and different industries, so what were you seeing at the event, and even outside of the event, on how AI is advancing and solving these real-world problems?

Paula Ramos: That is a great question, Christina. Because AI is advancing fast. We can see new things coming every single day, and we can see also real-world problems turning into amazing solutions with AI. So AI has impacted the world for a while; now there is more awareness for AI.

So AI, for example, has helped people to communicate with others using translations. AI can translate text between 100 languages to others. Another great example is a self-driving-car system that some vehicle brands are using to control the vehicle steering, acceleration, and braking, with the potential to reduce fatalities in traffic accidents. AI also can help doctors to diagnose cancer and develop personalized treatment plans, or accelerate the deployment of new drugs and treatments based on the capability that AI must predict the 3D structure of the proteins. AI is also helping a lot of farmers to reduce the use of pesticides and herbicides by up to 90%. So AI is helping humanity to solve their problems faster, making the human race more feasible.

Christina Cardoza: Yeah. It’s amazing to see AI advancing and evolving in all of these different industries. And like you said: it’s changing every day. And to make some of these capabilities —actually to make all of these capabilities possible, you really need developers building these solutions, staying on top of the next generation. And not only is AI in different industries evolving every day, but so are the capabilities, solutions, technologies to make it easier and more accessible for developers. It used to be the case where AI developers had a very specialized skill set. Now all developers are expected to have AI development skills, and the tools are making it easier for them to do so; it’s making them more accessible.

So I wanted to see what you thought about the importance of making AI more accessible for developers. What you’re seeing in this space and how developers can really leverage this transformation.

Paula Ramos: Christina, this is a great question, because this resonates a lot with the role that we have as AI evangelists at Intel. You know, we are trying to create more accessible information to developers and how they can accelerate the speed of AI innovation. So basically for that specific question I have three thoughts.

So, the first thing is just recapping what we are talking about—making  AI easier and more accessible for developers. So the first thing is to accelerate the speed of AI innovation. You know, this is super important, because the more developers who have access to AI tools, the faster the technology will advance. By making AI more accessible we can open up the field of new ideas and lead the deployment of new innovative solutions. Basically, imagine all the things that they can make right now.

So, the second point is to democratize AI. We need to be sure that AI is for everybody, and every single developer has the opportunity to benefit from this technology. By making AI more accessible we can help to bridge the gap—this adoption gap.

And the third one, but is not the less important, is that we need to solve the AI talent shortage. So, there are right now—and this is one of the top jobs that we can find—there is high demand for AI developers, and there are not enough developers in the world. There is a shortage of skilled workers to meet the demand. So making AI easier to learn and use we can help to train more developers and close this talent gap.

You know, developers should have the proper AI tools, but also access to the latest hardware improvement is important. So this is another tool, because this enables developers to build and deploy applications more efficiently and effectively.

Christina Cardoza: Yeah, absolutely. And, like you mentioned, Intel is making a lot of progress in this space to make it possible to democratize AI and give them the tools and the capabilities that they need. Like I mentioned when we caught up at Intel Innovation a few weeks ago, Intel has made a lot of these announcements solving some of these challenges or issues that we’re talking about.

So can you talk about some of the latest advancements or announcements—what’s going on at Intel exactly. What are the tools and the hardware and capabilities you guys have coming out to address some of these developer concerns?

Paula Ramos: I have a lot of fun at Innovation. That was an amazing event. We have two days full of new and exciting things coming from Intel, something that is coming in also in the next months. So it was great for me to see, for example, how startups are using silicon power with the capability of AI to solve real-world problems.

Personally, I’m so excited with the Intel announcement of the new AI PC. So, I had one of those in my hands for showing our generative-AI booth, and it showed a great performance. So, basically that new laptop or processor is a new generation; it is the Intel^® Core^™ Ultra processor with GPU incorporated, and also the process of a new element called NPU, Neural Process Unit.

And it’s also good to see how OpenVINO^™ and the inference—you know what OpenVINO is? But maybe listeners need to be more familiar with OpenVINO. So, OpenVINO is the inference and deployment framework that Intel has. And it also has the capability to run OpenVINO in the AI PC. It is really, really good. One of the things that I also realized during the keynote of Pat Gelsinger was that OpenVINO increased downloads in the past year 90%. So it is showing us the potential we have at Intel to leverage the AI era.

A lot of people are thinking that Intel is just a hardware company, but we are making a great job showing developers how they can easily improve their solutions using frameworks or systems such as OpenVINO. And Pat Gelsinger and his team made a great job, an excellent job, on stage. So, we saw Llama 2 chatbot, an LLM model, running on an AI PC, locally in a Windows machine. He did run generative-AI chatbots locally, instead of sending that data out to the cloud. It is more secure if we allow users to utilize AI without an internet connection, for sure. We have a lot of security reasons to do that.

And another thing that also was fun to me was using also generative AI in the keynote when Pat and his team were able to generate a song in the style of Taylor Swift. So it seems that Pat is a Swiftie.

Christina Cardoza: Yeah. And just looking at OpenVINO and the, just the awareness and how it was brought to the forefront at Intel Innovation really highlights how important AI has come to be to all industries. OpenVINO is an AI toolkit; I believe it just celebrated its fifth anniversary, but a couple of years ago at Innovation we weren’t hearing much about OpenVINO, and now it’s everywhere. Everybody’s using it, and, like you said in the keynote, we saw different examples of how OpenVINO is being used—like in the fitting room experience with FIT:MATCH and things like that.

You mentioned generative AI. I know OpenVINO is keeping on top of all the trends. Obviously generative AI is a hot topic these days, and OpenVINO came out with its latest release with generative-AI capabilities. So, how are you seeing—what’s the best way for the listeners who haven’t dabbled in OpenVINO yet, or for ones that want to take their AI-development efforts even further, how are you seeing them use OpenVINO? What’s the role of OpenVINO in their AI-development efforts, and what are just some other interesting use cases you see partners in the ecosystem leveraging this toolkit?

Paula Ramos: So I think OpenVINO, as I mentioned before and also you mentioned—that is the inference and deployment toolkit that Intel provides to developers on client and edge platforms—with OpenVINO Intel is making AI more accessible. So you can optimize neural network inference across multiple hardware platforms, and OpenVINO is powering AI at the edge.

An evidence of that is what I mentioned before, that we can see that the developer downloads have increased in 90% in the past year. We can see that OpenVINO makes generative AI more accessible. We are solving the real pain points to developers. We are working—the main goal of OpenVINO is to run the optimization and quantization with the models so we can reduce the size of the models, we can reduce the memory footprint, we can also run the models faster in our wide range of hardware. I’m talking about Intel and non-Intel hardware.

And the most important thing is that once you have the model in the intermediate representation format—that is the OpenVINO format—you can deploy it everywhere. And this is some of the differences that also we have with some of the competitors. So we can see successful stories in use cases with the adoption of OpenVINO.

So we can see that we are hosting every single industry. And this is just to give you an idea, an example: the smart—and I know that you know pretty well, but for listeners, OpenVINO is everywhere: in smart cities, manufacturing, retail, healthcare, and also agriculture. And, as I mentioned, this is just to give you an example.

For the new listeners or the new people that are in this field of AI, I invite you to google OpenVINO use cases, and you can see a lot of great examples. And also we have a great way to start if you want to do that: we have created the AI reference kits, and you can find that if you put, like, “Open Potential OpenVINO” and you can find that in Google.

Christina Cardoza: Yeah, that’s great. You know, obviously OpenVINO is, just like you said, in everything. There’s so much to learn about it, and there’s so much you can do with it. But OpenVINO isn’t the only sort of software tool kit that you guys have out there or is available to the developers. So I want to get into Intel^® Geti^™ a little bit.

Last year at Intel Open—Intel Innovation, you guys announced Intel Geti, which is designed, like you said in the beginning, to democratize AI not only for developers, but for business domain users. And I believe that it connects with OpenVINO to really connect business users and developers together. So this year it was great to see how much that Geti had matured over the last year. And this year at Intel Innovation we learned that there was a new software development kit for developers to start working with and start leveraging this solution even further.

So can you tell us a little bit more about Geti—how it’s going to be probably another three-part question, but how Geti works with OpenVINO and what developers can use with the SDK, how developers can leverage it?

Paula Ramos: Yes, yes, yes, for sure. I really want to make first an introduction about what Intel Geti is. So maybe listeners they don’t—are not familiar with that. So, Intel Geti platform is a computer vision platform that helps organizations to rapidly develop computer vision models. So, in short words, it’s a platform that brings all necessary things together. So we have annotation, training, optimization, testing.

There are a lot of benefits for using the Intel Geti platform: data scientists, machine learning professionals, systems integrators, and domain experts can work together using the same platform. This is because Intel Geti is an easy-to-use platform that also has the potential to control multiple aspects of the training and optimization process. And as, Christina, you mentioned that OpenVINO is behind scenes of Intel Geti platform for the optimization and quantization task, but also the platform can provide us modeling different formats. And one of these formats is the intermediate representation format OpenVINO. And I can deploy my model everywhere and have all benefits of the deployment with OpenVINO.

In another way, Intel Geti platform also offers a software development kit that is the SDK that helps users to take advantage of easy-to-use functionalities. So the Intel Geti SDK utilizes OpenVINO to build deployment pipelines and accelerate inference on various Intel hardware platforms that include CPUs, GPUs, and without needing to be an expert in computer vision. That is also the beauty of this platform.

With the SDK we can have real interaction with the project in the Intel Geti platform. So we can create projects, upload your annotation, upload new production data or previous data, and you can modify the trainable features, test the deployment running in the server for the testing purposes, and also download your deployment for running that locally.

The deployment with the Intel Geti SDK makes this super easy for developers since the SDK is agnostic to the computer vision task and also agnostic to the model architecture. So developers don’t need to prepare the data to the model input and don’t need to prepare the model output for showing the results. This is straightforward how we can run the inference using the Intel Geti SDK locally. So we have the advantage to use also SDK with the OpenVINO model server. That means that we can also scale the capabilities of our models in different deployments. I’m so excited with Intel Geti.

Christina Cardoza: Yeah, no, absolutely. With Intel Geti OpenVINO and all of the improvements in the Intel hardware that are coming out, you guys have really created an end-to-end solution not only for developers for businesses to make all of this happen and make it this next-generation of AI solutions and use cases possible.

Now, we’ve been talking about making it accessible and democratizing AI for developers mostly in a general sense. I want to dig down a little bit further in a specific area that you have done a lot of work in, and that’s empowering women to get into the AI field. And you’ve done a lot of work in this, and I’ve seen interviews that you’ve had about this topic.

So I’m curious, why are you so passionate about this initiative, and can you tell us anything about how we can get women more involved—why it’s important to, and what are some of the challenges or the barriers that they face getting into the AI space?

Paula Ramos: Yes, for sure. I’m so passionate with this topic. You know, personally I think I represent a global tech workforce of two minorities. The first minority is women in tech or in AI, and the second is Latin women in tech. We can interpret “tech” as also artificial intelligence or engineer in general. But just take a look at this statistic, and you can see also why I can see this is important—just that 50% of the global tech workforce are women and just 2% are Latin women. So this is a huge underrepresentation.

Basically, under that scenario I want to inspire more women in the world to work in AI. And I want to contribute to reduce the lack of the access to education and training, because we need to be clear that it’s also coming because we have a lack in education in Latin countries. In South America we have a lack in education, so we need to work more on that. Or maybe women are not super familiar from the very beginning in the education with technology. So we need to include more girls in STEM courses and all.

I also want to reduce discrimination and bias. You know, everyone deserves the opportunity to success in tech regardless of the gender. I think that this is a stereotype that we have in our minds. Women can also sit at these tables and have a serious discussions about technology. Women bring a unique perspective about how to solve problems. And I can see that this is a great point: that we have the unique skill to create products and services that meet the needs of all users. That doesn’t mean that the men don’t have that—for sure, they also have that, but women have that specific skill, you know, because we are moms, so we are designed to solve problems, these kind of problems.

And we can also see—and this is just an example that I have in my mind, on the top of my mind—is we have significant contributions to the field of AI. And just to give you an example, I want to talk about Dr. Fei-Fei Li. I don’t know if you’re familiar with her, but she is an AI researcher and Co-Founder of the Stanford Human-Centered AI Institute. And because of her work we started the work in deep learning. She contributed to the birth of deep learning; she developed the ImageNet initiative, and that initiative played a major role in the deployment of deep learning.

You know, this is the impact that one woman has made in AI, and this is remarkable. I can imagine how many ideas, how many women can contribute also to the field. I’m so excited, I’m so passionate with this, and I really want to inspire more women in this field.

Christina Cardoza: Absolutely. And it’s great to see women being recognized for their work in this field. You mentioned Fei-Fei: she accepted—got the Intel Innovation Award at the event earlier in October. So it’s great to see all this progress being made already in this space. And you made a great point: it’s not only about making tools and capabilities and technologies available to developers, but it’s really the education and the resource aspect—making those available so that more people can get in the field and we can make this really accessible to everybody, not just people that are already in the field. So I think that’s great.

Unfortunately, we are running a little bit out of time, but I wanted to throw it back to one last time before we go if there’s anything you wanted to add about the future of AI development, how developers can get started, and where you think this space is going.

Paula Ramos: This is a great thing, this is an important message that maybe I want to share with developers. The first question is, where are your dreams? And you can achieve those goals, because AI is a powerful tool that has the potential to make the world a better place for everyone. And we need to build an AI that is inclusive, fair, and beneficial to all.

So, try new technologies, new models and algorithms. You can use OpenVINO, why not? In my personal experience it will help you to achieve your professional and personal goals regarding AI, for sure. So try to participate and be an active contributor in an open-source project, and stay tuned for the latest trends in AI to make AI more practical. This is my other advice. AI has a lot of tools, the engine is in your imagination. That’s all, Christina. I’m glad that you invited me. That was super fun, and I hope that listeners can find this also helpful for their career in AI in general. Thank you, Christina.

Christina Cardoza: Absolutely. Thanks for joining us. It’s been a very informative and insightful conversation, and I can’t wait to see what else Intel does in this space. You know, you guys are making improvements every day, and every day we’re publishing articles on insight.tech showcasing exactly how AI is changing and transforming the world in different industries, and Intel technologies are all powered behind it.

So I would invite all of our listeners to take a look at insight.tech to see how other industries are leveraging Intel and AI, as well as keep up to date on Intel. Visit their website and some of their platforms that they have to see what else you can do. I know Intel has a lot of reference kits and, Paula, you’ve done a lot of different tutorials and videos on intel.com also, walking developers through some of these steps and capabilities. What is the link or the website how developers can get to that and learn a little bit more about what to do with the development kits, with AI experts like yourself?

Paula Ramos: Yes. You can find that if you put in Google, “Open Potential Intel OpenVINO.” You can find the link, or also we can share the link maybe in the description of this podcast and you kind of start from scratch. So, if you are not familiar with AI, also you can start from scratch developing different kinds of solutions for smart manufacturing, retail, and more. So, more things are coming to that space. We are trying to create tutorials, we are trying also to create mindful messages that you can apply in your career or in your actual positions. Stay tuned, because we have more things coming for 2024.

Christina Cardoza: Absolutely. Well, I’m excited to see what else comes out over the next year. But until then, thank you again for joining us, and thanks to our listeners for tuning in. Until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel^® trademark information.

This transcript was edited by Erin Noble, copy editor.