Red Hat Principal Software Engineer Ricardo Noriega de Sota presented an overview of the edge computing landscape and a demo highlighting the potential of AI at the edge during a keynote presentation at the January 18, 2024, CODECO Industrial Workshop. The talk addressed challenges faced during edge adoption and explored the state of the art in open source for developing and scaling services across cloud and edge environments. The workshop was part of the HiPEAC (High-Performance Edge and Cloud Computing) conference in Munich, Germany, and it focused on demonstrating new concepts under development for flexible and decentralized edge-cloud orchestration.
From the cloud to the edge
The computer industry has been trying to centralize workloads for decades. To address that need, we built datacenters with thousands of servers and distributed computing platforms—a.k.a. the cloud. Now medical, automotive, manufacturing, and agricultural industries are modernizing and generating huge amounts of data. At the same time, more and more data-collecting devices are connected to the Internet, such as smart lights, sensors, IP cameras, and even cars. Much of this data must transit through public networks to be processed in a centralized cloud location. This raises many concerns about security, network capacity, resource usage, and data currency.
Edge computing can address these concerns by putting computing power closer to where the data is generated. Consider an anomaly detection model on a factory floor. Sending a video stream back to the cloud to perform the inference process is prohibitively costly and increases the likelihood of data leaks, network congestion, and latency. By contrast, a GPU right on site makes this process viable.
AI at the edge
During the workshop demo, Noriega demonstrated how to run two different AI models in an edge field-deployed device. The demo used NVIDIA Jetson Orin AGX running Red Hat Device Edge. Red Hat Device Edge is the conjunction of an edge-optimized operating system such as RHEL Edge (OSTree-based) and a lightweight Kubernetes distribution such as MicroShift. A webserver on top of that showed YoloV8, an object detection algorithm connected to a camera, and Vicuna 7b, a large language model (LLM) running locally.
The goal of the demo was to connect eyes (the camera) to a brain (the LLM). As the camera detected people, chairs, and other objects in the real environment of the conference room, presenters used a chatbot to ask the LLM questions requiring interpretation of YoloV8’s object recognition, such as the number of people or chairs in the room. While this demo was not as complicated as an anomaly detection model in a factor, it illustrates the potential of leveraging AI technology for cloud-edge computing.
The CODECO project
The CODECO (Cognitive Decentralized Edge to Cloud Orchestration) project is building an open source software framework, pluggable to Kubernetes, that improves the efficiency and robustness of edge-cloud architecture. The CODECO platform enables cross-layer orchestration among decentralized data flow, computation, and networking services involving applications that can run on both the edge and on the cloud simultaneously. In particular, the project extends the traditional Kubernetes definition of an application, augmenting it with additional non-functional parameters designed to ensure that a broad range of application requirements are considered, such as desired QoS, energy expenditure and application robustness, as well as application resource, network and data requirements. The CODECO framework components then work together to ensure that this version of the application is delivered in real terms, primarily through intelligent scheduling of the application.
A Smart City use case from the University of Göttingen, Germany, highlights the power of the cloud-edge combination. Cameras positioned on traffic lights evaluate the load on the road. The evaluation is done at the edge since the bandwidth (4G cellular network) does not permit sending the video stream to the datacenter. Features of the picture are sent to the datacenter, where a global view of the city is created. Based on this view, the traffic control system sends commands to change the traffic lights in order to optimize the traffic in the entire city.
CODECO involves a consortium of 16 industry and research partners across EMEA, including Fortiss, Inova, Intracom Telecom, Athena, Atos, the University of Göttingen, Siemens, Netcompany Intrasoft, the Eclipse Foundation, the University of Piraeus, i2CAT, Telefónica, the Polytechnic University of Madrid, Red Hat, and IBM. Red Hat engineers are in charge of developing the management plane that enables moving workloads to optimize resource usage.
Project milestones
To date, the project has defined the overriding architecture and partners have been busy developing their individual components. We are now at the exciting stage of integrating these components to deliver an end-to-end working system for a single cluster environment. Following this we will turn our focus to extending the system to a multicluster cloud-edge environment. For further details on the project please contact Josh Salomon or Ray Carroll. Further information can be found on the CODECO project website, and you can follow the project on LinkedIn.
CODECO is funded by the European Union under Grant Agreement No. 101092696. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Research Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
