By Ahmed Sanaullah
In 2023, Red Hat Research announced the launch of the Co-Design (CoDes) research lab during the Massachusetts Open Cloud (MOC) Alliance Workshop. Our goal was to build an ecosystem that could deliver on the immense value proposition of non-traditional hardware. Non-traditional hardware here refers to both specialized hardware (e.g., Field Programmable Gate Arrays and Application-Specific Integrated Circuits) and atypical configurations of typically general-purpose hardware, such as microcontrollers, single-board computers, emerging CPU microarchitectures, and application-specific infrastructure configurations. As an incubator for this innovative set of technologies, CoDes can provide the fundamental stepping stones needed to make non-traditional hardware part of mainstream research and development across cloud and edge deployments.
In the year since the announcement, the CoDes lab has grown significantly and successfully in pursuit of this goal. With the 2024 MOC Alliance Workshop just around the corner, we’re taking this opportunity to look back at why the CoDes lab was needed and what we have accomplished, and to give you a peek at the exciting journey ahead.
Why hardware-software co-design?
We were motivated to build the CoDes research lab by two simple but critical needs. First, researchers need access to cutting-edge hardware, which is often underutilized (or not available at all) in shared environments, and this access must be provided in a way that makes development efforts reusable and extensible. Second, maximizing the value of non-traditional hardware requires simultaneous innovation in both hardware and software through co-design (hence the lab’s name).
From a hardware standpoint, it was important to provide researchers the option of hands-on access to make configuration changes and experimentation easier. This enables the design and deployment of hardware in a manner that facilitates effective complexity offload from software. We thus created CoDes as an on-premise lab located at Boston University as part of the Red Hat Collaboratory and is open to students, researchers, and engineers for undertaking all sorts of hardware-centric projects—from blinking LEDs to controlling robots to managing datacenters.
For software, we needed to build an ecosystem that would progressively approach traditional CPU-like levels of usability when targeting non-traditional hardware. CoDes provides a consistent development environment across all machines, and all projects must be compatible with this environment. There is also an emphasis on building extensible frameworks so that greater complexity can be iteratively (and easily) added to projects. Our goal with this approach is to make CoDes a place where innovations from one project, such as FPGA infrastructure, can enhance other projects, such as federated learning.
One example of the type of project enabled by CoDes is PCIe stack development for FPGAs. By reconfiguring the FPGA to emulate a VirtIO device, we can use upstream VirtIO drivers for host-device communication. This, in turn, means we don’t have to rely on drivers from device vendors. While the project has a large value proposition, it cannot be done easily in shared environments since it requires giving privileged access to the software stack. Moreover, anytime the hardware changes, the machine must be power cycled to re-enumerate the device. CoDes provides this project with the sandbox environment needed to build out the fundamental pieces and then eventually move it into shared environments once it reaches the desired level of maturity and reliability. Also, the CoDes machine hosting this project power cycles in just a few seconds.
CoDes progress
This year’s progress breaks down into three parts: hardware, organization, and projects.
Hardware
The hardware we added to CoDes covers a broad spectrum of types, sizes, and capabilities. We added multiple machines for running the CoDes software ecosystem and interfacing with wired and/or wirelessly connected devices. These machines were a combination of both pre-built and custom-built setups, with the latter designed specifically for improving hardware development turnaround times.
We also added several FPGAs to CoDes, including boards based on AMD/Xilinx Ultrascale+ (CVP-13), AMD/Xilinx 7 series (Cmod A7, Arty A7, Nexys Video, Genesys 2, Alinx Artix-7 PCIe), Lattice ECP5 (ECP5 -5G Versa) and Lattice ice40 (icoBOARD). CoDes also has microcontrollers (ESP32s, Arduinos) and single-board computers (Raspberry Pis). Finally, we added components such as sensors, analog/digital circuit elements, actuators, peripherals, memories, and networking resources to support prototyping and experiments with the above hardware.
Organization
CoDes is organized into multiple testbeds, each with its own software ecosystem, machine specification, and available specialized hardware. We currently have three testbeds in CoDes: an AI/ML testbed, a SmartNIC testbed, and a Wireless and IoT testbed.
The AI/ML testbed machine is the largest one in CoDes, both in terms of compute/memory resources and the size of the attached FPGA. This allows us to deploy computationally intensive machine learning tasks on the CPU cores and experiment with frameworks that offload these tasks to FPGAs. The SmartNIC testbed contains multiple high-end FPGAs (with up to 400G bandwidth per device) plugged into a machine that provides ultra-fast boot times. The former allows us to experiment with multiple different datacenter communication configurations (e.g., FPGA-FPGA direct connectivity, FPGA as bump-in-the-wire), while the latter reduces the turnaround time for research targeting the CPU-FPGA PCIe interface.
The Wireless and IoT testbed is based on a pre-built machine that interfaces a large number of smaller devices over a wired and/or local wireless network connection. This allows us to experiment with use cases where resource-constrained devices are deployed in the field and used for tasks such as directly interfacing sensors/actuators and processing data at the source.
Projects
One of the most prominent projects incubated in CoDes was the Dynamic Infrastructure Services Layer (DISL) project, which we demoed in October 2023. The demo showcased a novel method of enabling application developers to deploy their workloads on FPGAs within seconds (as opposed to days or months). As part of the demo, we also highlighted the ability to wirelessly reconfigure the FPGA and remotely deploy and manage machine learning models at the edge. DISL capabilities have attracted customer interest in various areas, from medical devices to particle physics, and even to Kubernetes in space. CoDes has also enabled a number of projects around the application of AI/ML to systems research in the past year. This includes automatic code annotation for hardware generation, compiler tuning, and hardware place-and-route tuning.
Future directions—plus robots
Looking ahead, we are aiming to hit many key milestones in 2024. This includes more testbeds (including a RISC-V FPGA softcore testbed and a streaming AI testbed), more FPGAs (including a server with 4x Intel Stratix 10 FPGA-based boards), more RISC-V servers (we have already started adding Milk-V machines to the lab), more connectivity options (multi-hundred inter-node and intra-node Gbps links), more demos (including using DISL as a platform generator for microcontrollers/single-board-computers/CPUs, compiler tuning, DISL AI/ML offloading, DISL VirtIO and networking, and Fedora on RISC-V ), and many more components.
Oh, and robots—we’ll be adding robots to the CoDes research lab!
CoDes is currently only accessible to projects that are part of the Red Hat Collaboratory at Boston University. However, our aim is to expand CoDes accessibility in the near future. Check out CoDes on GitHub to see projects as they are added, and contact Heidi Dempsey or Ahmed Sanaullah if you’re interested in collaborating with us!
About the author
Ahmed Sanaullah is a Senior Data Scientist at Red Hat. His current focus is building open source tooling for FPGAs that enables developers to easily and efficiently create custom hardware solutions, regardless of prior hardware development expertise. He received his PhD in Computer Engineering from Boston University in 2019, winning the Outstanding CE Dissertation Award.
