Members of the Red Hat Research team and several of our collaborators are presenting at DevConf.US at Boston University, held August 14-16, 2024. This is the sixth annual convening of this Red Hat-sponsored technology conference for community projects and professional contributors to free and open source technologies. The conference is free, and students and recent graduates are encouraged to attend (attendee coaching is available for conference newbies).
Look for presentations, demos, workshops, and meetups in AI and data science; cloud, hybrid cloud, and hyperscale infrastructure; future tech and open research; and PhD highlights. Click on a talk title to see it on the DevConf.US schedule. Keynotes from the conference will be livestreamed, and all other talks will be recorded to be shared online at a later date.
Artificial intelligence and data science
- AI Lab Recipes: Cooking up AI applications on your laptop
Speakers: Sally O’Malley and Michael Clifford
Session recording
What if we told you there’s a GitHub repository that makes building, running, and developing AI powered applications from your laptop as easy as making toast? In this session, we’ll explore ai-lab-recipes. This repository is a collaboration of data scientists and application developers that brings together best practices from both worlds. The result is a set of containerized AI-powered applications and tools that are fun to build, easy to run, and convenient to fork and make your own! We’ll show you how easy it is to spin up a local code generation assistant with only two simple commands.
- Building trust with LLMs
Speakers: Surya Pathak and Hema Veeradhi
Session recording
Have you ever questioned the reliability of Large Language Models (LLMs)? This talk is designed to be your guide through the intricate process of LLM evaluation, equipping you with practical insights to navigate the complexities of implementing LLMs in real-world applications. We will go over the fundamentals of LLM evaluation, then delve into more specialized techniques such as model based evaluation using LangChain criteria metrics. We will also cover human based evaluation and different evaluation benchmarks. Using a text generation demo application, we’ll compare the different evaluation techniques, highlighting their pros and cons. Throughout the session, we will address common challenges that you may face when assessing the quality of your LLMs and how to overcome them.
- Detoxification of LLMs using TrustyAI Detoxify and HuggingFace transformer reinforcement library
Speaker: Christina Xu
Session recording
Detoxification of large language models is challenging because it requires the curation of high quality, annotated data that needs to align with human values. The standard protocol for LLM detoxification is to perform prompt tuning and then supervised fine tuning on a pretrained model. While HuggingFace’s Supervised Finetuning Trainer (SFT) streamlines this protocol, it still requires high quality, human aligned training data which is expensive to curate. TrustyAI Detoxify is an open source library for scoring and rephrasing toxic content generated by LLMs. During this talk, Christina will show how TrustyAI Detoxify can be leveraged to rephrase toxic content for supervised fine-tuning.
- LLMs 101: Introductory workshop
Speakers: Surya Pathak, Hema Veeradhi, and Aakanksha Duggal
Are you curious to learn about Large Language Models (LLMs), but unsure how and where to begin? During this workshop, we will delve into the fundamentals of LLMs and guide you in selecting the appropriate open source models for your requirements. Attendees will gain practical knowledge and skills to effectively harness the capabilities of LLMs in real-world applications. They will understand the challenges associated with managing computational resources and learn how to overcome them. By the end of the workshop, participants will be equipped with the tools to set up and deploy LLMs, evaluate model performance, and implement them in various natural language processing tasks.
- Self-hosted LLMs: a practical guide
Speakers: Hema Veeradhi and Aakanksha Duggal
Session recording
Have you ever considered deploying your own large language model (LLM), but the seemingly complex process held you back from exploring this possibility? This talk aims to provide a comprehensive introductory guide, enabling you to embark on your LLM journey by effectively hosting your own models on your laptops using open source tools and frameworks. We will discuss the process of selecting appropriate open source LLM models from HuggingFace, containerizing the models with Podman, and creating model serving and inference pipelines. By the end of this talk, attendees will possess the necessary skills and knowledge to navigate the exciting path of self-hosting LLMs.
Cloud, hybrid cloud, and hyperscale infrastructure
- Open education in the New England Research Clouds (NERC)
Speaker: Isaiah Stapleton
Session recording
The Open Education Project (OPE) leverages modern open source technologies to create an open environment and platform in which educators can create, publish, and operationalize high-quality open source materials, while students require no more than access to a web browser to access them. In this presentation, we’ll first explore the architecture of OPE, focusing on utilizing the OPE SDK to construct courses with “books” and containers. We then look at how we have hosted large courses on NERC using Red Hat OpenShift AI. We will highlight the successful deployment of OPE on NERC OpenShift, demonstrating how NERC infrastructure supports the flexibility and accessibility of open education.
- Power Efficiency Aware Kubernetes Scheduler
Speaker: Han Dong
Session recording
While there have been a variety of Kubernetes Enhancement Proposals (KEPs) to improve its scheduler plugins for a range of requirements, unfortunately, we find there hasn’t been a focus on sustainability. Towards this, we propose a new scheduler, PEAKS (Power Efficiency Aware Kubernetes Scheduler), that can factor in sustainability goals such as power utilization and carbon footprint. PEAKS leverages the Kepler (Kubernetes-based Efficient Power Level Exporter) project, which uses eBPF to probe system-wide utilization and dynamically expose these metrics via Prometheus. Given these metrics, PEAKS makes dynamic decisions to recommend nodes for pod scheduling, addressing power inefficiencies on underutilized nodes. In this talk, I will present ongoing work from collaborators at Boston University, Red Hat, and IBM in deploying PEAKS in a variety of scenarios ranging from synthetic to more realistic microservices-based deployments.
Future tech and open research
- Open source for open hardware: the RISC-V software ecosystem (meetup)
Speaker: Jeffrey “Jefro” Osier-Mixon
Let’s get together and discuss RISC-V and the universe forming around it. This talk covers advances in the RISC-V software ecosystem. RISC-V is a flexible and fully open hardware architecture, curated by the non-profit RISC-V International foundation (riscv.org) using the best practices of open source. While the open source community has responded with a great deal of distributed effort, there are notable gaps in support, particularly for commercial-grade applications. The industry has responded by forming the RISE Project (riseproject.dev), a collaboration among 20+ organizations to support and advance the software ecosystem by providing engineering and financial resources.
PhD highlights and interns
- Porting and generalizing Dynamic Privilege in Linux
Speaker: Arlo Albelli
Session recording
Recent work in our research group introduced the notion of Dynamic Privilege and the attendant kernel mechanisms to introduce it to the Linux kernel. This permits the exploration of several interesting optimizations and novel approaches to system specialization. The initial implementation was developed in x86_64. In this talk, we will present our work in porting the core primitives for Dynamic Privilege to ARM64 and discuss the details of this approach. Through a comparison of the ARM64 and x86 implementations, we will differentiate the functional goal of Dynamic Privilege from the underlying architectural mechanisms. Finally, we will discuss how our experiences introducing the mechanism to ARM64 inform a natural path towards a RISC-V implementation which we will briefly introduce.
- UKELELE
Speaker: Eric Munson
Session recording
Modern server applications are built around an event loop where they wait for some form of input then perform some computation in response to the type of input received. At the heart of this event loop is a method for the application to communicate interest in certain events to the operating system and for the operating system to return a list of requested events which have occurred to the process when requested. The abstractions built for this event handling each come with a cost to performance. In this talk we will quantify this cost and propose a system, UKELELE, which provides an abstraction for event handling that is easy to use, as quantified by the amount of change required to port to it, and more performant. We will also explore optimizations available by building on UKL to create shorter code paths for event handling and measure each of their impacts on application performance. UKELELE shows an improvement in overall throughput and more a reduction in tail latency when making use of all our available optimizations.
- Keynote, Day 3
Start the day off with a view into real student experiences with open source! Undergraduates from the UMass Lowell SoarCS summer program will present summaries of their projects and answer questions. Selected Red Hat interns will also speak about their ongoing projects and experiences as interns.
