Red Hat associates from locations across North and South America collaborate with primarily North and South American-based researchers on many research projects. In addition to long-standing formal arrangements with Boston University, the Mass Open Cloud Alliance, and the University of Massachusetts, we support student and faculty research and open source development work for undergraduates, Master’s, and PhD students. We also teach classes, mentor students, deliver technology workshops, and support outreach programs that improve diversity in computer science and engineering.
If you are a student interested in a project opportunity, please contact us. If you are a Red Hatter interested in submitting a project, please copy this template and email your idea to Heidi Dempsey, Research and Innovation Director, North America.
Meetings are currently on pause; see details on prior meetings below.
Catch recordings of meetings on the Americas Research Interest Group YouTube Channel.
Access meeting notes here.
Machine Learning Tuning of Kernel Policies Towards Energy Efficiency in Diverse Hardware and Software [Americas Research Interest Group Meeting, July 2023]
Abstract: As global data center energy use continues to rise, a core goal operating systems (OS), which is to enable higher efficiency and get work done while consuming fewer resources is magnified due to increasingly constrained energy budgets. Our work focuses on revealing how three fundamental aspects of an OS: 1) interrupt coalescing, 2) processor sleep state, and 3) dynamic voltage frequency scaling, can impact the performance and energy efficiency of network processing through a diverse hardware and software experimental study. We built upon our previous work which establishes how a state-of-the-art machine learning technique, Bayesian optimization, can be used by an operator to dynamically adjust service-level agreement (SLA) and energy goals while supporting a real world in-memory key-value store workload. This was made possible by the insight that being able to externally control interrupt coalescing helps stabilize application latency periods such that it becomes easier to control performance-energy trade-offs and magnify its benefits with processor frequency scaling while utilizing specialized sleep states.
We theorize that this insight is generally applicable across diverse sets of hardware and SLA-driven network applications. Almost all modern CPU architectures expose a degree of dynamic voltage frequency scaling such that it can trade off instruction execution speed with a reduction in energy use. Furthermore, modern NICs and their device drivers are typically developed to be configured via the ethtool networking utility, which often provide interfaces that enable user defined interrupt coalescing rates. Adjusting these rates can improve overall software stack efficiency as system overheads such as interrupt processing, OS book-keeping, and cache misses are amortized or eliminated by the batched handling of packets. To demonstrate this theory, we undertook a diverse experimental study to demonstrate how Bayesian optimization can be applied across various CPUs and NICs while running a diverse set of SLA-driven network applications. We utilize experimental hardware from both the Massachusetts Open Cloud (MOC) and CloudLab to demonstrate the generality and useability of Bayesian optimization as a mechanism to dynamically target SLA and energy goals.
Materials from Meeting
Join us for the next Red Hat Research Americas Research Interest Group Meeting on July 18, 2023 at 3PM EDT. The meeting is open to Red Hatters and our research partners.
Meeting Agenda
Machine Learning Tuning of Kernel Policies Towards Energy Efficiency in Diverse Hardware and Software
Han Dong, Boston University
North America RIG Meetings Archive
News
Meet the new 2023 Red Hat Collaboratory Research Incubation Award Recipients
We are pleased to announce the recipients of the 2023 Red Hat Collaboratory Research Incubation Awards. The awards are part of a joint effort between Red Hat and Boston University (BU) aimed at recognizing, supporting and funding research in cloud computing, and to advance the state of the art in open source technologies such as distributed systems, security, operating systems, and networking.
Call for Project Ideas (North America Research Interest Group)
Are you a Red Hatter with a project idea that needs a little help getting started? Could your project benefit from some intern power? Red Hat Research (RHR) may be able to help by providing intern and research support to get your idea off the ground or to the next...
Affiliated Universities
University of Massachusetts, Lowell
University of Massachusetts, Amherst
Northeastern University
People
Events
Related Projects
Title Summary Research Area Universities hf:tax:research_area Curator Operator Curator is an air-gapped infrastructure consumption analysis tool for the Red Hat OpenShift Container Platform. The curator retrieves infrastructure … Boston University Improving Cyber Security Operations using Knowledge Graphs AbstractThe objective of this project is to improve the workflow and performance of security operation centers, including automating several of … AI-ML, Cloud-DS, Testing and Ops Boston University ai-ml cloud-ds testing-and-ops Minimal Mobile Systems via Cloud-based Adaptive Task Processing The high cost of robots today has hindered their widespread use. Specifically, a limiting factor involves extensive hardware and software … AI-ML, Cloud-DS Boston University ai-ml cloud-ds Co-Ops: Collaborative Open Source and Privacy-Preserving Training for Learning to Drive Note: This project is a continuation of OSMOSIS: Open-Source Multi-Organizational Collaborative Training for Societal-Scale AI Systems. AbstractCurrent development of autonomous … AI-ML, Cloud-DS Boston University ai-ml cloud-ds CoDes : A co-design research lab to advance specialized hardware projects CoDes research lab provides the infrastructure and engineering foundation needed to support co-design based specialized hardware research. The lab is currently located at Boston University, as part of the Red Hat – Boston University collaboratory. AI-ML, Cloud-DS, Hardware and the OS Boston University ai-ml cloud-ds hardware-and-the-os Prototyping a Distributed, Asynchronous Workflow for Iterative Near-Term Ecological Forecasting AbstractThe ongoing data revolution has begun to fuel the growth of near-term iterative ecological forecasts: continually-updated predictions about the future … Boston University FHELib: Fully Homomorphic Encryption Hardware Library for Privacy-preserving Computing Note: Please visit the Privacy-Preserving Cloud Computing using Homomorphic Encryption project page for information on a related project. In today’s … Cloud-DS, Hardware and the OS, Security, Privacy, Cryptography Boston University cloud-ds hardware-and-the-os security-privacy-cryptography SECURE-ED: Open-Source Infrastructure for Student Learning Disability Identification and Treatment The project aims to develop an infrastructure that would enable users to input data about an individual student and receive … Boston University Relational Memory Controller Note: See the Near-Data Data Transformation project page for information about the work that led to this project. Abstract: Data movement … Boston University Learned Cost-Models for Robust Tuning Note: Please see the Robust Data Systems Tuning project page for earlier results associated with this research. Abstract: Data systems’ performance is … Boston University Open-Source Toolchain Optimization for FPGA CAD Additional details to be added soon! Project Poster Link to full size project poster Hardware and the OS UMass Lowell hardware-and-the-os Characterizing Microservice Architectures Microservice architectures are the default method for building distributed applications in industry. Though the basic tenants of this architectural style … Cloud-DS, Testing and Ops Emory University, Tufts University cloud-ds testing-and-ops Understanding accuracy decay in online image retrieval systems within the context of open-set classification and unsupervised clustering Image retrieval systems are extremely useful to political scientists and human rights advocates attempting to understand the scope and spread of disinformation in massive datasets. However, in standard image retrieval tasks the corpus of images is unchanging as time moves forward. When considering online disinformation this is clearly not the case. Image retrieval in an online system can essentially be modeled as an open-set problem, where there is no guarantee that the classes of images seen before will have any correspondence to the classes of images seen at present or in the future. AI-ML, Cloud-DS University of Notre Dame ai-ml cloud-ds Automated detection of memory safety vulnerabilities in Rust In comparison to C, the Rust language provides significant memory safety guarantees through its concept of lifetimes and its borrow-checker. … Testing and Ops Columbia University testing-and-ops Tuning the Linux kernel The Linux kernel is a complicated piece of software with multiple components interacting with each other in complex ways. The … AI-ML, Hardware and the OS Boston University ai-ml hardware-and-the-os Disinformation Detection at Scale The increased prevalence of fake and manipulated visual media on the Internet has led to social and technical dilemmas in … AI-ML, Security, Privacy, Cryptography UNICAMP - Universidade Estadual de Campinas, University of Notre Dame ai-ml security-privacy-cryptography AI for Cloud Ops This project aims to address this gap in effective cloud management and operations with a concerted, systematic approach to building and integrating AI-driven software analytics into production systems. We aim to provide a rich selection of heavily-automated “ops” functionality as well as intuitive, easily-accessible analytics to users, developers, and administrators AI-ML, Cloud-DS, Hardware and the OS Boston University ai-ml cloud-ds hardware-and-the-os Creating a global open research platform to better understand social sustainability using data from a real-life smart village A BU team is working with SmartaByar and the Red Hat Social Innovation Program in order to create a global … AI-ML, Cloud-DS, Security, Privacy, Cryptography Boston University ai-ml cloud-ds security-privacy-cryptography DISL: A Dynamic Infrastructure Services Layer for Reconfigurable Hardware Open programmable hardware offers tremendous opportunities for increased innovation, lower cost, greater flexibility, and customization in systems we can now … Cloud-DS, Hardware and the OS Boston University cloud-ds hardware-and-the-os Practical Programming of FPGAs with Open Source Tools This project has evolved from the Practical programming of FPGAs in the data center and on the edge project. Please see … Cloud-DS, Hardware and the OS Boston University cloud-ds hardware-and-the-os Near-Data Data Transformation BU faculty members Manos Athanassoulis and Renato Mancuso will work with Red Hat researchers Uli Drepper and Ahmed Sanaullah to create a hardware-software co-design paradigm for data systems that implements near-memory processing. Cloud-DS, Hardware and the OS Boston University cloud-ds hardware-and-the-os Towards high performance and energy efficiency in open-source stream processing. BU faculty members Vasiliki Kalavari and Jonathan Appavoo will work with Red Hat researcher Sanjay Arora to create an open-source … Hardware and the OS Boston University hardware-and-the-os OSMOSIS: Open-Source Multi-Organizational Collaborative Training for Societal-Scale AI Systems The goal of our project is to develop a novel framework and cloud-based implementation for facilitating collaboration among highly heterogeneous research, development, and educational settings. AI-ML, Cloud-DS Boston University ai-ml cloud-ds Privacy-Preserving Cloud Computing using Homomorphic Encryption Note: Please visit the FHELib: Fully Homomorphic Encryption Hardware Library for Privacy-preserving Computing project page for information on a related … Cloud-DS, Hardware and the OS, Security, Privacy, Cryptography Boston University cloud-ds hardware-and-the-os security-privacy-cryptography Serverless Streaming Graph Analytics In this project, we will focus on graph streams that can be used to model distributed systems, where workers are represented as nodes connected with edges that denote communication or dependencies. Cloud-DS, Testing and Ops Boston University cloud-ds testing-and-ops Enabling Intelligent In-Network Computing for Cloud Systems With the network infrastructure becoming highly programmable, it is time to rethink the role of networks in the cloud computing … Cloud-DS, Testing and Ops Boston University cloud-ds testing-and-ops Linux Computational Caching In this speculative work we are attempting to explore a biologically motivated conjecture on how memory of past computing can be stored and recalled to automatically improve a system’s behavior. AI-ML, Cloud-DS, Hardware and the OS Boston University ai-ml cloud-ds hardware-and-the-os The Open Education Project (OPE) In this project we are developing an exemplar set of materials for an introductory computers systems class that exploits, Jupyter, Jupyter Books, OpenShift and the the Mass Open Cloud to develop and deliver a unique educational experience for learning about how computer systems work. AI-ML, Cloud-DS Boston University ai-ml cloud-ds Symbiotes: A New step in Linux’s Evolution This work explores how a new kind of software entity, a symbiotie, might bridge this gap. By adding the ability for application software to shed the boundary that separates it from the OS kernel it is free to integrate, modify and evolve in to a hybrid that is both application and OS. Hardware and the OS, Security, Privacy, Cryptography Boston University hardware-and-the-os security-privacy-cryptography Intelligent Data Synchronization for Hybrid Clouds The goal of this project is to design configurable synchronization solutions on a common platform for a wide range of edge computing scenarios relevant to Red Hat. These solutions will be thoroughly validated on a state-of-the-art testbed capable of emulating realistic environments (e.g., smart cities). AI-ML, Cloud-DS, Testing and Ops Boston University ai-ml cloud-ds testing-and-ops Secure cross-site analytics on OpenShift logs The project aims to explore whether cryptographically secure Multi-Party Computation, or MPC for short, can be used to perform secure cross-site analytics on OpenShift logs with minimum client participation. Cloud-DS, Security, Privacy, Cryptography, Testing and Ops Boston University cloud-ds security-privacy-cryptography testing-and-ops Robust Data Systems Tuning Note: Please see the Learned Cost-Models for Robust Tuning project page for research that has grown from this project. See … AI-ML, Hardware and the OS Boston University ai-ml hardware-and-the-os Robust LSM-Trees Under Workload Uncertainty We introduce a new robust tuning paradigm to aid in the design of data systems with uncertain assumptions by modeling the behavior of the system and then utilizing these models in conjunction with techniques in robust optimization. Our approach is demonstrated through tuning a popular log-structured merge-tree based storage engine, RocksDB Hardware and the OS Boston University hardware-and-the-os Does efficient, private, agnostic learning imply efficient, agnostic online learning? Users of online services today must trust platforms with their personal data. Platforms can choose to enable privacy by default … Boston University Are Adversarial Attacks a Viable Solution to Individual Privacy? Users of online services today must trust platforms with their personal data. Platforms can choose to enable privacy by default … Security, Privacy, Cryptography Boston University security-privacy-cryptography Hybrid Cloud Caching A fundamental goal of the Hybrid Cloud Cache project is to allow simplified integration into existing data lakes, to enable caching to be transparently introduced into hybrid cloud computation, to support efficient caching of objects widely shared across clusters deployed by different organizations, and to avoid the complexity of managing a separate caching service on top of the data lake Boston University, Northeastern University Volume Storage Over Object Storage This project creates a hybrid storage system composed of a high-speed local device (e.g. Optane) to store short term data, along with a write-once object store (e.g, Ceph RGW) to store data blocks permanently. Cloud-DS Boston University, Northeastern University cloud-ds Elastic Secure Infrastructure This project encompasses work in several areas to design, build and evaluate secure bare-metal elastic infrastructure for data centers. Cloud-DS, Security, Privacy, Cryptography, Testing and Ops Boston University cloud-ds security-privacy-cryptography testing-and-ops Open Cloud Testbed The Open Cloud Testbed project will build and support a testbed for research and experimentation into new cloud platforms – the underlying software which provides cloud services to applications. Testbeds such as OCT are critical for enabling research into new cloud technologies – research that requires experiments which potentially change the operation of the cloud itself. AI-ML, Cloud-DS, Hardware and the OS, Security, Privacy, Cryptography, Testing and Ops Boston University, Northeastern University, UMass Amherst ai-ml cloud-ds hardware-and-the-os security-privacy-cryptography testing-and-ops Kernel Techniques to Optimize Memory Bandwidth with Predictable Latency Recent processors have started introducing the first mechanism to monitor and control memory bandwidth. Can we use these mechanisms to enable machines to be fully used while ensuring that primary workloads have deterministic performance? This project presents early results from using Intel’s Resource Director Technology and some insight into this new hardware support. The project also examines an algorithm using these tools to provide deterministic performance on different workloads. Hardware and the OS Boston University hardware-and-the-os Unikernel Linux This project aims to turn the Linux kernel into a unikernel with the following characteristics: 1) are easily compiled for any application, 2) use battle-tested, production Linux and glibc code, 3) allow the entire upstream Linux developer community to maintain and develop the code, and 4) provide applications normally running vanilla Linux to benefit from unikernel performance and security advantages. Hardware and the OS Boston University hardware-and-the-os Fuzzing Device Emulation in QEMU Hypervisors—the software that allows a computer to simulate multiple virtual computers—form the backbone of cloud computing. Because they are both ubiquitous and essential, they are security-critical applications that make attractive targets for potential attackers. Hardware and the OS, Security, Privacy, Cryptography, Testing and Ops Boston University hardware-and-the-os security-privacy-cryptography testing-and-ops Automatic Configuration of Complex Hardware In this project, we pursue three goals towards this understanding: 1) identify, via a set of microbenchmarks, application characteristics that will illuminate mappings between hardware register values and their corresponding microbenchmark performance impact, 2) use these mappings to frame NIC configuration as a set of learning problems such that an automated system can recommend hardware settings corresponding to each network application, and 3) introduce either new dynamic or application instrumented policy into the device driver in order to better attune dynamic hardware configuration to application runtime behavior. Hardware and the OS Boston University hardware-and-the-os Quest-V, a Partitioning Hypervisor for Latency-Sensitive Workloads Quest-V is a separation kernel that partitions services of different criticality levels across separate virtual machines, or sandboxes. Each sandbox encapsulates a subset of machine physical resources that it manages without requiring intervention from a hypervisor. In Quest-V, a hypervisor is only needed to bootstrap the system, recover from certain faults, and establish communication channels between sandboxes. Hardware and the OS Boston University hardware-and-the-os Performance Management for Serverless Computing Serverless computing provides developers the freedom to build and deploy applications without worrying about infrastructure. Resources (memory, cpu, location) specified … Cloud-DS Boston University cloud-ds