As the universe of open research clouds keeps expanding, so does the visibility they provide.
In 2014, Orran Krieger, Professor of Electrical and Computer Engineering at Boston University (BU), and Peter Desnoyers, Associate Professor at the Khoury College of Computer Sciences at Northeastern University, launched an initiative called the Mass Open Cloud (MOC). Since then, the MOC has focused on providing researchers with a cloud that enables visibility into the cloud’s underlying infrastructure. Such transparency is essential to better understanding performance and other metrics associated with cloud operations.
Learn more about the history of the MOC in “Massachusetts Open Cloud advances research and innovation,” RHRQ 1:1 (May 2019).
But transparency—often for good reasons— isn’t provided by public cloud providers, which are essentially defined around providing an abstraction at a higher level of the hardware/ software stack. As a result, customers have no visibility into the underlying infrastructure’s design, behavior, or operation. Public clouds also use a variety of proprietary technology and vendor-specific APIs. Therefore, in addition to providing visibility and access to the underlying infrastructure—thereby enabling researchers to investigate infrastructure design choices that would optimally match with new applications— the MOC set out to provide a cloud that would emphasize open standards and interoperability.
The expanded partnership will maintain a deployment that includes everything from event reporting and log management to user onboarding and quota reporting.
The latest news from the MOC is the formation of an expanded partnership. It will maintain an on-premises Kubernetes-based Red Hat® OpenShift® deployment that includes everything from event reporting and log management to user onboarding and quota reporting. It’s essentially a minimum equipment list for operating something like OpenShift in the real world. In addition, there will be other services that run on top of that core infrastructure—for example, Open Data Hub services.
The new alliance of MOC supporters plans to establish a staff that will take responsibility for decisions such as vetting the services that will go into this template and determining that, when a new version of a particular service comes out, it is OK to use. Red Hat intends to fund a project manager, an engineering manager, and four or five developers whose primary job will be effectively curating services to run on top of the Kubernetes infrastructure. Initially, the infrastructure will run supported OpenShift products based on prior donations to BU and Harvard. As participation increases, the infrastructure will expand to include upstream community-supported code as well.
Ongoing activity isn’t limited to the MOC, however. A wide variety of overlapping initiatives are taking place in the Commonwealth of Massachusetts, a state with many academic institutions and a long history of research and commercial activity in computer science and engineering. Most of the hardware associated with these initiatives is housed at the Massachusetts Green High Performance Computing Center (MGHPCC), an intercollegiate high-performance computing facility in Holyoke, MA, completed in 2012. It is a joint venture of BU, Harvard, MIT, Northeastern, and the University of Massachusetts system. In addition to the current MOC-operated OpenShift on bare metal cluster, the original MOC OpenShift on OpenStack cluster in the MGHPCC and the New England Research Cloud (NERC) provide resources for researchers.
NERC is a newer joint project of BU and Harvard. It will provide access to both OpenStack and OpenShift clusters for researchers at those schools. With Red Hat’s assistance, NERC has set up a production OpenStack deployment, which is nearly ready for production. NERC’s next project will be to set up a single large OpenShift cluster running on bare metal. The long-term ambition for BU and Harvard is for NERC to become a shared cloud resource for all Massachusetts universities (of which there are many) and eventually for the broader Northeast US region. Because of the way funding works in a research university context, this will be an incremental process that will roll out access to new institutions and research projects over time.
A final related initiative is Operate First, also physically housed at the MGHPCC as a segment of the MOC. Operate First is a project to extend the open source development model to include operating, testing, and validating code in a production cloud environment. In addition to the MOC, Operate First is closely associated with a variety of overlapping initiatives, including OpenInfra Labs (under the Open Infrastructure Foundation) and the Red Hat Collaboratory at Boston University. OpenInfra Labs also hosts the Telemetry Working Group, one of the working groups included under the Operate First umbrella. The working group’s goal is to improve the observability of distributed systems through the automatic collection and transmission of data about a system.
Learn more about Operate First in “Ops is the new code: Operate First brings open source to operations,” RHRQ 3:4 (February 2022).
As we’ve seen, a great deal of varied activity is taking place in Massachusetts to provide the computing foundations for ongoing research. The expanded MOC partnership is the latest noteworthy news item, but new developments continue to happen on many different fronts.