Three principal software engineers and sought-after research collaborators share their insights on this critical EU innovation incubator.
In February 2021, the European Union launched Horizon Europe, the next phase in its flagship Framework Programme for Research and Innovation. Horizon Europe, which will fund research from 2021 to 2027, was created to drive innovation, research, and development initiatives, emphasizing EU social, economic, and environmental challenges.
The program has been a significant source of support for researchers in open source technology.
As participants in this and other funding initiatives, we want to share with you why this program is worth learning more about—no matter where you are in the world—and how funded projects grow and, ideally, bear fruit. Our goal is to motivate you to explore funding possibilities for your own open source research project. We are Red Hatters, but we are also active members of the broader open source community, so the more people who use this information, the better.
Why Horizon Europe?
Horizon Europe’s emphasis on open science and open source technology makes it an attractive funding opportunity for open source software engineers. The program evolved from Horizon 2020, which provided financial support for research projects that promoted industrial competitiveness, advanced scientific excellence, or solved social challenges. The BigDataStack project, for example, to which both Luis and Josh contributed, was funded by Horizon 2020 to deliver a complete collection of open and interoperable technology building blocks for a variety of big data stakeholders.
Horizon Europe refines Horizon 2020 goals and increases the available funds to €95.5 billion—nearly a sixty percent increase from the previous research program, if all funds are used. The new funding pillars are Excellent Science, Global Challenges and European Industrial Competitiveness, and Innovative Europe. This change in focus reflects the European Commission’s interest in strengthening international cooperation and emphasizing breakthrough innovations—which makes the program a good fit for the collaborative work done in open source research communities. A dedicated European Innovation Council will support and fund those initiatives of a more disruptive nature that might be too risky for private investors.
The focus on innovation and cooperation makes the program a good fit for the collaborative work done in open source research communities.
The European Commission has also established a robust open science policy. Open science practices are part of the award criteria during the application process, and mandatory open access to publications is enforced and applied throughout the whole program. Participants are also encouraged to use the European Open Science Cloud where applicable.
The European Commission has also committed to increasing the use of open source technology by making it a key element for its digital strategy, reinforcing its finding that open source software and hardware have shown significant economic benefits for the European Union. This explicit change in strategy is recognition of the power of open source, pushing its principles and practices into broader acceptance.
The value of collaboration
Our previous work on Horizon projects has demonstrated the value of collaborating for the businesses that partner together and the universities they partner with. When collaborating with consortium partners, we leverage our open source approach to working by promoting the sharing and reuse of software assets, knowledge, and technical skills. This way of working acts as a catalyst for innovation because it fosters cross-pollination between the ideas of the multiple parties involved. Bringing new people together allows us to:
- combine unlike ideas and methods
- find new use cases for products that are already developed
- build a network of researchers and company partners
- create actual innovative software products in an agile way
- improve diversity in the team
—among many other benefits.
In our current work, for instance, we apply our expertise in integration, automation, and working in the open to extend the lower layers of infrastructure to match the needs of the high-level systems deployed on top. Our understanding of Kubernetes and extensive experience with Red Hat® OpenShift® are also valuable strengths we bring to a project, especially since modern cloud-based projects are almost always based on Kubernetes.
This collaborative approach has also allowed us to connect with an ecosystem of organizations with similar goals. This includes academic partners such as the National Technical University of Athens (Greece), Newcastle University (UK), the Polytechnic University of Madrid (Spain), and the University of Piraeus (Greece), and consulting and service companies like Atos, in addition to multiple technology companies.
It is true that working across institutions and geographic divides creates a challenge in terms of communicating the goals and desired outcome of the projects effectively, but overall there are many more benefits for all the academic institutions and companies involved.
What makes for a good research project?
From an EU perspective, a good research project is one that promotes excellent science that measurably advances the state of the art, makes a measurable impact on challenges that affect people worldwide, or produces innovations that will help the European Union act as a leader in emerging technologies. (For more information on how projects are evaluated, see the section What does a proposal require?) But how do we, as engineers, determine what makes a research project a good fit for us?
We look for projects that fit the directions we want to take in our own work. For example, projects with one or more of the following elements are a good fit for us:
- Work on managing workloads on the edge
- Work that creates new open source projects
- Work that improves existing open source projects
A good project is one that helps us understand potential new use cases for the ecosystem of open source technologies we contribute to. Gaining a better understanding of state-of-the art research allows us to identify new requirements to enhance and optimize the technologies we work on. For example, the BigDataStack project mentioned earlier featured major performance improvements to the kuryr project, and it contributed the xskipper project to the open source community.
Read “BigDataStack delivers with contributions from industry and university partners” in RHRQ 3:2 (Aug 2021).
In 2021, we and our colleagues at Red Hat contributed to the submission of twelve Horizon Europe proposals, most of them at the invitation and encouragement of other academic or industry participants. This represents a huge milestone in our contributions.
Many of these proposals targeted the funding opportunity “Future European platforms for the edge: Meta operating systems.” The topics for that call perfectly align with our current work related to cloud computing technologies and their applications to edge, automation, and management platforms. That, together with the EU push for open source and Red Hatters’ previous good work on innovative open source contributions, led to the record number of proposals in which Red Hat engineers play an essential role in building out infrastructure layers and automation. For example, our past work as engineers on Orbit, Superfluidity, and BigDataStack led to renewed invitations to participate in follow-up projects.
What does a proposal require?
Many good proposals don’t get funded, simply because of the funding granter’s fixed budget. To be seriously considered for funding, a proposal should contain all of the following:
- A demonstration of how the proposal goes beyond the state of the art, with a statement of potential for clear outcomes and impacts, including a way to measure outcome and impact
- A gender equity plan that provides a clear roadmap to identify and assess any gender bias and strategies to correct it
- A high-quality and efficient plan of work, including clearly defined roles and tasks for each participant, based on their expertise
- A commitment to openness and contributing to open source communities (working in upstream communities is not required, but it will hugely increase the impact of the project)
- A collaboration plan indicating consortium members (depending on the call for proposals, requirements may include small or large companies, academic institutions, or research institutes)
- A plan for exploitation, dissemination, and standardization, including blog posts, webinars, conference talks, and other means of transmission
An EC webinar is available to view with more detailed explanations of these criteria, along with tools to help develop and document them.
What happens when a proposal is accepted?
The follow-up process has a number of milestones to make sure the project continues in accordance with the work promised in the proposal. We submit reports and create deliverables, such as reports or software assets. The European Commission, in turn, provides audits and revisions as needed to keep the project on track. At the end of the project, the team delivers a final report that includes a technical report with the description of the project’s working packages, the results, conclusions, and the socioeconomic impact. The report must also include a financial report and a dissemination and exploitation plan.
The dissemination and exploitation plan is one of the most important results of a project, from the Commission’s perspective. Without dissemination, the project will have no impact! Work on this element of the project should start at the very beginning, keeping in mind that all of the parties involved in the project’s consortium are responsible for disseminating results.
A critical factor for increasing adoption—and therefore impact—is to make the software components reusable. Ideally, these components will be used not only by consortium partners but also by a broader community, such as other projects, universities, or other EU companies. An open source model is one of the best ways to encourage reuse. That is especially true when we are not only building a new software component and making the code available, but also developing new research projects and contributing to upstream communities (e.g., the Linux kernel, OpenStack components, Kubernetes). For instance, a modification done at the Linux kernel level is immediately available for millions of users worldwide, which hugely increases the project’s impact.
Research that corresponds to corporate strategic goals gives product engineers an opportunity to work on projects with a larger scope than their daily work allows.
It’s important to us to choose projects that our company will likely support long-term. While it’s possible for a company to take EU grant money, complete a project, and then stop contributing after a project finishes, we want to know that the work will continue. Pushing for upstream acceptance also ensures that the project is aligned with open source community efforts and can be maintained after the project ends. In addition, research that corresponds to corporate strategic goals gives product engineers an opportunity to work on projects with a larger scope than their daily work allows. What we’re doing might not always be on the roadmap right now, but if it fits with the directions our employer is interested in, there’s a better chance that the technologies we’re working on will make a lasting impact on the problems we’re trying to solve.
What do you get out of it?
We each appreciate different things about participating in collaborative projects like these. The opportunity to work on less mature, more speculative technology is a benefit, and so is the opportunity to develop topics we are already working on in a wider scope. For example, in the PHYSICS project that Josh and Luis currently work on, there was an opportunity to introduce cost minimization on public clouds by scheduling workloads to cheaper instances, and in many of the proposals still in evaluation by the Horizon committee there is an opportunity to develop the concept of storage at the edge.
Luis adds, “For me, it is a great opportunity to learn about new technologies that may be mainstream in the next few years. Nowadays, hybrid cloud and containers are the default, but I started working on those in research projects ten years ago.” The same may happen with current work on FaaS for the PHYSICS project; perhaps it will be a new hot topic in a couple of years. It is also a great way to learn about possible needs and use cases, which allows us to evolve an upstream project to account for them.
Josh observes that working on these projects with Red Hat backing has given us a great opportunity to influence the project. Since Red Hat is considered the infrastructure leader in almost all the projects we’ve participated in, we can determine how the infrastructure behaves. “Also,” Josh says, “as someone who does not have a PhD, I enjoy the opportunity to work with academic researchers and help combine academic research with working in industry, something that I missed during my studential era.”
“For me, It is a great way to network with other colleagues and different organizations,” Carlos adds. “I get the opportunity to contribute to state-of-the-art projects and participate in building a community between universities and technology companies.”
To learn more about research projects supported by Red Hat Research and various granting agencies, and to find other ways to get involved, visit research.redhat.com.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779747.