An open, inclusive culture is one of Red Hat Research’s not-so-secret ingredients for forging innovative research collaborations. In this interview, Senior Distinguished Engineer Larry Woodman explains how the DE community and Red Hat Research play complementary roles in advancing cutting-edge open source technology—and how others can get involved.
A researcher and engineer with over 36 years of kernel development experience, Larry has been especially active in unikernel Linux and open source education projects with the Red Hat Collaboratory at Boston University (BU). We interviewed Larry following his participation in the fifth annual Red Hat Distinguished Engineer eXchange (DEX) 2022, held in Raleigh, N.C., October 4-6. He talked about the role of Distinguished Engineers (DEs) in guiding the direction of Red Hat’s technology development, his work as an engineer and researcher, and why mentorship and teaching have become vital parts of his career.
For those who don’t know, what is the Distinguished Engineer program?
Right now, there are over 10,000 engineers at Red Hat, and there are about 71 currently designated as DEs, with about a dozen of us senior DEs. Our role, in addition to doing development, is to help guide the company in terms of technical direction: what products to work on, what not to do, what industry trends are likely to be important in the future. Another part of that is helping more junior engineers grow in their position at Red Hat.
So what do DEs do when they gather for an event like DEX?
The official mission of the DE community is “to foster a community of trusted technical leaders who guide our technology strategy and direction, build consensus across the company, and collaboratively craft the future of Red Hat.” So that’s what we’re all about.
This is the fifth annual event, but obviously the last two, in 2020 and 2021, were virtual. I like the in-person events, where we can meet people, sit together, and collaborate. The water cooler talk is a big part of getting together; I get as much out of the breaks as I do the presentations.
Our keynote speakers for the three days were Chris Wright, Tim Cramer, and Matt Hicks, respectively. Throughout the conference we had conversations with Mike Ferris on corporate strategy and Ashsesh Badani, Joe Fernandes, and Sarwar Raza on the intersection of the engineering and product organizations. These discussions about long-term technology strategy—plus insight into what customers are facing— give the DE community what we need to know to work on the most important challenges.
Those usually happen in the morning, and then the rest of the day is given to breakouts, networking happy hours, and birds-of-a-feather sessions. These are the opportunities to build relationships with new and established DEs and have those free-flowing, unstructured conversations that can lead to new collaborations.
You went to DEX wearing both your DE hat and your Red Hat Research hat. What did that involve?
Broadly speaking, I was educating people about what Red Hat Research does and encouraging both DEs and the engineers they mentor to get engaged with the work we’re doing. My view of Red Hat Research is that we’re trying to figure out what direction we should help guide the company toward where the industry is going to grow—where it’s most lucrative, but also where the technology is most interesting. That aligns pretty directly with the DE mission.
So I spent a lot of time during the unstructured sessions getting together with people to talk about how to get involved with research. (Red Hat Research director) Hugh Brock gave a lightning talk highlighting 17 posters we brought to the event to showcase some of our current research and explain how to get involved, and then I was available to talk to people about the specific projects and answer questions.
How did that go over? Were people interested in those projects, or in doing research in general?
I got a lot of people asking questions about various posters and the associated projects, especially on work related to unikernel Linux. I also talked a lot about the Jupyter Books in open source education project I participate in, headed by Jonathan Appavoo at BU. A lot of people are interested in authoring or co-authoring a textbook using something like that. Several people in the DE community have PhDs, and when you leave the university after getting your doctorate, you leave research behind. So they were interested in what kind of research was happening and how they could participate.
We’re trying to figure out what direction we should help guide the company … where the industry is going to grow.
In general, what interests people the most—PhDs or not—is the chance to experiment. When you’re working in a product development group, trying something and having it not work out is literally considered a failure. Trying something and failing consumes engineering resources that were not budgeted for. Whereas we, in the research organization—we budget for failure, if you will. We don’t go looking for failures, obviously, but we can test out multiple solutions.
Here’s a good example from the unikernel project. One of the issues we ran into was that when you’re in user mode, you have something called dynamic stack extension, which is a very useful thing because it prevents you from allocating all the data structures and memory unless you really need them. But because there’s no user mode in the unikernel, you can attempt dynamic stack extension, and since you haven’t prepared for this, the system would crash.
We envisioned several possible solutions, from the most basic to the really complicated. And rather than just going with one solution, the interns on our team prototyped multiple solutions. None of those solutions really panned out, but we didn’t consider the other tests failures. That’s the learning that’s associated with research. That’s a breath of fresh air when you’re usually working in a development environment.
What about you? What do you personally like about working in the research program?
My participation involves a lot of working with PhD students at BU, helping mentor and guide them with the research that they’re doing. The education element is an essential part of the Red Hat Research charter.
At this point in my engineering career, one of the things that I enjoy most is teaching. I think I’ve been lucky in life and had good opportunities in the work that I do, and I like giving back. It’s rewarding. Being a mentor is the same: it feels like you’re giving back. I want to give my time to teach and mentor individuals who are really interested in moving forward and helping themselves. I think that’s what mentorship is all about—helping somebody help themselves.
Also, there’s no better way to learn something than to teach it. This has nothing to do with my university teaching, but I’m a pilot. I have an airplane, and I’ve taught several relatives how to fly. I’m concerned about their safety—their mothers and fathers will have my head if anything happens to them!—so I become much better at it myself by teaching them. The same is true with teaching computer science and operating systems and so on—it makes me better.
What interests people the most is the chance to experiment.
Also, the students I work with, in my experience, are the cream of the crop. Often at the end of teaching a class some of the really interested students apply for internships, and then from those internshi[s a few will go on to graduate and become full-time employees at Red Hat. When I’ve asked about the people who have been hired, their managers think they’re fantastic. Some of these young former students are likely future DEs. Even if they don’t go on to participate in research at Red Hat—which we hope they will!—they’re part of the talent pipeline that Red Hat Research has created to build a strong engineering community.
Speaking of participating in research at Red Hat—if DEs, their mentees, or other engineers are interested in engaging in research opportunities, where should they start?
There are a lot of ways to get involved—contributing to a current project, teaching a course, mentoring research students and interns, participating in Research Interest Group (RIG) meetings—but they can just reach out to academic@redhat.com and start the conversation. People can also get involved with DE-led projects through DE Passions Working Groups, which are community-led efforts in areas DEs are focusing on, such as accelerating the use of eBPF to replace in-kernel code, hybrid cloud services, governance, outreach, and creating value. Even if research or teaching is not part of your day-to-day work, the time you give to it is advancing your own knowledge and abilities, and it’s helping set the course for what you’ll be working on in the future.
