AI’s impact on datacenter energy use is the subject of debate. The need to run AI workflows more efficiently and cost-effectively is not. A new Red Hat-IBM Research study advocates for a layered approach built on open source solutions.

The rapid uptake of cloud-native architectures, edge computing, and now artificial intelligence has led to remarkable efficiency and innovation in our industry. However, this digital revolution is bringing with it a serious environmental cost. Information and communication technologies are currently responsible for as much as just under 4% of global greenhouse gas emissions, putting our sector on a level with aviation. Since computational demands are increasing all the time, it is no longer sufficient to rely merely on individual green coding suggestions or small improvements in facility management. What we need is a comprehensive and educational approach that links every line of code to its actual physical effect on the planet.

This imperative led to my collaboration with colleagues at IBM Research—Umamaheswari Devi, Krishnasuri Narayanam, Asser N. Tantawi, and Chandra Narayanaswami—on a comprehensive paper titled “Sustainable computing: state-of-the-art and future outlook.” Sustainable computing is particularly appropriate for this type of paper since it covers a broad spectrum of abstractions, ranging from the mechanics of power grids and semiconductor physics to container orchestration and software algorithms. Combining in-depth research with practical experience in cloud-native orchestration, telemetry, and open source governance, we aimed at creating a neutral, vendor-agnostic basis that the whole technology ecosystem can refer to and possibly build on.

The architecture of sustainability

A fundamental assumption behind our work is that sustainable computing should be seen as a series of interconnected layers, beginning with the energy grid and then moving on through datacenter infrastructure, hardware design, software development, and operational management. 

Electrical grid

At the base of this stack is the electrical grid, since the environmental impact of digital activities is determined by the carbon intensity, defined as the amount of carbon dioxide equivalent emitted per kilowatt-hour produced. As renewable energy sources such as solar and wind vary according to the weather and the time of day, the carbon intensity of the grid changes continuously. In order to cope with this variability, operational computing must evolve to become carbon-aware by means of techniques such as temporal shifting, which postpones non-urgent workloads to times when clean energy is available, and spatial shifting, which sends portable containerized applications to areas where the energy mix is cleaner.

Datacenter infrastructure

Further up the stack, the non-IT energy use associated with cooling and power distribution in datacenter facilities is the focus. Although Power Usage Effectiveness figures in the past led to quick improvements in efficiency, the efficiency of large hyperscale facilities has generally stabilised in recent years. Incremental reductions can still be achieved through structural measures such as direct-to-chip liquid cooling, hot and cold aisle containment, and automated thermal control systems. Nevertheless, operational energy makes up only one part of the issue. 

Hardware design

IT hardware also has considerable embodied emissions through the Scope 3 greenhouse gases resulting from the extraction of raw materials, the manufacturing of silicon wafers, and the assembly process. Hence, in order to improve the sustainability of hardware, we must look at energy proportionality, a measure of how closely a device’s power consumption matches its active workload. Although modern processors have improved their peak-to-idle power ratios, storage arrays and networking equipment still consume a large amount of power even when not in use.

Sustainable software engineering

In terms of software, the application logic has a direct effect on the use of hardware resources. Sustainable software engineering involves paying attention to algorithmic efficiency, using efficient data structures, and making thoughtful choices about the programming language. Languages that are compiled, such as C and Rust, carry out the same operations using only a small fraction of the CPU instruction cycles and memory overhead that are needed by interpreted languages such as Python. By reducing algorithmic complexity and cutting down on code bloat, the amount of energy consumed at the level of the physical processor is directly reduced. 

System operations

In practice, achieving digital sustainability means viewing the operation of a system as a single equation in which carbon emissions are equal to the product of the carbon intensity of the electricity grid, the amount of power consumed, and the length of time the job runs. Operators can adjust the processor’s clock speed during periods when the grid has a high carbon intensity by means of dynamic voltage and frequency scaling, thus balancing the need for execution latency with environmental impact.

Implementing sustainable practices

To show how these theoretical concepts function in the real world, our study includes implementations throughout the industry. Google DeepMind used predictive machine learning models on the sensor data in datacenters  and thus reduced its cooling energy consumption by 40%. IBM’s Hursley Data Center achieved maximum host utilization by means of automated resource rebalancing, all while staying within its service-level agreements. This in turn avoided both unnecessary capital expenditure and excessive power usage. 

Automated spatial shifting … obtained reductions in both carbon emissions and operating costs in the double digits.

In cloud-native environments, specialized orchestration tools such as the Power Efficiency Aware Kubernetes Scheduler (PEAKS) use machine learning models that analyze the power-versus-utilization curves of the nodes in order to position container pods at locations where the hardware is working at peak energy efficiency. Moreover, European cloud providers have successfully put into practice automated spatial shifting so as to redirect non-critical workloads across their regional datacenters according to the real-time intensity of the grid, obtaining reductions in both carbon emissions and operating costs in the double digits.

When assessing these measures, it is important for organizations to carry out accurate carbon accounting using established frameworks such as the Greenhouse Gas Protocol. They have to understand the difference between direct Scope 1 emissions; Scope 2 emissions, which result from the purchase of electricity; and indirect Scope 3 emissions throughout the value chain. Importantly, empirical research shows that depending exclusively on market-based accounting tools, such as unbundled renewable energy certificates, can hide the true impact on the grid. For example, the figures for carbon savings can be as much as 55% higher than the location-based estimates that take into account the actual physical flow of electrons. 

In the future, new approaches will continue to alter this situation. The latest technology advances in sustainable computing include low-power neuromorphic architectures that mimic event-driven spiking neural networks, TinyML models that operate on microcontrollers and require very little memory, confidential federated learning, and Small Modular Reactors that supply zero-carbon baseload power to datacenters.

How do we enable adoption of sustainable computing?

The technology job market is being fundamentally changed by this evolution, with traditional positions being transformed and new areas of specialization emerging. Software developers are becoming green software engineers and are now carrying out energy-consumption profiling in continuous integration pipelines. Infrastructure engineers are moving on to become sustainable cloud architects capable of integrating liquid cooling and carrying out carbon-aware workload distribution. Site Reliability Engineers are now setting up real-time observability dashboards, and sustainability auditors are ensuring that regulatory compliance is maintained throughout the digital supply chains.

The cumulative energy consumed by inference can exceed training energy over a model’s lifetime because of the scale and frequency of user requests, and is therefore the main problem to focus on.

In the business world, especially in the context of the rapid development of artificial intelligence, awareness and application of these principles are urgent. Although model training is highly energy intensive, the cumulative energy consumed by inference can exceed training energy over a model’s lifetime because of the scale and frequency of user requests, and is therefore the main problem to focus on. Model compression techniques such as pruning and quantization, together with cloud-native technologies enabling smart workload placement, have an important role to play in AI inference optimization. 

Open source plays the key role in enabling these solutions to be adopted on a global scale. For example, Kepler, an open source tool co-founded by Red Hat and contributed to the Cloud Native Computing Foundation, makes use of extended Berkeley Packet Filter probes within the Linux kernel to measure the power consumption at a micro level in container pods. By making real-time energy data available to everyone, open source collaboration ensures that sustainable computing becomes a transparent industry standard rather than something that remains the exclusive possession of companies with proprietary systems.