Managing Energy Use in AI Data Centers

Is AI's Growth Sustainable? How to Make Generative Applications Greener. The rise of generative AI tools like ChatGPT and others has been remarkable, but their environmental impact is often overlooked. The data center industry, housing these systems, accounts for up to 3% of global greenhouse gas emissions, with energy consumption doubling every two years. Hyperscale cloud providers like Amazon AWS, Google Cloud, and Microsoft Azure play a significant role in powering these models, leading to major carbon footprints. Understanding the carbon footprint lifecycle of AI models is crucial. Large generative models consume extensive energy during training, and fine-tuning can be a more energy-efficient option. Inference sessions, though less energy-intensive, involve many more sessions, contributing to ongoing energy consumption. Efforts to reduce energy usage include employing less computationally expensive approaches like TinyML and using large models only when significantly valuable. To make AI greener, companies can use existing models from providers instead of creating new ones. Fine-tuning existing models on specific content domains consumes less energy and provides more value. Utilizing energy sources from carbon-friendly regions and monitoring carbon emissions can significantly reduce AI's environmental impact. Reusing models and resources, incorporating AI activity into carbon monitoring, and encouraging green AI practices are crucial steps in promoting sustainability. 1. Prioritize Fine-Tuning: Instead of training new generative models from scratch, focus on fine-tuning existing models for specific content domains. Fine-tuning consumes less energy and provides more value to businesses. 2. Explore Energy-Conserving Methods: Adopt energy-conserving computational approaches like TinyML for processing data. TinyML allows running ML models on low-powered edge devices, significantly reducing energy consumption. 3. Re-use and Open Source Models: Opt for reusing open-source models instead of creating new ones. Recycling tech can lower the carbon impact of AI practices and reduce the need for energy-intensive model development. 4. Monitor Carbon Emissions: Include AI activity in carbon monitoring practices to understand the carbon footprint of AI-related operations. Share footprint numbers to make informed decisions about AI partnerships. 5. Choose Green Energy Sources: Select cloud providers and data centers that prioritize environmentally friendly power resources. Running AI models in regions with carbon-free energy sources can significantly reduce operational emissions. Have you already considered the impact of using compute-heavy applications on our planet? Are you tracking the impact of compute in your sustainability report? #genai #aivalue #sustainableai #sustainability

The Future of Data Centers: Unlocking Efficiency with Direct-to-Chip Liquid Cooling As the demand for high-performance data centers continues to rise, particularly driven by AI workloads, innovative cooling solutions are critical to maintaining both performance and sustainability. Direct-to-chip liquid cooling (DTC) is emerging as a game-changer, enabling data centers to handle higher densities, reduce energy consumption, and lower environmental impact. Why Direct-to-Chip Cooling Matters Traditional air cooling systems struggle to keep pace with the growing heat loads generated by AI accelerators, such as Nvidia's latest GPUs. DTC cooling, by placing liquid-cooled cold plates directly on CPUs and GPUs, offers significantly higher thermal efficiency. This reduces the risk of thermal throttling and enables uniform heat distribution, ensuring optimal performance under extreme computational loads. Energy Efficiency and PUE One of the key advantages of DTC systems is their energy efficiency. By directly removing heat from the source, they reduce the need for air circulation, cutting down on cooling energy consumption by up to 15%. This improvement in energy use lowers the Power Usage Effectiveness (PUE), making DTC an ideal solution for hyperscalers looking to green their operations. Supporting AI Workloads With AI models demanding greater computational power, DTC cooling helps prevent hardware overheating, crucial for maintaining performance. Its ability to handle heat loads makes it ideal for data centers with high-density AI workloads. Overcoming Implementation Challenges Implementing DTC systems requires careful planning, from setting up plumbing for CDUs to ensuring leak-free operations. Regular maintenance, including maintaining good fluid quality, is essential to prevent blockages and maintain system efficiency. Two-Phase Cooling Innovation Two-phase cooling enhances DTC performance by using phase transitions to improve heat transfer. Microchannels in cold plates foster efficient bubble formation, helping manage heat in high-power setups, particularly for AI applications. Sustainability and Future-Proofing DTC systems reduce energy consumption and align with green design goals. Integrating these systems with renewable energy sources and heat recovery methods makes them a key part of sustainable data centers, especially in space-constrained urban areas. Conclusion: Paving the Way for Next-Gen Data Centers Direct-to-chip liquid cooling is set to play a critical role in shaping the future of data centers, particularly those powering AI-driven workloads. By offering a scalable, energy-efficient, and sustainable solution, DTC systems help data centers meet both performance and environmental goals. As this technology continues to evolve, it will ensure that data centers remain both powerful and green. #LiquidCooling #DirectToChip #DataCenters #AICooling #Sustainability #EnergyEfficiency #CoolestDC #GreenDataCenters #AI Image credit: DALL.E

𝐇𝐲𝐛𝐫𝐢𝐝 𝐀𝐂/𝐃𝐂 𝐏𝐨𝐰𝐞𝐫 𝐝𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞: 𝐟𝐨𝐫 𝐡𝐲𝐩𝐞𝐫𝐬𝐜𝐚𝐥𝐞 𝐀𝐈 𝐝𝐚𝐭𝐚 𝐜𝐞𝐧𝐭𝐞𝐫𝐬. System Overview 1. 𝐔𝐭𝐢𝐥𝐢𝐭𝐲 𝐀𝐂 𝐆𝐫𝐢𝐝 Electrical power enters the facility from the utility network at 11–33 kV AC, providing a reliable source for large-scale campuses. 2. 𝐌𝐞𝐝𝐢𝐮𝐦-𝐕𝐨𝐥𝐭𝐚𝐠𝐞 (𝐌𝐕) 𝐒𝐰𝐢𝐭𝐜𝐡𝐠𝐞𝐚𝐫 MV switchgear performs switching, isolation, protection, and fault interruption while distributing power safely throughout the facility. 3. 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 / 𝐒𝐨𝐥𝐢𝐝-𝐒𝐭𝐚𝐭𝐞 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 (𝐒𝐒𝐓) Power is stepped down and converted for internal distribution. Future facilities are expected to increasingly adopt Solid-State Transformers (SSTs) because they offer: Higher efficiency Faster control Bidirectional power flow Native integration with renewable energy and battery storage 4. 𝐇𝐢𝐠𝐡-𝐕𝐨𝐥𝐭𝐚𝐠𝐞 𝐃𝐂 𝐁𝐮𝐬 Instead of repeatedly converting between AC and DC, the facility distributes power through a 380–800 VDC bus, significantly reducing conversion losses while improving power density. 5. Battery Energy Storage System (BESS) A DC-connected battery system provides: UPS functionality Peak shaving Load balancing Renewable energy buffering Fast ride-through during grid disturbances Direct DC coupling also eliminates several unnecessary conversion stages. 6. DC Rack Busway A high-voltage DC busway distributes power directly above the AI racks, reducing cable complexity while supporting modular expansion for future capacity. 7. AI Compute Racks Each rack receives high-voltage DC and converts it locally to 48 VDC, followed by point-of-load converters that generate the low voltages required by GPUs, CPUs, HBM memory, networking hardware, and storage devices. Why Hybrid AC/DC? Traditional AI data centers typically achieve 90–93% end-to-end electrical efficiency due to multiple AC/DC and DC/DC conversion stages. A hybrid AC/DC architecture can increase overall efficiency to 96–98% or higher, reducing: Electrical losses Heat generation Cooling requirements Power infrastructure footprint Total operating costs For a 100 MW AI campus, improving efficiency from 93% to 98% can eliminate approximately 5 MW of continuous losses—equivalent to several million kilowatt-hours of energy savings each year. Question for the engineering community: Do you see 380–400 VDC becoming the dominant standard for AI facilities, or will the industry move directly toward 800 VDC architectures for next-generation hyperscale deployments? #AIInfrastructure #DataCenters #ElectricalEngineering #PowerDistribution #HighVoltageDC #BatteryStorage #SolidStateTransformer #Hyperscale #EnergyEfficiency #DataCenterDesign #PowerSystems #EngineeringInnovation

The next evolution of sustainable AI isn’t just about using more efficient hardware—it’s about Autonomous AI Agents that code with sustainability in mind. These agents are designed to operate independently, learning and adapting as they go, and have the potential to transform software development by writing energy-efficient code. They don't just optimize for speed; they prioritize minimal resource consumption. Why This Matters for Sustainability Modern AI models consume massive amounts of power, yet software development still prioritizes performance over energy efficiency. Agentic AI could change that paradigm by: ✅ Reducing Computational Waste: AI agents could select or generate the most efficient algorithms based on real-time constraints instead of defaulting to resource-heavy models. For example, they could optimize database queries to reduce data retrieval and processing or dynamically adjust resource allocation based on demand. ✅ Automating Green Software Principles: AI-driven frugal coding practices could optimize data structures, reduce redundant calculations, and minimize memory overhead. This could involve choosing the most energy-efficient programming language or framework for a specific task. ✅ Measuring & Optimizing in Real Time: The reward function would be clear: lower energy consumption, less latency, and reduced emissions—all while maintaining accuracy. ✅ Parallel & Distributed Optimization: AI agents could continuously refine codebases across thousands of cloud instances, improving sustainability at scale. AI-Driven Innovation Archive for Green Coding One of the most exciting ideas in autonomous coding is the "Green Code Archive"—an AI-generated repository of energy-efficient code snippets that could continuously improve over time. Imagine: 🔹 Reusing optimized code instead of reinventing energy-intensive solutions. 🔹 Carbon-aware coding suggestions for green data centers & renewable energy scheduling. 🔹 AI-driven legacy refactoring, automating migration to sustainable architectures. Measuring AI’s carbon footprint after the fact isn’t enough—the goal should be AI that reduces energy use at the source. The future of sustainable tech isn’t just about efficient hardware—it’s about intelligent, autonomous software that optimizes itself for minimal environmental impact. While this technology is still emerging, challenges remain in areas like training complexity and robust validation. However, the potential benefits for a greener future are undeniable. Learn more about leading with Agentic AI and its transformative potential in my book, "Empowering Leaders with Cognitive Frameworks for Agentic AI: From Strategy to Purposeful Implementation" (link in the comments section). #agenticai #greenai #sustainability

My COP30 Ask: Let’s Be Honest About AI’s Energy Problem If you’re not familiar with it, COP is the UN’s annual climate conference — basically the one place where world leaders try (and often struggle) to agree on what we’re all going to do about a warming planet. This year it’s #COP30, and while almost every industry is under review, AI is the elephant in the room. I build enterprise AI for a living. I love this space. I believe in its ability to transform businesses. But I’m also realistic enough to say this out loud: AI is burning through energy like it’s unlimited. And most of the public conversations tiptoe around that. Training a single frontier model can use the electricity of several thousand U.S. homes. Cooling data centers now requires millions of gallons of water. And global data-center demand is expected to double by 2026 — largely because of AI. So if I could ask global leaders at COP30 for one thing, it would be this: Don’t regulate AI outputs first. Regulate AI energy use. Because the next wave of AI won’t be about chatbots, it will be about infrastructure-level systems that run 24/7. And for companies in my industry, enterprise AI, here are the three things we can do right now without waiting for policymakers: 1. Stop pretending bigger is always better. Most enterprise use cases don’t need a 500B-parameter model. A lean 7B or 13B model can do the job with a fraction of the energy. It’s time to optimize for efficiency, not press releases. 2. Design AI that doesn’t live in a permanent “on” state. A lot of energy is wasted on redundant inference. Scheduling, caching, and right-sizing models reduce compute burn without reducing capability. 3. Treat energy like a first-class metric. We obsess over latency and token cost. We need the same dashboards for energy-per-task. If leaders saw how much compute is wasted on oversized prompts or unnecessary calls, they’d redesign their workflows tomorrow. AI can accelerate climate progress ... absolutely! But only if we stop pretending its environmental footprint doesn’t exist. If COP30 can push the industry toward transparency and efficiency, AI can scale and stay sustainable. We just need the courage to say it out loud.

NVIDIA has unveiled a new liquid-cooling architecture that could significantly reduce one of the AI industry’s fastest-growing sustainability challenges: water consumption. Traditional AI data centers rely on cooling towers that evaporate millions of gallons of water each year to remove heat from high-performance servers. NVIDIA’s new approach replaces that process with a closed-loop warm liquid cooling system that delivers coolant directly to AI chips, eliminating the need for most on-site water use. One of the biggest innovations is that the system operates with liquid temperatures of up to 45°C (113°F). Because the coolant is already warm, expensive energy-intensive chillers are no longer required. The liquid continuously circulates through a sealed loop, carrying heat away from the chips while being reused rather than replaced. According to NVIDIA, conventional cooling can consume approximately 2.6 million gallons of water per megawatt annually. In many deployments, this new design can reduce that figure to nearly zero, while also lowering electricity consumption and improving overall energy efficiency. As AI infrastructure scales globally, innovation is no longer just about faster GPUs it’s about building data centers that are faster, more efficient, and environmentally sustainable. Cooling technology may become just as important as compute in defining the next generation of AI infrastructure. #NVIDIA #AI #DataCenters #Sustainability #LiquidCooling #GreenComputing #Infrastructure #GPU #EnergyEfficiency #CloudComputing #FutureOfAI

AI field note: In 2025, AWS data centers used 0.12 liters of water per kilowatt-hour, over 7x more water-efficient than the industry average of 0.84. That efficiency improved even as AI pushed compute demand higher. Here's how we did it.   Cooling a data center presents a three-way tradeoff: water use, energy use, and the temperature margin that keeps servers reliable. Push hard on one and pressure shows up somewhere else. Cool with little energy and you use more water. Cool with little water and you spend more energy on chillers, which draw 25 to 35% more electricity, often when the grid is most stressed. Keep both water and energy low and the servers run warmer, closer to their limits.   We asked if the cooling threshold we had treated as fixed actually had room to move. If the system can operate safely at a higher threshold before water-assisted cooling kicks in, you can keep water and energy low without sacrificing reliability.   So we tested it. Thousands of hours of operational data across campuses showed we could safely raise that threshold, within tested operating conditions, without increasing failure rates. Water-assisted cooling now starts only around 85°F. About 90% of the time, the data centers cool with outside air alone.   The results hold at scale, not just per unit of compute. In Northern Virginia, our largest region by load, water use fell 42% in a year while capacity grew. Across the sites we own and operate, total water withdrawn fell 2% from 2024 to 2025, even as the number of buildings rose. As per-unit efficiency improved, total use went down.   On the hottest hours, when air alone isn't enough, the systems use a small amount of evaporative water rather than switching to chillers that would spike electricity demand when the grid can least absorb it. A little water during peak heat is a lower total burden on the surrounding community than a lot of electricity at the same moment.   The savings for our most common data center designs came from a lot of systems innovation, and from proving that a constraint we'd long accepted as fixed could actually move. In this era, a lot of fixed constraints are worth re-testing.

We’ve been treating AI’s energy problem as a datacenter problem. But the datacenter is the most optimized part of the stack. The real inefficiency is happening higher up. Google estimates that 60% of AI’s energy now comes from inference. Meta says 60 to 70%. AWS, 80-90% of its ML compute demand. A single prompt is insignificant. Billions across apps, agents & API calls aren’t. We’re on track for trillions. The mismatch? We’re optimizing infrastructure while most of the energy and cost are created at the application layer. Even the cleanest datacenter can’t compensate for a stack that sends every request to a frontier model, runs jobs at the costliest hours of the grid, and treats all queries as equally urgent. The biggest gains won’t come from better cooling or more renewable PPAs. They will come from how we design, route & operate the models themselves. There is a way to architect sustainability as a first-order principle in the AI lifecycle itself. So what does a more efficient stack look like? I’m seeing some really cool stuff these days. It begins with grid-aware infra. Platforms like Emerald AI align compute with renewables, shifting batch workloads to cleaner hours and routing traffic to cleaner regions. Crusoe rethinks the foundation entirely by converting stranded natural gas and heat recovery into compute. Training visibility changes how teams build. CodeCarbon exposes the emissions of every experiment forcing real decisions. Once numbers are visible, priorities shift. Does a 2% accuracy gain justify 10× more compute? Then comes inference intelligence. ChatGPT's routing prevents unnecessary over-computing, while GreenPT builds efficiency into the foundation so every inference run uses less power by default. One optimizes after building. The other designs for efficiency from the start. This is where FinOps & sustainability converge. User visibility matters too. Most people have no idea how much energy their prompts consume. When that information becomes visible in real time, behavior shifts. People choose lighter models, batch calls, and refine their prompting. Shared baselines are emerging. The GSF’s SCI turns sustainability into a measurable standard. GPU-level tools like Neuralwatt replace estimates with real power data and expose waste at the hardware level. But none of this works if the layers stay disconnected. This is the logic behind Antarctica: a single observability layer that connects cost, usage, energy, and user behavior across cloud and AI. Grid carbon intensity, training emissions, inference energy, hardware telemetry, and application analytics converge into one source of truth. To make inefficiency measurable at the point of decision. And in AI, every inefficiency appears twice: once as wasted energy and once as wasted dollars. So let’s make this practical now. I’m putting together a shared list of tools that actually improve efficiency across the AI stack. Which ones would you recommend?

The data center industry faces twin pressures: skyrocketing demand for AI compute capacity and increasing energy demands of the grid. But new AI factories can actually support energy and grid efficiency, rather than hurt it.   In my interview with Data Centre Magazine, I discuss reshaping power design to build responsibly: ➡️ First, transitioning to an 800V DC architecture can reduce total energy costs by roughly 9%, delivering power directly to the rack. ➡️ Second, battery storage allows operators to use peak shaving during high-stress periods, effectively turning data centers into virtual power plants. ➡️ Finally, hybrid retrofitting with high-efficiency components selectively targets heavy AI loads while keeping existing structures standing.   The best data centers of the future will be integrated energy hubs working with utilities to deliver reliable, sustainable power.   Read the full interview here: https://lnkd.in/eCnDV_5q   #DataCentres #DataCenters #AIFactories #EngineeredToOutrun