Working with GPUs - Part10 - Energy Storage Systems, BBUs, and CBUs explained

As AI clusters transition from tens of kilowatts to over 200 kw per rack, traditional server power architectures are hitting their physical limits. High-density GPU accelerators like Nvidia Hopper, Blackwell, and liquid-cooled OCP ORV3 HPR racks introduce extreme dynamic power loads that challenge legacy data center distribution.

To prevent continuous voltage droops, nuisance trips, and power supply shutdowns under bursting workload conditions, the data center industry is shifting its approach to power delivery. Centralized power management is giving way to a multi-tiered Energy Storage System (ESS) network distributed across every level of the facility - from containerized yards down to sidecar racks, rack-level Battery Backup Units (BBUs) and Capacitor Backup Units (CBUs).

In this post, we’ll explore the paradigm shift in data center energy storage, examine why AI changes the stakes, break down multi-tier ESS deployment levels, and highlight the architectural planning required for safe, efficient operation.

Why Energy Storage Matters Beyond Central UPS

Historically, data center backup power was straightforward: a massive, centralized room-level Uninterruptible Power Supply (UPS) paired with diesel generators handled every outage. Today, that model is breaking down.

• Distributed Deployment: Energy storage is no longer confined to a single, centralized grey-space room. Modern high-density architectures distribute storage across multiple tiers: facility-level yards, row-level rows/sidecars, and rack-level modules.
• Proximity to the IT Load: Moving energy storage closer to the compute load fundamentally changes the engineering conversation. Instead of pushing massive transient currents over long busbars from a distant UPS, local buffering absorbs shocks/ spikes right where they happen.
• Flexible Footprints: Today, ESS components can live wherever physical efficiency dictates - whether in dedicated equipment rooms, active white space, grey service corridors, or outdoor container yards.

Why AI Changes the Stakes for Energy Storage

Traditional cloud workloads follow relatively stable, predictable power utilization curves. Artificial intelligence and machine learning training jobs, however, behave entirely differently.

• Extreme Power Dynamics: Large AI clusters experience rapid load changes, massive peak currents, and intense local power densities. When thousands of tensor cores synchronize to process a massive matrix multiplication, the instantaneous jump in current draw can cause catastrophic voltage droops.
• Multi-Tiered Time Bands: Different energy storage technologies operate across vastly different time scales:
• Nanoseconds to Milliseconds: Supercapacitors absorb high-frequency ripples and microsecond spikes.
• Seconds to Minutes: Rack-level lithium-ion BBUs provide ride-through power for short grid glitches or graceful checkpoint shutdowns.
• Hours: Facility-level battery energy storage systems and generators cover extended utility outages.
• The "One-Size-Fits-All" Fallacy: No single energy storage technology can cover every time band efficiently. High-power density demands a hybrid approach where specialized technologies handle specific layers of the power delivery network.

Image credits: academy.opencompute.org

Levels of ESS Deployment

In modern high-performance AI data centers, energy storage is deployed across distinct, coordinated tiers:

Image credits: academy.opencompute.org

Rack-Level BBUs: Standardized via the OCP ORv3 HPR architecture, rack-level battery module enclosures slide directly into the vertical 48V DC busbar. They provide 2 to 4 minutes of local ride-through power.

Supercapacitor Buffers (CBUs): Placed right at the rack as physically separate modular enclosures, supercapacitors neutralize sub-millisecond voltage droops and transient spikes caused by rapid GPU/TPU execution shifts, shielding battery chemistry from repetitive thermal stress. CBU play a critical role in power smoothing within modern high-density AI infrastructure as the multi-node GPU clusters shift from low-power states to intense tensor execution almost instantaneously, they generate violent current spikes and high-frequency voltage ripples. CBUs smooth out these power instabilities and the power supply, and BBU units are shielded from the "bursty" nature of AI workloads. The CBU acts as a localized shock absorber, turning erratic, volatile power demands into a smooth, continuous load profile for the upstream rectifiers and lithium-ion batteries.

Sidecar Enclosures: For extreme 100 kw+ racks, rectifiers, BBUs, and CBUs are frequently grouped into dedicated vertical power sidecars attached to the side of the compute rack.

Centralized UPS & Containerized BESS: Massive lithium-ion or advanced chemistry battery arrays deployed in dedicated power rooms or outdoor modular yards. These act as the primary bridge between sudden grid failure and emergency generator startup.

Onsite Generation Integration: Modern resilient data centers increasingly integrate localized microgrid components - such as solar arrays, fuel cells, or advanced generators - into the facility ESS loop to stabilize incoming power and reduce reliance on fragile public grids.

The Recharge Challenge: Managing Grid Limits and PUE

Deploying distributed energy storage introduces a critical engineering consideration: every ESS must eventually recharge.

• The Coordination Problem: If a large fleet of rack-level BBUs or facility BESS units all initiate high-current recharging simultaneously after a minor grid glitch or peak-shaving event, they can inadvertently breach facility max-demand thresholds.
• Impact on PUE: Uncoordinated recharging spikes total facility load, driving up power consumption and negatively impacting Power Usage Effectiveness (PUE) metrics.
• Architectural Integration: Recharging cycles must be algorithmically scheduled and integrated directly into the site's power management firmware, ensuring that energy replenishment happens smoothly during off-peak windows or throttled alongside current IT workload demands.

As AI infrastructure scales into the next generation of high-density computing, treating power delivery and energy storage as an intelligent, distributed ecosystem is no longer optional - it is the baseline for uncompromised uptime.

References

• academy.opencompute.org
• Power solutions of CBU and BBU

Working with GPUs - Part9 - Power shelves in OCP ORV3 High Power Rack

The ORV3 HPR Power Shelf is a standardized, ultra-high-density power distribution system designed under the Open Compute Project (OCP) framework. Essentially, it acts as a centralized power hub for a server rack. Instead of every individual server having its own power supply unit (PSU) throwing off heat and hogging space, the power shelf sits in the rack, takes high-voltage AC utility power and converts it into a single, massive pool of 48V DC power. This power is then delivered to the entire rack via a heavy-duty vertical copper backplane (busbar).

AI and LLM workloads are notoriously bursty; a GPU cluster can spike from an idle state to maximum power draw in microseconds. The ORV3 HPR power shelf features high pulse-load capabilities (often supporting up to 150% load capacity for transient windows) and active current sharing. This smooths out dynamic loading and prevents voltage sags without tripping upstream data center breakers. Equipped with an integrated Power Management Controller (PMC), these shelves expose real-time metrics, black-box fault logging, and granular thermal monitoring via standard APIs like DMTF Redfish over Gigabit Ethernet. This allows infrastructure teams to optimize power provisioning, balance loads accurately, and proactively manage hot spots in the data center.

Power shelf is a group c, third party component present in Nvidia GB200/300 NVL72 rack. 

• Currently Nvidia supports Delta and LiteOn power shelves. 
• They are 33KW EIA 1 RU (6 x 5.5KW PSUs) units with additional bulk capacitors and 60A whip support. 
• Nvidia GB300 NVL72 MGX rack has single bus bar in the middle, and total 6 power shelves located at top and bottom of the rack. 
• The rack power consumption is approximately 120kW, and the 6 power shelves will provide N+2 redundancy.

Power Shelf firmware

• PMC firmware is based on Open BMC.
• The firmware directly governs how power is managed, balanced, and protected across the entire rack.
• It controls the internal switching frequencies, power factor correction (PFC), and voltage regulation loops of the individual PSUs.
• It ensures that if you have six PSUs in a shelf, they all pull their weight equally. If one PSU lags, the firmware recalibrates the others in microseconds to prevent overloading a single unit.
• It dictates how the shelf handles massive, sudden spikes in power when GPUs transition from idle to 100% utilization.
• It hosts the communication protocols (like Modbus, PMBus, or Redfish over Ethernet) used by the Power Management Controller (PMC) to talk to your rack-level orchestrators.

Accessing Power shelf

• Power Shelf has a PMC (Power Management Controller) - connected via ethernet port.
• This PMC usually gets connected to OOB network.
• You can access it via web UI, SSH, or Redfish.
• Also supports SNMP.

Updating Power shelf platform firmware

• We need to update two things:
• PMC firmware
• PSU firmware
• This can be done using nvfwupd utility.
• Notes:
• LiteOn PSUs may only be updated one at a time. To select the PSU to update, a special JSON file containing the “LiteOnPowerDeviceId” value is required.
• Delta PSUs update simultaneously and no special JSON file is required.
• After the update completes, PowerShelf components automatically activate the new firmware.
• Starting with NVFWUPD 2.1.0, OnReset activation is supported for Delta and LiteOn PowerShelf platforms. By default, all updates use immediate activation.
• OnReset activation applies only to PMC firmware updates on Delta and LiteOn PowerShelf platforms. PSU firmware updates always use immediate activation.

Operational reality and firmware update

• You don't need to patch power shelf firmware monthly. Usually, a biannual or annual cadence or aligning updates with major hardware maintenance windows is standard.
• When expanding clusters or adding new generations of compute nodes to existing racks, updating the power shelf firmware ensures the power delivery system is fully compatible with the power sequencing behaviors of the newer servers.
• Modern ORV3 shelves support hitless/lossless firmware updates. This means you can flash the firmware on the Power Management Controller (PMC) or individual PSUs sequentially while the rack remains fully powered and live, eliminating the need to take your compute offline just to update the power system.

References

• PowerShelf Platform Update Examples — NVIDIA Firmware Update Guide
• LITEON GPU Power Solutions
• Requirements/Considerations of Next Generation ORv3 PSU and Power Shelves
• Power Shelf Management with Redfish - presented by Delta
• Scalable Rack-Level Power for AI Servers | FSP OCP ORv3 Power Shelf

Working with GPUs - Part8 - Powering and cooling AI accelerators with OCP ORV3 HPR

When we talk about working with GPUs at scale, we usually focus on the software stack, CUDA tuning, or optimizing workloads across NVLink, etc. But if you are an infrastructure SRE managing bare-metal clusters, you quickly run into a much harsher physical reality: Power and Thermals.

As we deploy next-generation AI platforms like Nvidia’s GB200/ 300 NVL72 architectures - the power demands are obliterating traditional infrastructure. Standard server racks are physically hitting a wall. To keep these high-density clusters running without melting down, the industry is rapidly transitioning to the Open Compute Project’s (OCP) Open Rack Version 3 (ORV3) High Power Rack (HPR).

Here is what you need to know about how this architecture feeds and cools modern GPU nodes.

The GPU infrastructure bottleneck

Traditional data center racks rely on a 19-inch width standard with standard 44.45 mm Rack Units (RU). In that legacy model, every individual server chassis houses its own AC-to-DC power supply units (PSUs).

If you try to stuff a modern cluster of high-TDP GPUs into that traditional 19-inch racks, you run into immediate problems:

• Cable chaos: The back of the rack becomes choked with heavy AC power cords, blocking vital airflow.
• Efficiency loss: Converting AC to DC at every single node generates massive heat and power waste.
• Weight limits: GPU nodes are incredibly dense and heavy; standard frames simply aren't structurally rated for them.

How ORV3 changes the game for GPU compute

Developed collaboratively by hyper-scalers like Meta, Google, and Microsoft, the ORV3 standard throws out the legacy playbook to accommodate modern accelerator demands.

Instead of treating the rack like a cabinet for isolated servers, ORV3 treats the entire rack as a single, unified compute machine:

• Native 21-inch bays and Open Units (OU): Provides a wider internal bay (21 inches wide vs. the traditional 19 inches). It replaces RUs with Open Units (OU = 48mm), offering more structural space for complex GPU heat sinks and optimized front-to-rear airflow.

21-inch-wide rack

44 OU

• Centralized 48V DC busbar: Individual server power supplies are completely gone. Instead, 3-phase AC or High Voltage DC (HVDC) enters a centralized power shelf, which converts it to 48V DC. This power is run down a copper busbar mounted at the rear center of the rack.

48V busbar

Busbar BarKlip connector

• Power shelves: The ORV3 HPR power shelf acts as a centralized power hub that converts incoming 3-phase AC (from the overhead busways) into 48V DC power distributed via the rear busbar. Housing high-density 5.5 kW Power Supply Units (PSUs), a single 1U power shelf delivers up to 33kW of total output. Racks deploy multiple shelves in N+N or N+1 redundant configurations - complete with integrated Power Monitor Modules (PMM) to supply reliable, cable-free energy to high-TDP GPU clusters.

AC input overhead busways

AC input - Power shelf - DC output - Busbar

Power shelf with multiple PSUs inside it

GB200 NVL72 ORV3 HPR rack

• Blind-mate infrastructure: When you slide a heavy GPU compute node into the rack, it connects directly to the 48V DC busbar via copper clips (blind-mate connections). No power cables required. The specialized copper clip/ jaw that physically clamps onto the vertical busbar blade to transmit high-current DC power is called a Busbar BarKlip connector. Blind-mate connectors are widely used for liquid cooling as well in modern high-density data centers. In the OCP ORV3 HPR architecture, the concept of "blind-mating" applies to both power and coolant distribution. Instead of manually hooking up coolant hoses to the back of a server, liquid-cooled blind-mate connectors (often referred to as BMQC or Blind Mate Quick Connectors) allow fluid lines to engage automatically as the compute tray slides into the rack.

Liquid cooling blind-mate connectors and manifold

• Heavy duty chassis support: The frame is built to support up to 1400 kg, meaning it won't buckle under a full stack of liquid-cooled accelerators.

The ORV3 HPR specification

While standard ORV3 configurations top out at 18 kW to 36 kW per rack, modern AI workloads easily blow past those thresholds. To keep up with platforms like the Nvidia GB200 pushing rack limits to 140kW and beyond, the OCP community introduced the ORV3 HPR (High Power Rack) variant which is an extension of ORV3.

• Power density: 92 kW to 140 kW+ / rack
• Power shelf capacity: 5.5 kW PSUs (33 kW total per power shelf)
• Cooling architecture: Blind-Mate Direct Liquid Cooling (DLC) Manifolds

To handle massive electrical currents without thermal runaway, the HPR upgrades to a massive 80 kg busbar with deeper tracking and aggressive grounding. More importantly, it addresses the massive heat generated by high-TDP GPUs by integrating blind-mate liquid cooling manifolds right into the chassis. Just like the power clips, the liquid cooling loops engage automatically when the node is seated.

What’s next: The 1-Megawatt sidecar

As we look forward, GPU power requirements show no signs of slowing down. As clusters head toward 1 Megawatt (MW) per rack, the OCP community is already developing Project Mount Diablo. This next step introduces a disaggregated Power Rack Sidecar, moving the massive rectifiers completely outside of the main compute rack so we can fill every square inch of the primary frame with pure, liquid-cooled GPU compute.

Working with GPUs at scale means understanding the infrastructure that keeps them alive. Without open standards like ORV3 HPR solving the physical limitations of power delivery and fluid dynamics, the next leap in AI compute wouldn't even be able to turn on.

References

• Open Rack and ORv3: Past, Present, and Future - OCP Academy
• ORV3_Rack07302024
• OCP ORv3 High Power Rack HPR Ecosystem Solution

Hope it was useful. Cheers!

Working with GPUs – A Practical Blog Series

This blog series captures practical learnings from working with GPUs in real‑world environments, with a focus on operations, reliability, and scale. Each post deep‑dives into specific aspects of GPU systems based on hands‑on experience, incidents, and operational challenges. Together, these articles aim to share actionable insights, highlight common pitfalls, and help teams build more robust and predictable GPU operations.

Part 01: Using nvidia-smi
Part 02: Memory fault indicators
Part 03: Using dcgmi
Part 04: Thermal issues
Part 05: XID errors
Part 06: H100 SXM5 architecture
Part 07: GPU has fallen off the bus
Part 08: Powering and cooling AI accelerators with OCP ORV3 HPR
Part 09: Power shelves in OCP ORV3
Part 10: Energy storage systems, BBUs, and CBUs