Working with GPUs - Part10 - Energy Storage Systems, BBUs, and CBUs explained

As AI clusters transition from tens of kilowatts to over 200 kw per rack, traditional server power architectures are hitting their physical limits. High-density GPU accelerators like Nvidia Hopper, Blackwell, and liquid-cooled OCP ORV3 HPR racks introduce extreme dynamic power loads that challenge legacy data center distribution.

To prevent continuous voltage droops, nuisance trips, and power supply shutdowns under bursting workload conditions, the data center industry is shifting its approach to power delivery. Centralized power management is giving way to a multi-tiered Energy Storage System (ESS) network distributed across every level of the facility - from containerized yards down to sidecar racks, rack-level Battery Backup Units (BBUs) and Capacitor Backup Units (CBUs).

In this post, we’ll explore the paradigm shift in data center energy storage, examine why AI changes the stakes, break down multi-tier ESS deployment levels, and highlight the architectural planning required for safe, efficient operation.

Why Energy Storage Matters Beyond Central UPS

Historically, data center backup power was straightforward: a massive, centralized room-level Uninterruptible Power Supply (UPS) paired with diesel generators handled every outage. Today, that model is breaking down.

• Distributed Deployment: Energy storage is no longer confined to a single, centralized grey-space room. Modern high-density architectures distribute storage across multiple tiers: facility-level yards, row-level rows/sidecars, and rack-level modules.
• Proximity to the IT Load: Moving energy storage closer to the compute load fundamentally changes the engineering conversation. Instead of pushing massive transient currents over long busbars from a distant UPS, local buffering absorbs shocks/ spikes right where they happen.
• Flexible Footprints: Today, ESS components can live wherever physical efficiency dictates - whether in dedicated equipment rooms, active white space, grey service corridors, or outdoor container yards.

Why AI Changes the Stakes for Energy Storage

Traditional cloud workloads follow relatively stable, predictable power utilization curves. Artificial intelligence and machine learning training jobs, however, behave entirely differently.

• Extreme Power Dynamics: Large AI clusters experience rapid load changes, massive peak currents, and intense local power densities. When thousands of tensor cores synchronize to process a massive matrix multiplication, the instantaneous jump in current draw can cause catastrophic voltage droops.
• Multi-Tiered Time Bands: Different energy storage technologies operate across vastly different time scales:
• Nanoseconds to Milliseconds: Supercapacitors absorb high-frequency ripples and microsecond spikes.
• Seconds to Minutes: Rack-level lithium-ion BBUs provide ride-through power for short grid glitches or graceful checkpoint shutdowns.
• Hours: Facility-level battery energy storage systems and generators cover extended utility outages.
• The "One-Size-Fits-All" Fallacy: No single energy storage technology can cover every time band efficiently. High-power density demands a hybrid approach where specialized technologies handle specific layers of the power delivery network.

Image credits: academy.opencompute.org

Levels of ESS Deployment

In modern high-performance AI data centers, energy storage is deployed across distinct, coordinated tiers:

Image credits: academy.opencompute.org

Rack-Level BBUs: Standardized via the OCP ORv3 HPR architecture, rack-level battery module enclosures slide directly into the vertical 48V DC busbar. They provide 2 to 4 minutes of local ride-through power.

Supercapacitor Buffers (CBUs): Placed right at the rack as physically separate modular enclosures, supercapacitors neutralize sub-millisecond voltage droops and transient spikes caused by rapid GPU/TPU execution shifts, shielding battery chemistry from repetitive thermal stress. CBU play a critical role in power smoothing within modern high-density AI infrastructure as the multi-node GPU clusters shift from low-power states to intense tensor execution almost instantaneously, they generate violent current spikes and high-frequency voltage ripples. CBUs smooth out these power instabilities and the power supply, and BBU units are shielded from the "bursty" nature of AI workloads. The CBU acts as a localized shock absorber, turning erratic, volatile power demands into a smooth, continuous load profile for the upstream rectifiers and lithium-ion batteries.

Sidecar Enclosures: For extreme 100 kw+ racks, rectifiers, BBUs, and CBUs are frequently grouped into dedicated vertical power sidecars attached to the side of the compute rack.

Centralized UPS & Containerized BESS: Massive lithium-ion or advanced chemistry battery arrays deployed in dedicated power rooms or outdoor modular yards. These act as the primary bridge between sudden grid failure and emergency generator startup.

Onsite Generation Integration: Modern resilient data centers increasingly integrate localized microgrid components - such as solar arrays, fuel cells, or advanced generators - into the facility ESS loop to stabilize incoming power and reduce reliance on fragile public grids.

The Recharge Challenge: Managing Grid Limits and PUE

Deploying distributed energy storage introduces a critical engineering consideration: every ESS must eventually recharge.

• The Coordination Problem: If a large fleet of rack-level BBUs or facility BESS units all initiate high-current recharging simultaneously after a minor grid glitch or peak-shaving event, they can inadvertently breach facility max-demand thresholds.
• Impact on PUE: Uncoordinated recharging spikes total facility load, driving up power consumption and negatively impacting Power Usage Effectiveness (PUE) metrics.
• Architectural Integration: Recharging cycles must be algorithmically scheduled and integrated directly into the site's power management firmware, ensuring that energy replenishment happens smoothly during off-peak windows or throttled alongside current IT workload demands.

As AI infrastructure scales into the next generation of high-density computing, treating power delivery and energy storage as an intelligent, distributed ecosystem is no longer optional - it is the baseline for uncompromised uptime.

References

• academy.opencompute.org
• Power solutions of CBU and BBU

Working with GPUs - Part9 - Power shelves in OCP ORV3 High Power Rack

The ORV3 HPR Power Shelf is a standardized, ultra-high-density power distribution system designed under the Open Compute Project (OCP) framework. Essentially, it acts as a centralized power hub for a server rack. Instead of every individual server having its own power supply unit (PSU) throwing off heat and hogging space, the power shelf sits in the rack, takes high-voltage AC utility power and converts it into a single, massive pool of 48V DC power. This power is then delivered to the entire rack via a heavy-duty vertical copper backplane (busbar).

AI and LLM workloads are notoriously bursty; a GPU cluster can spike from an idle state to maximum power draw in microseconds. The ORV3 HPR power shelf features high pulse-load capabilities (often supporting up to 150% load capacity for transient windows) and active current sharing. This smooths out dynamic loading and prevents voltage sags without tripping upstream data center breakers. Equipped with an integrated Power Management Controller (PMC), these shelves expose real-time metrics, black-box fault logging, and granular thermal monitoring via standard APIs like DMTF Redfish over Gigabit Ethernet. This allows infrastructure teams to optimize power provisioning, balance loads accurately, and proactively manage hot spots in the data center.

Power shelf is a group c, third party component present in Nvidia GB200/300 NVL72 rack. 

• Currently Nvidia supports Delta and LiteOn power shelves. 
• They are 33KW EIA 1 RU (6 x 5.5KW PSUs) units with additional bulk capacitors and 60A whip support. 
• Nvidia GB300 NVL72 MGX rack has single bus bar in the middle, and total 6 power shelves located at top and bottom of the rack. 
• The rack power consumption is approximately 120kW, and the 6 power shelves will provide N+2 redundancy.

Power Shelf firmware

• PMC firmware is based on Open BMC.
• The firmware directly governs how power is managed, balanced, and protected across the entire rack.
• It controls the internal switching frequencies, power factor correction (PFC), and voltage regulation loops of the individual PSUs.
• It ensures that if you have six PSUs in a shelf, they all pull their weight equally. If one PSU lags, the firmware recalibrates the others in microseconds to prevent overloading a single unit.
• It dictates how the shelf handles massive, sudden spikes in power when GPUs transition from idle to 100% utilization.
• It hosts the communication protocols (like Modbus, PMBus, or Redfish over Ethernet) used by the Power Management Controller (PMC) to talk to your rack-level orchestrators.

Accessing Power shelf

• Power Shelf has a PMC (Power Management Controller) - connected via ethernet port.
• This PMC usually gets connected to OOB network.
• You can access it via web UI, SSH, or Redfish.
• Also supports SNMP.

Updating Power shelf platform firmware

• We need to update two things:
• PMC firmware
• PSU firmware
• This can be done using nvfwupd utility.
• Notes:
• LiteOn PSUs may only be updated one at a time. To select the PSU to update, a special JSON file containing the “LiteOnPowerDeviceId” value is required.
• Delta PSUs update simultaneously and no special JSON file is required.
• After the update completes, PowerShelf components automatically activate the new firmware.
• Starting with NVFWUPD 2.1.0, OnReset activation is supported for Delta and LiteOn PowerShelf platforms. By default, all updates use immediate activation.
• OnReset activation applies only to PMC firmware updates on Delta and LiteOn PowerShelf platforms. PSU firmware updates always use immediate activation.

Operational reality and firmware update

• You don't need to patch power shelf firmware monthly. Usually, a biannual or annual cadence or aligning updates with major hardware maintenance windows is standard.
• When expanding clusters or adding new generations of compute nodes to existing racks, updating the power shelf firmware ensures the power delivery system is fully compatible with the power sequencing behaviors of the newer servers.
• Modern ORV3 shelves support hitless/lossless firmware updates. This means you can flash the firmware on the Power Management Controller (PMC) or individual PSUs sequentially while the rack remains fully powered and live, eliminating the need to take your compute offline just to update the power system.

References

• PowerShelf Platform Update Examples — NVIDIA Firmware Update Guide
• LITEON GPU Power Solutions
• Requirements/Considerations of Next Generation ORv3 PSU and Power Shelves
• Power Shelf Management with Redfish - presented by Delta
• Scalable Rack-Level Power for AI Servers | FSP OCP ORv3 Power Shelf