POWER DELIVERY Protection and telemetry in AI with 800 V hot swap

Related Vendors

Diotec Semiconductor AG

MERSEN France SB

This article addresses the growing power demands of AI server environments, driven by the increasing power consumption of GPUs. It highlights the shift from 48 V to 800 V power delivery architectures and explores engineering approaches to high-voltage hot-swap protection as data center infrastructure evolves.

As AI workloads intensify, GPUs in server environments are driving unprecedented power demands—prompting a shift toward an 800 V power delivery architecture at the rack level. This elevated voltage introduces new challenges for system protection and monitoring, particularly during live tray insertion and removal. To address these challenges, next-generation hot-swap controllers are required to manage high-voltage inrush currents while providing robust telemetry for system diagnostics and safety.

Why does 800 V matter?

Two key trends are driving the transition to higher voltage power delivery at the rack level in AI server infrastructure.

1. Rising GPU Power Consumption: As GPUs continue to scale in compute performance, their power requirements are increasing significantly.

2. Higher Compute Density per Rack: To maximize performance and minimize interconnect latency for large-scale AI training and inference, more GPUs are being consolidated within a single rack. This co-location improves bandwidth efficiency and reduces communication overhead by shortening data paths between accelerators.

To meet the resulting surge in rack-level power demand, the industry is moving toward a disaggregated power architecture. In this model, traditional power components such as power distribution units (PDUs), battery backup units (BBUs), and capacitor units (CUs) are relocated from the main IT rack to an adjacent power sidecar. This separation not only supports higher voltage delivery, such as 800 V, but also frees up valuable space within the rack for additional compute resources.

RESISTOR CALCULATION

How to calculate voltage drop across a resistor?

The existing approach to power delivery

AI racks operate with a 48 V busbar (Figure 1). AC is converted to 48 V_DC using in-rack power supply units (PSUs). The 48 V bus feeds IT equipment (server PSUs, GPU nodes, switches), BBUs with their own bidirectional DC-to-DC converters, and supercapacitor units (SCUs) for fast ride-through and transient handling.

Each node on a rack (for example, a server tray) requires the capability to be switched in and out during live operation. In practice, service technicians replace servers via hot-swapping, allowing them to remove a node, swap it, and push it back in without shutting down the entire rack.

During this event, the node’s hot-swap controller disconnects power internally, allowing the node to be removed safely. The rack’s busbar stays energized and continues feeding all other nodes, BBUs, and SCUs. Once a replacement node is inserted, precharge circuits limit inrush before full connection, a key function of the hot-swap controller. The node then boots and rejoins the AI cluster automatically.

Figure 2 shows the power distribution of an AI server from 48 V busbar to processor. The hot swap controller is clearly shown as the first interface to allow the technician to safely and efficiently remove the node.

Evolution to higher voltages

With the increased power load from AI processors, a new horizon in higher DC voltage distribution to the rack is emerging. ADI is on the cutting edge of these solutions and working closely with industry leaders to solve such problems.

A new voltage distribution domain is defined to the rack—800 V. This is fed from a sidecar directly to the IT rack. Figure 3 shows the proposed architecture.

The primary driver for transitioning to higher rack-level voltages is the escalating power demand per rack. As power increases, so does the current flowing through the busbars. This rise in current necessitates larger and heavier busbars to maintain acceptable thermal and electrical performance—posing significant challenges in terms of mechanical practicality and system design. By increasing the delivery voltage, the required current can be proportionally reduced, enabling the use of more compact and manageable busbar dimensions and interconnects. This shift to higher voltage is essential for maintaining scalable, efficient, and mechanically feasible power distribution in next-generation AI server racks.

It’s worth noting there can exist intermediary steps in this architecture evolution where a 48 V busbar is used in the rack. In this case, PSUs are scaled to higher powers, accepting 800 V at the rack and downconverting to 48 V to marry with existing busbars. However, this is not a longer-term state as power will be limited (maybe up to 250 kW per rack) and the goal is to allow for maximum compute capability in a single rack, driving power to 1 MW per rack by the end of the decade. PSUs and BBUs are therefore best positioned outside of the IT compute rack, as shown in Figure 3.

In the future, solid-state transformers are likely to be used to distribute high voltage directly to each IT compute rack without the need for these sidecars.

POWER SAFETY

Bourns launches compact high-voltage GDT surge protectors

Taking hot swap to the next level

Designing hot-swap protection for 800 V architectures presents several distinct technical challenges that must be addressed:

Power Density: This is a critical factor, as the 800 V hot swap circuitry ultimately resides on the server card within the IT rack. Space on these cards is highly valuable and will face growing constraints as rack densities increase, fitting more servers into each rack. As a result, high voltage hot swap solutions must be designed to occupy the smallest possible footprint.

High Voltage Control and Protection: Transitioning to 800 V introduces significant safety challenges. While 48 V poses minimal shock risk, these higher voltages can be life-threatening. Consequently, hot swap circuitry must handle large current surges within microseconds, making precise control essential. Managing the inrush current ramp is critical to avoid damage to the rack or harm to technicians. The system must also coordinate timing, detect overcurrent or undervoltage events, and shut down gracefully when required.

Telemetry: As the first component in the node’s power path, hot-swap controllers are positioned for data acquisition. They must precisely measure voltage, current, and power while logging events such as overcurrent, undervoltage, and thermal shutdowns. In addition, reporting temperatures from the power switch (MOSFET) or nearby PCB areas delivers value by enabling real-time load current profiling, which improves energy demand forecasting and supports predictive maintenance.

Component manufacturers are actively developing high-voltage hot-swap controller solutions to address these emerging architectural requirements, focusing on compact form factors, precise high-power control, and enhanced data acquisition accuracy.

Conclusion

This article highlights the critical shift toward 800 V power delivery in AI server racks to accommodate the increasing power requirements of advanced GPUs and support higher compute density. The new architecture involves moving power components to a separate sidecar, optimizing space for compute resources in the main rack. Next-generation high-voltage hot-swap controllers effectively manage inrush currents, offer comprehensive telemetry for system diagnostics and safety, and ensure reliable operation in high-voltage environments.

About the author

*Donal McCarthy is a marketing director at ADI focused on data center power. Donal holds a B.E.E. from University College Cork, an M.B.A. from Boston College, and a marketing diploma from the Irish Management Institute in Dublin. Donal’s held various roles including design engineer at M/A COM, field sales engineer and marketing roles at Hittite, and marketing management and director roles at Analog Devices.

(ID:50846966)