What Is a GPU Server? Components, Server Classes and Site Requirements
Summary
- A GPU server is a data center server built around several GPUs that run many calculations at once, for AI training and inference.
- Operators buy three classes: PCIe servers, 8-GPU HGX servers and rack-scale systems such as the NVIDIA GB300 NVL72.
- Power and cooling are the gating check, since one air-cooled 8-GPU server can draw 15 kW, more than the most typical rack in most facilities.
Introduction
GPU-accelerated servers now account for more than half of the industry's sales. IDC reported in September 2026 that they generated $87.4 billion in the second quarter, up 28.1% year over year and 52.6% of a record $166.3 billion market. TrendForce forecast in October 2026 that AI servers will generate about $479 billion this year, nearly 75% of a projected $639.2 billion, with shipments up 32.2%.
That buildout meets the same limit at every scale: most data center racks were built to draw less power than a single 8-GPU server needs. A server delivered to a site that can't power or cool it earns nothing until the facility catches up, so the server class and the site have to be chosen together.
This guide explains what's inside a GPU server, the three classes operators buy, the power and cooling each one needs, and how a purchase is priced and financed.
What a GPU Server Is and What's Inside It
A GPU server is a data center machine that pairs CPUs with several graphics processors, chips built for parallel math. The GPUs let it run AI training, inference and HPC (high-performance computing) workloads that would take a CPU-only server far longer.
A CPU works through sequential, general-purpose tasks on a few powerful cores. GPUs take the opposite approach. NVIDIA's explainer says they "break complex problems into thousands or millions of separate tasks and work them out at once." That parallel design suits the matrix math behind AI models.
Inside, a GPU server holds the GPUs with their high-bandwidth memory (HBM, memory stacked on the GPU package), host CPUs, system memory and storage. It also carries network adapters, power supplies and cooling sized for the GPUs. An NVIDIA HGX B300 system carries 2.1 TB of GPU memory across eight GPUs, as listed in the HGX B300 hardware reference.
Many buyers miss that HGX is a component. NVIDIA HGX "is available in a single baseboard with eight" Rubin, Blackwell or Blackwell Ultra SXM GPUs. OEMs (original equipment manufacturers such as Supermicro and Dell) build that baseboard into complete servers.
Because of that, an equipment schedule should list both the system model and its builder. DGX is NVIDIA's own complete system built around the same GPUs, and we compare it with HGX in a separate article.
| CPU server | GPU server | |
|---|---|---|
| Main processors | CPUs only | CPUs plus multiple GPUs |
| Workload type | Sequential, general-purpose tasks | Parallel work such as AI training, inference, HPC and rendering |
| Memory | System memory | GPU memory (HBM on SXM GPUs, GDDR7 on some PCIe cards) plus system memory |
| Typical power per server | Fits standard rack power budgets | Far higher (see the site section below) |
| Cooling | Air | Air or liquid, depending on server class |
The Three GPU Server Classes
Operators buy GPU compute in three classes, and the class decides the unit of purchase, the cooling and the rack plan. PCIe and HGX servers are bought and racked one server at a time, while a rack-scale system arrives as a complete rack. Financings originated through USD.AI cover all three classes, as the cost section below shows.
| Class | GPU form factor | GPUs per unit | Unit of purchase | Cooling | Example system |
|---|---|---|---|---|---|
| PCIe server | PCIe cards in standard slots | Up to 8 | Server | Air-cooled or liquid-cooled card versions | NVIDIA RTX PRO Server with RTX PRO 6000 Blackwell Server Edition |
| 8-GPU HGX server | SXM GPUs on one HGX baseboard | 8 | Server | Air-cooled designs available | Supermicro 8U HGX B300 system |
| Rack-scale system | GPUs and Grace CPUs linked across one rack | 72 GPUs and 36 CPUs | Rack | Fully liquid-cooled | NVIDIA GB300 NVL72 |
PCIe GPU servers
PCIe GPU servers are the entry class for smaller deployments and own-use buyers. PCIe GPUs are cards that plug into standard server slots. NVIDIA's RTX PRO Servers from Cisco, Dell, HPE, Lenovo and others support up to eight RTX PRO 6000 Blackwell GPUs, per its May 2025 announcement.
The RTX PRO 6000 Blackwell Server Edition is a PCIe Gen 5 card drawing up to 600 W (configurable), with 96 GB of GDDR7 memory. GDDR7 is graphics memory mounted on the card, a different type from the HBM on SXM GPUs. SXM and PCIe GPUs also differ in how they connect and how much power they draw, which we cover in a separate article.
8-GPU HGX servers
An HGX server puts eight SXM GPUs on one HGX baseboard. SXM GPUs socket directly onto the baseboard, which gives them higher performance and a direct NVLink connection, NVIDIA's GPU-to-GPU interconnect. On the HGX B300, NVIDIA lists fifth-generation NVLink at 1.8 TB/s per GPU, so all eight GPUs can work on one model together.
The OEM sets the chassis, CPUs, power supplies and cooling around the baseboard. Two HGX B300 servers from different builders can therefore place different demands on a site. We compare the main OEM server manufacturers in a separate article.
Rack-scale systems
In a rack-scale system, the rack is the unit you buy and deploy. NVIDIA describes the GB300 NVL72 as a "fully liquid-cooled, rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs and 36 Arm-based NVIDIA Grace CPUs."
That design makes the site review weigh as heavily as the system itself. A facility has to bring liquid cooling to the rack before the system goes in. The GB300 rack-scale reference flags facility cooling as a planning item. We cover NVL72 systems in more depth in a separate article.
What a GPU Server Demands of a Site
One air-cooled 8-GPU server can draw 15 kW at full load, more than the most typical rack in most facilities. A four-server rack at 60 kW sits above anything most facilities run, which is why site power and cooling get checked before a purchase order.
Supermicro lists those figures for its 8U air-cooled HGX B300 system: 15 kW maximum node draw at full load, and 60 kW per four-node rack. They are one OEM's maximum figures for one system. A server under lighter load draws less, but the site has to be sized for the maximum.
Uptime Institute found in July 2026 that the average of modal (most typical) rack densities surpassed 11 kW, up from 9 kW in 2025. Excluding a few new high-density facilities, the average modal density is 7.8 kW. Most facilities still have no racks at 30 kW or above, though 24% now have some, up from 19% in 2025.
Cooling is the second constraint. In AFCOM's 2026 survey, 36% of respondents had liquid cooling deployed. Another 28% planned to adopt it within the next 12–24 months.
Redundancy is the third check. Uptime defines Tier III or Tier IV facilities by how they handle maintenance and failure. A Tier III facility is concurrently maintainable, with redundant components so equipment can be serviced without an outage, and Tier IV adds fault tolerance.
From the financing side, a deployment review confirms each item below before a purchase order. The same checklist applies to data center operators financing GPUs they sell compute from.
| Check | What to confirm |
|---|---|
| Power per rack | The site's rack power budget covers the servers' maximum draw |
| Cooling type | Air, direct liquid or facility water, matched to the server class |
| Rack space and weight | Free rack units, rack weight limits and floor loading |
| Network fabric | GPU-to-GPU and storage networking, cabling and switch ports |
| Colocation agreement | Signed space, power and term that match the deployment |
| Installation verification | Servers installed, burned in and verified online |
What a GPU Server Costs and How It Gets Financed
A complete 8-GPU NVIDIA B300 server costs about $600,000 at Q4 2026 pricing, or about $75,000 per GPU, according to our pricing research at USD.AI. That figure is server-only and pre-tax. Networking, racks, installation and tax add to a deployment's total cost.
Our GPU cluster cost article breaks those additions down, and our H100 rental price article covers renting instead of buying. Ownership economics also depend on how fast servers lose value, covered in our GPU depreciation article. Our GPU residual-value warranty article explains that structure for deals that include it.
Financings originated through USD.AI span all three server classes, as USD.AI's loan history shows. They include RTX PRO 6000 servers in Montréal in February and March 2026, and in Spring, Texas, in March 2026. In April 2026, a financing covered 576 NVIDIA B300 GPUs across 72 servers in Newport, Washington. A GB200 NVL72 facility in British Columbia followed in September 2026.
Conclusion
The server class sets the unit you buy, the cooling it needs and the site that can host it. Site power is the gate, since a 15 kW server has to land in a rack and facility built for it. At Q4 2026 pricing, a complete 8-GPU B300 server runs about $600,000 before networking, racks, installation and tax.
GPU financing through USD.AI covers new deployments and refinances GPUs already in service across these classes. Collateral sits in Tier III or Tier IV data centers. Every deal is scored by Compute IQ on the operator, the customer and the offtake contract. Loans cover up to 80% of deployment cost, with the operator contributing at least 20% as equity, over 3–5 year terms with straight-line monthly amortization.
In an illustrative example at Q4 2026 pricing, an operator buys 10 B300 servers (80 GPUs) for $6.0 million. A loan of up to 80% comes to $4.8 million, and the operator contributes at least $1.2 million in equity. On a 3-year straight-line schedule, monthly principal is about $133,000.
Operators can read more about financing for neocloud operators. Talk to USD.AI about your deployment and site plan.
FAQ
What is a GPU server used for?
A GPU server is used for AI training and inference, HPC (high-performance computing) and rendering. Its GPUs split these jobs into many small tasks and run them at the same time.
What is the difference between a GPU server and a CPU server?
A GPU server adds multiple GPUs for parallel processing, while a CPU server relies on CPUs built for sequential, general-purpose work. GPU servers also carry GPUs with high-bandwidth memory and draw far more power per server.
How much does a GPU server cost?
A complete 8-GPU NVIDIA B300 server costs about $600,000 at Q4 2026 pricing, server-only and pre-tax, based on USD.AI's pricing research. Networking, racks, installation and tax add to the total.
How much power does a GPU server need?
An 8-GPU server can need up to 15 kW, the full-load maximum Supermicro lists for its air-cooled 8U HGX B300 system. Uptime Institute puts the average of modal (most typical) rack densities just above 11 kW, so one such server can exceed a typical rack.
Can a normal data center host a GPU server?
It depends on the site's power per rack and its cooling. Most facilities have no racks at 30 kW or above, according to Uptime Institute. A 60 kW rack of four 8-GPU servers needs a site built for high density.
