1

BIZON G8000 – 8-10 TITAN RTX rackmount GPU server with liquid cooling for deep learning. Review, benchmarks, noise level, temperatures.

Introduction



Most of you guys have seen tons of liquid-cooled builds from top YouTubers like Linus Tech Tips or JayzTwoCents and you may think nothing will shock you anymore and you have already seen the craziest liquid cooling builds ever.

Until now.

Today we will show you something that will blow your mind and this thing is sick!

We are introducing worlds first dual Xeon RACKMOUNT SERVER with 8 x TITAN RTX GPUs with the custom liquid cooling system!

8 TITAN RTX deep learning server
And this is the craziest and most expensive liquid cooled computer you have ever seen.


Problem


Before we start, let me tell what brings us to this crazy project.

Most of the people may think that liquid cooling is something made for gamers only. It just looks great because of the look of the liquid, that’s why gamers like it.

it’s true when you are dealing with one or two GPUs. The amount of heat dissipated by the GPUs will be reasonable for air cooling.

But what about multi-GPU configurations? 4, 8 or more GPUs?

We have 4 GPU liquid-cooled desktop workstation. More details »

But today’s article is about 8-10 GPU rackmount server.


The main question people ask is why liquid cooling is so expensive and why I have to overpay for it. We are here to answer you.

There are three main benefits of liquid cooling.

1. Low noise.

2. Low thermals.

3. Higher performance.

The latest generation of NVIDIA RTX GPUs in multi GPU configurations dissipate so much heat that you need liquid cooling to keep GPU within optimal temperature range.

Blower fans solve the problem for RTX 2080 Ti, but there is no blower fan for TITAN RTX.

TITAN RTX offers the best performance for reasonable price ($2500). TITAN RTX performance comparable to NVIDIA Tesla V100 ($7000).

Sounds good right? Then why buying Tesla V100 for that crazy money instead if using TITAN RTX and build the fastest GPU server in the world for reasonable price?

Let’s build the server with Titan and see what happens.

I am installing the GPUs, and technically they fit. But there a huge problem we are going to show you.

NVIDIA TITAN RTX’s cooling system design block them from using in multi-GPU configurations. As you can see on the photo, when you install one Titan next to another, the fans are completely blocked. No clearance between them. I am pretty sure the GPU will not be able to dissipate the heat.

titan clearance
Let’s run the benchmarks and see what happens.



Benchmarks


Test Bench:
Air cooling build:
4 x TITAN RTX

Liquid cooling:
BIZON G8000
GPUs: 4 x TITAN RTX + 4 x RTX 2080 Ti
Note: We will activate 4 TITAN RTX only when running the benchmarks to match the air cooling build.


Deep Learning Benchmark – Air cooling


Our server have two settings for the fan speed. Let’s start from normal fan speed.

Benchmark specs:
  • Model: Resnet 50
  • Tensorflow 1.13
  • Data: Synthetic
  • 1000 iterations
  • Batch size: 64
  • FP32


Normal fan speed:

Temperature test:
8 TITAN RTX deep learning server - air cooling - benchmark 1

Performance:
8 TITAN RTX deep learning server - air cooling - performance

Time:
8 TITAN RTX deep learning server - air cooling - noise



Max fan speed:

Temperature test:
8 TITAN RTX deep learning server - air cooling max fan - temperature

Performance:
8 TITAN RTX deep learning server - air cooling max fan - performance

Noise level:
8 TITAN RTX deep learning server - air cooling max fan - noise


Results:

Temperature:
As I have expected after a few minutes, temperatures immediately goes up to critical 85C-90C.
This is the “red line” for the GPUs. Once the GPUs reach that level, they automatically activate thermal throttling to protect itself from damages.

Thermal throttling:
When your cooling solution can no longer dissipate heat fast enough to keep temperatures within a safe range, your graphics card starts to dump performance to keep the temperatures low.


Performance:
On the normal fan speed we can see up to 5X lower performance due to overheating.
Even on maximum fan speed we can see up to 3X lower performance due to overheating.

Moreover, when the GPU is permanently working under high temperatures, it will damage the GPUs and significantly decrease the lifespan.


Noise level:
Noise level at normal fan speed is acceptable.

At max fan speed mode we have seen 91 db noise level and this is totally unacceptable. Noise level above 85 dB are harmful. You need to wear sound reduction ear muffs for construction workers to stand near the server. Even if you think to place the server in the server room, you will hear it from outside of the server room pretty clear and coworkers will complain for sure.

Note:
Sound is measured in decibels (dB). A whisper is about 30 dB, normal conversation is about 60 dB, and a motorcycle engine running is about 95 dB. Noise above 85 dB over may start to damage your hearing. Loud noise above 120 dB can cause immediate harm to your ears.

The National Institute for Occupational Safety and Health (NIOSH) in the US recommends that the equivalent noise level that a worker is exposed to should be limited to 85 dB(A) for an eight-hour day to minimize the risk of hearing damage.
Source: https://www.osha.gov/SLTC/noisehearingconservation/loud.html



The server comes with 8 x strong 8000 RPM server fans, and they are not enough to keep the TITANs cool. Because there is just no clearance between the GPUs. And that’s where liquid cooling became a must.

Despite those facts, we can see a few companies on the market selling TITAN RTX servers without liquid cooling.

Now let’s compare with liquid cooling.


Deep Learning Benchmark – Liquid cooling


Idle Temperature (8 GPUs: 4 x TITAN RTX + 4 x RTX 2080 Ti):
8 TITAN RTX deep learning server - liquid cooling - performance
Temperature test:
8 TITAN RTX deep learning server - liquid cooling - temperature

Performance:
8 TITAN RTX deep learning server - liquid cooling - performance

Noise level:
8 TITAN RTX deep learning server - liquid cooling - noise level

Time:
8 TITAN RTX deep learning server - liquid cooling - time
Look at these temperatures!
46C-50C is just perfect! This is really low for any GPU.

Noise level is within the normal range and we can talk with no problem.

We can even place this server in the office, lab or living room.

No thermal throttling. No overheating. 100% stable performance.

Performance is very stable (we tested for 48 hours).




Rendering (Luxmark) – Air cooled server


LuxMark is a OpenCL cross-platform benchmark tool.

Temperature test:
8 TITAN RTX deep learning server - air cooling - temperature

Score:
8 TITAN RTX deep learning server - air cooling - luxmark score


Rendering (Luxmark) – BIZON G8000 Liquid cooled server


4 x TITAN RTX:

Temperature test:
8 TITAN RTX deep learning server - liquid cooling - temperature

Score:
8 TITAN RTX deep learning server - liquid cooling - luxmark score


4 x TITAN RTX + 4 x RTX 2080 Ti:

Temperature test:
8 TITAN RTX deep learning server - liquid cooling - temperature

Score:
8 TITAN RTX deep learning server - liquid cooling - luxmark score



Benchmarks – Conclusion


Liquid cooled NVIDIA TITAN RTX:

  • Idle noise level: up to 40 db
  • Idle temperature: 27-29C
  • Max load noise level (stress test): up to 50 db
  • Max load temperature (GPUs): 45-55C
  • Performance: 100%. No overheating –> No performance drop. 24/7 stable.


Air cooled NVIDIA TITAN RTX (normal fan speed):

  • Idle noise level: up to 70 db
  • Idle temperature: 40C
  • Max load noise level (deep learning benchmark): up to 80 db
  • Max load temperature (GPUs): 86-88C
  • Performance (deep learning): 6X times lower (150 vs. 900)
  • Performance (rendering / luxmark): 2X times lower (92000 vs. 190000)
  • Warning! GPUs are overheating and dump down the performance up to 6X times.

Air cooled NVIDIA TITAN RTX (max fan speed):

  • Idle noise level: 92 db (warning!)
  • Idle temperature: 40C
  • Max load noise level (deep learning benchmark): 92 db (warning!)
  • Max load temperature (GPUs): 86-88C
  • Performance (deep learning): 3X times lower (300 vs. 900)
  • Performance (rendering / luxmark): 2X times lower (92000 vs. 190000)
  • Warning! GPUs are overheating and dump down the performance up to 3X times.






But can it mine? Mining – BIZON G8000 Liquid cooled server


Yes!

4 x TITAN RTX + 4 x RTX 2080 Ti:

Temperature test:
8 TITAN RTX deep learning server - liquid cooling - mining temperature

Hashrate:
8 TITAN RTX deep learning server - liquid cooling - mining hashrate





Conclusion


There are various use cases for the 8-10 GPU server:
AI research, machine learning, various computation tasks, speech and image recognition, computer vision, rendering, or even gaming.

The benchmarks of BIZON G8000 liquid cooled server with 8 TITANs are much better than we may expect.

Top of the line GPUs like TITAN RTXes are extremely hot and require liquid cooling in multi-GPU configurations.

Liquid cooling is the best solution for rapid heat removal due to its unmatched thermal performance. By using liquid cooling we get rid of the loud fans. The liquid cooling will provide 24/7 stability, low noise, and long-life for components.

Noise level of BIZON liquid cooling system is up to 50% lower versus air cooling: 50 dB vs. 90 dB

Temperatures are up to 50% lower as well: 50C vs 90C.

Performance is up to 3 times faster with liquid cooling versus air cooling. GPUs do not overheat and do not activate thermal throttling.

Your investment in the liquid cooling system will pay for itself in a few months.

It’s been an incredible project and BIZON team did a great job.

BIZON liquid cooled experts and engineers spend months for research, development, and building the system and custom components. We have tested hundreds of different configurations, components, materials, coolants, pumps, radiators to find the optimal solution to provide the lowest noise level, temperatures and performance.

Since its a commercial solution there is two liquid cooling systems for better stability.

Each CPU and 4 GPUs are handled by its dedicated radiator and pump.
That’s why you can see two colors.

Liquid cooling loop includes cooling for dual CPUs and all the graphics cards. So the server is perfect not just for GPU related use cases but for CPU use cases as well. Moreover it can handle GPU and CPU tasks at the same time.

Unlike most commercial solution that looks ugly and boring,
this beast looks great. It’s not just a regular piece of black metal. It’s a piece of art. It’s like a Bugatti in liquid cooled world!

For more details visit BIZON G8000 product page. You can place the order online and customize the system.

Thank you for reading!