The Deload Podcast interviews founders and builders of frontier technology companies that are transforming how we live. The mission of the podcast expands that of The Deload: To educate you about where the world is going to make smarter growth investments.
If you haven’t subscribed, join more than 1,500 rational optimists:
The State of GPU Demand
Aaron Ginn is the CEO and co-founder of Hydra Host.
Hydra offers a software platform for data centers that helps them manage inventory and make it available to customers. Dozens of data centers use their platform.
Aaron posted a great recap of the conversation on Twitter that I highly recommend reading.
Here are the key notes from the conversation with Aaron:
Hydra is in the business of UX, just like Tesla and Apple. Data centers are in the business of hardware and real estate. They need help with UX. Hydra provides it, enabling data center access as easy as AWS + the flexibility of bare metal.
Running on cloud infrastructure is constant state of unhappiness where ease of use is nice, but it comes with limits on security and cost. Hydra tries to solve that problem by bringing simple software solutions that help address some of the cost and security.
Energy requirements of GPUs have increased linearly. This pushes bespoke data center creation for facilities that are optimized for GPUs. For example, Scandinavian countries can be optimal places for data centers given lower energy costs, alternative energy sources, and cooling. NVIDIA is trying to spread itself out globally, too, so bespoke data centers are finding access to high-end chips.
NVIDIA’s marketing is so powerful pushing the high end chips, it’s in a sense driving excess customer demand. There’s a movement with many companies wanting A100 or H100 chips when they could just use A10s at a much lower cost. Aaron uses the analogy of a Volkswagen (A10/A40) vs an Audi (A100) vs a Lamborghini (H100). People underestimate what a Volkswagen can do.
Demand for A100s has been particularly high because NVIDIA isn’t making them anymore and they’re about 50% the cost to run as an H100. In some places, A100s are getting to be as expensive to access as H100s. Starting to see some give up on A100s and just going down market to Volkswagens per the prior analogy because they don’t need the power.
Aaron doesn’t believe AI is a product, it’s a feature of something else. To run a feature, you generally run it in your own data center. If it’s something that’s adding critical value over time, you don’t want it run offsite. You want to own the IP and interaction points to refine the AI over time. That’s the only way it makes sense.
Aaron believes in making money from day one. Someone is willing to pay me to do something. Is that something I’m willing to do? That’s a better question than just hitting a ball and trying to see if it’s a home run.
Follow Aaron on:
Follow Hydra on:
Episode Time Stamps
1:30 What Hydra does: Airbnb for co-location/co-location on demand.
3:50 Why data centers need Hydra.
6:50 CPU vs GPU pricing and use cases.
7:50 Importance of location for data centers + GPUs.
10:00 NVIDIA trying to spread out its footprint.
14:00 Customers think they need higher-end chips than they really do.
16:45 The state of GPU demand.
17:30 AI as a product vs feature.
21:30 The return to self-managed infrastructure.
25:00 Hydra’s focus on high-performance compute vs commodity.
27:50 The auto analogy for GPUs.
31:50 Aaron’s unique philosophies for company building.
Disclaimer: My views here do not constitute investment advice. They are for educational purposes only. My firm, Deepwater Asset Management, may hold positions in securities I write about. See our full disclaimer.
The State of GPU Demand with Hydra Host CEO Aaron Ginn