SF Compute: Commoditizing Compute

Evan Conrad, co-founder of SF Compute, joined us to talk about how they started as an AI lab that avoided bankruptcy by selling GPU clusters, why CoreWeave financials look like a real estate business, and how GPUs are turning into a commodities market.

Chapters:

00:00:05 - Introductions

00:00:12 - Introduction of guest Evan Conrad from SF Compute

00:00:12 - CoreWeave Business Model Discussion

00:05:37 - CoreWeave as a Real Estate Business

00:08:59 - Interest Rate Risk and GPU Market Strategy Framework

00:16:33 - Why Together and DigitalOcean will lose money on their clusters

00:20:37 - SF Compute's AI Lab Origins

00:25:49 - Utilization Rates and Benefits of SF Compute Market Model

00:30:00 - H100 GPU Glut, Supply Chain Issues, and Future Demand Forecast

00:34:00 - P2P GPU networks

00:36:50 - Customer stories

00:38:23 - VC-Provided GPU Clusters and Credit Risk Arbitrage

00:41:58 - Market Pricing Dynamics and Preemptible GPU Pricing Model

00:48:00 - Future Plans for Financialization?

00:52:59 - Cluster auditing and quality control

00:58:00 - Futures Contracts for GPUs

01:01:20 - Branding and Aesthetic Choices Behind SF Compute

01:06:30 - Lessons from Previous Startups

01:09:07 - Hiring at SF Compute

Chapters

• 00:00:00 Introduction and Background
• 00:00:58 Analysis of GPU Business Models
• 00:01:53 Challenges with GPU Pricing
• 00:02:48 Revenue and Scaling with GPUs
• 00:03:46 Customer Sensitivity to GPU Pricing
• 00:04:44 Core Weave's Business Strategy
• 00:05:41 Core Weave's Market Perception
• 00:06:40 Hyperscalers and GPU Market Dynamics
• 00:07:37 Financial Strategies for GPU Sales
• 00:08:35 Interest Rates and GPU Market Risks
• 00:09:30 Optimal GPU Contract Strategies
• 00:10:27 Risks in GPU Market Contracts
• 00:11:25 Price Sensitivity and Market Competition
• 00:12:21 Market Dynamics and GPU Contracts
• 00:13:18 Hyperscalers and GPU Market Strategies
• 00:14:15 Nvidia and Market Competition
• 00:15:12 Microsoft's Role in GPU Market
• 00:16:10 Challenges in GPU Market Dynamics
• 00:17:07 Economic Realities of the GPU Market
• 00:18:03 Real Estate Model for GPU Clouds
• 00:18:59 Price Sensitivity and Chip Design
• 00:19:55 SF Compute's Beginnings and Challenges
• 00:20:54 Navigating the GPU Market
• 00:21:54 Pivoting to a GPU Cloud Provider
• 00:22:53 Building a GPU Market
• 00:23:52 SF Compute as a GPU Marketplace
• 00:24:49 Market Liquidity and GPU Pricing
• 00:25:47 Utilization Rates in GPU Markets
• 00:26:44 Brokerage and Market Flexibility
• 00:27:42 H100 Glut and Market Cycles
• 00:28:40 Supply Chain Challenges and GPU Glut
• 00:29:35 Future Predictions for the GPU Market
• 00:30:33 Speculations on Test Time Inference
• 00:31:29 Market Demand and Test Time Inference
• 00:32:26 Open Source vs. Closed AI Demand
• 00:33:24 Future of Inference Demand
• 00:34:24 Peer-to-Peer GPU Markets
• 00:35:17 Decentralized GPU Market Skepticism
• 00:36:15 Redesigning Architectures for New Markets
• 00:37:14 Supporting Grad Students and Startups
• 00:38:11 Successful Startups Using SF Compute
• 00:39:11 VCs and GPU Infrastructure
• 00:40:09 VCs as GPU Credit Transformators
• 00:41:06 Market Timing and GPU Infrastructure
• 00:42:02 Understanding GPU Pricing Dynamics
• 00:43:01 Market Pricing and Preemptible Compute
• 00:43:55 Price Volatility and Market Optimization
• 00:44:52 Customizing Compute Contracts
• 00:45:50 Creating Flexible Compute Guarantees
• 00:46:45 Financialization of GPU Markets
• 00:47:44 Building a Spot Market for GPUs
• 00:48:40 Auditing and Standardizing Clusters
• 00:49:40 Ensuring Cluster Reliability
• 00:50:36 Active Monitoring and Refunds
• 00:51:33 Automating Customer Refunds
• 00:52:33 Challenges in Cluster Maintenance
• 00:53:29 Remote Cluster Management
• 00:54:29 Standardizing Compute Contracts
• 00:55:28 Unified Infrastructure for Clusters
• 00:56:24 Creating a Commodity Market for GPUs
• 00:57:22 Futures Market and Risk Management
• 00:58:18 Reducing Risk with GPU Futures
• 00:59:14 Stabilizing the GPU Market
• 01:00:10 SF Compute's Anti-Hype Approach
• 01:01:07 Calm Branding and Expectations
• 01:02:07 Promoting San Francisco's Beauty
• 01:03:03 Design Philosophy at SF Compute
• 01:04:02 Artistic Influence on Branding
• 01:05:00 Past Projects and Burnout
• 01:05:59 Challenges in Building an Email Client
• 01:06:57 Persistence and Iteration in Startups
• 01:07:57 Email Market Challenges
• 01:08:53 SF Compute Job Opportunities
• 01:09:53 Hiring for Systems Engineering
• 01:10:50 Financial Systems Engineering Role
• 01:11:50 Conclusion and Farewell