GPU Supply Shortfall Obstructing AI Advancements, Asserts Tory Green

57

Tory Green, Chief Operating Officer of io.net, a decentralized GPU cloud service, has indicated that the gap between GPU supply and demand is becoming a critical barrier to progress in artificial intelligence (AI). Green suggests that the crux of the matter is the inability of manufacturers to scale up production promptly in response to the soaring demand.

The Disparity Between GPU Availability and Need

Green sheds light on the severity of this imbalance by noting that the computational requirements for machine learning (ML) training have been increasing at a rate of 10x every year and a half since 2010. However, during the same timeframe, the actual growth in computational capacity has only doubled, a discrepancy he attributes to the considerable time required to create new supply chains.

To mitigate the detrimental effects of GPU shortages on the advancement of AI, Green proposes the adoption of a decentralized physical infrastructure network (DePIN). He explains that such a network would distribute physical computing resources over various locations, owned by different stakeholders.

In contrast with conventional cloud services, which are constrained by geographical and physical limitations, a decentralized network has the potential for near-limitless scalability by leveraging the collective capabilities of a worldwide network of nodes, Green states.

When addressing the potential for scarcity within DePINs, Green, through written correspondence to CryptokenTop.com News, underscores how geographical dispersion and collaborative efforts can prevent such dilemmas.

Here are Tory Green’s complete responses to the inquiries posed.

CryptokenTop.com News (BCN): With the GPU market leader Nvidia reaching a market valuation of $1 trillion due to heightened GPU demand, could you discuss the factors driving this demand-supply gap and its impact on AI innovation?

Tory Green (TG): The advent of AI has drastically escalated the need for GPU computational prowess, particularly as deep learning and transformer models necessitate substantial processing capability for their training, refinement, and application.

These models become increasingly intricate, thus amplifying the demand. For instance, the computational requirements for ML training have surged by 10x every 18 months since 2010, whereas computing power has only seen a twofold increase.

Manufacturers are unable to rapidly adjust to this surge due to extended lead times for developing new supply. This scenario creates several issues for engineers, such as:

• Long wait times: Access to GPUs through cloud services can take weeks.
• Limited choice: Options regarding GPU hardware, location, and security level are constrained.
• High costs: Acquiring quality GPUs comes at a premium, as demonstrated by OpenAI’s daily expenditure of $700K to operate ChatGPT.

These factors collectively hinder the pace of AI innovation.

BCN: Who, aside from traditional centralized cloud services, possesses substantial GPU compute capacity in both the Web2 and Web3 sectors?

TG: A significant portion of GPU resources is located outside major cloud service providers. Notable holders of GPU compute capacity include:

• Data centers: The U.S. has thousands of independent data centers with an average use rate between 12% and 18%.
• Crypto miners: With Ethereum’s migration to Proof-of-Stake, miners are exploring alternative uses for their GPU resources.
• Consumer GPUs: Although they constitute 90% of the total supply, a majority are dormant in personal homes.

By harnessing these disparate sources, io.net aims to deliver extensive computing power at a fraction of the cost associated with traditional cloud services.

BCN: How does a decentralized physical infrastructure network (DePIN) alleviate compute power scarcity for startups venturing into AI?

TG: A DePIN decentralizes physical computing resources across numerous locales and owners, in contrast to centralization.

This decentralized approach enables a nearly infinite scaling capability by integrating the collective strength of a global node network, as opposed to traditional clouds bound by physical and geographical limits.

BCN: Given that your firm io.net is set to enhance GPU computation accessibility and flexibility, how do you position yourselves in relation to established entities like AWS and Azure?

TG: A DePIN model inherently provides several benefits:

• Robust Computing Power: A decentralized network merges the computing power of all nodes, increasing with each new addition, unlike traditional clouds which are geographically and physically limited.
• Cost-Effectiveness: DePINs can reduce expenses by cutting out intermediaries and optimizing resource allocation, whereas traditional clouds levy substantial fees.
• Scalability: DePINs can expand organically as more resources are contributed, whereas scaling in cloud services involves complex processes.
• Enhanced Security and Reliability: The distributed nature of a DePIN provides better defense against attacks and inherent redundancy, as opposed to centralized clouds that offer a single point of failure.
• Unrestricted Access: DePINs offer permissionless access, which starkly contrasts with the contract-bound and KYC-heavy requirements of services like AWS.

Moreover, DePINs like io.net allow for the immediate creation of distributed clusters, a process that can take weeks with traditional cloud services. Most critically, decentralized networks have the potential to disrupt the oligopoly of the current cloud market, expanding supply and dramatically lowering consumer costs while increasing profits for suppliers.

BCN: As the infrastructure of Web3 is still developing, do you foresee DePIN and similar AI-aligned technologies gaining mainstream traction?

TG: Indeed, DePINs present a promising technology that has yet to achieve widespread adoption, primarily due to limited demand. Nonetheless, a DePIN dedicated to AI addresses a pressing need by resolving GPU shortages and offering a superior customer experience at reduced costs. Additionally, it enables resource owners to enhance their profitability. AI may very well provide the catalyst for DePINs to achieve mass-market appeal.

BCN: With the current GPU shortage being a matter of demand surpassing supply, could a similar situation arise with DePIN, and how might it be addressed?

TG: The prospects of DePINs facing a comparable scarcity are slim, for several reasons:

• Varied Hardware Contributions: A DePIN can source computing power from a wide range of hardware, as opposed to the centralized dominance of a few players in the current market.
• Resource Optimization: DePIN systems are inherently designed to make optimal use of network resources, ensuring efficient use of computing power.
• Natural Scalability: Increasing demand within a DePIN encourages more participants to contribute resources, facilitating natural infrastructure growth.
• Global Distribution: The decentralization of a DePIN ensures worldwide engagement, which can mitigate regional shortages or disruptions in the supply chain.
• Community-driven Strategies: Decentralized communities are capable of developing innovative approaches to managing resource scarcity, unlike traditional systems that rely heavily on regulation and bureaucratic control.

In essence, the inherently decentralized and diversified structure of a DePIN naturally precludes the supply constraints and bottleneck issues encountered in the prevailing centralized systems.

Share your thoughts on this interview in the comments section below.

Frequently Asked Questions (FAQs) about Decentralized GPU Cloud

More about Decentralized GPU Cloud

• GPU Market Analysis
• Decentralized Cloud Services
• Impact of GPU Shortages on AI
• DePIN Technology Explained
• Challenges and Solutions for AI Startups
• Interview with Tory Green
• Web3 Infrastructure Development
• Future of AI and Decentralization