(spainter_vfx/Shutterstock)
AI’s insatiable demand for knowledge has uncovered a rising drawback: storage infrastructure isn’t maintaining. From coaching basis fashions to working real-time inference, AI workloads require high-throughput, low-latency entry to huge quantities of information unfold throughout cloud, edge, and on-prem environments. Conventional storage techniques have typically struggled beneath the burden of those calls for, creating bottlenecks that may drastically delay innovation within the AI area.
As we speak, DDN unveiled Infinia 2.0, a major replace to its AI-focused, software-defined knowledge storage platform designed to eradicate the inefficiencies in AI storage and knowledge administration. The corporate says Infinia 2.0 acts as a unified, clever knowledge layer that dynamically optimizes AI workflows.
“Infinia 2.0 is not only an improve—it’s a paradigm shift in AI knowledge administration,” DDN CEO Alex Bouzari says, emphasizing how Infinia builds on the corporate’s deep-rooted experience in HPC storage to energy the following era of AI-driven knowledge companies.
A rendering of a large-scale Infinia 2.0 configuration from DDN’s Past Synthetic digital occasion.
As AI adoption grows, the challenges of scale, pace, and effectivity grow to be extra obvious. LLMs, generative AI functions, and inference techniques require not solely large datasets however the potential to entry and course of them sooner than ever. Conventional storage options wrestle with efficiency bottlenecks, making it tough for GPUs to obtain the info they want shortly sufficient, limiting total coaching effectivity. On the identical time, organizations should navigate the fragmentation of information throughout a number of areas, from structured databases to unstructured video and sensory knowledge. Transferring knowledge between these environments creates inefficiencies, driving up operational prices and creating latency points that sluggish AI functions.
DDN claims Infinia 2.0 solves these challenges by integrating real-time AI knowledge pipelines, dynamic metadata-driven automation, and multi-cloud unification, all optimized particularly for AI workloads. Moderately than forcing enterprises to work with disconnected knowledge lakes, Infinia 2.0 introduces a Information Ocean, a unified international view that eliminates redundant copies and allows organizations to course of and analyze their knowledge wherever it resides. This is supposed to cut back storage sprawl and to permit AI fashions to look and retrieve related knowledge extra effectively utilizing a sophisticated metadata tagging system. With nearly limitless metadata capabilities, AI functions can affiliate huge quantities of metadata with every object, making search and retrieval operations dramatically sooner.
Infinia 2.0 integrates with frameworks like TensorFlow and PyTorch, which the corporate says eliminates the necessity for complicated format conversions, permitting AI execution engines to work together with knowledge on to considerably pace up processing occasions. The platform can be designed for excessive scalability, supporting deployments that vary from a number of terabytes to exabytes of storage, making it versatile sufficient to satisfy the wants of each startups and enterprise-scale AI operations.
Efficiency is one other space the place Infinia 2.0 may very well be a breakthrough. The platform boasts 100x sooner metadata processing, decreasing lookup occasions from over ten milliseconds to lower than one. AI pipelines execute 25x sooner, whereas the system can deal with as much as 600,000 object lists per second, surpassing the restrictions of even AWS S3. By leveraging these capabilities, DDN asserts that AI-driven organizations can guarantee their fashions are educated, refined, and deployed with minimal lag and most effectivity.
(Supply: DDN)
Throughout a digital launch occasion right this moment referred to as Past Synthetic, DDN’s claims have been bolstered by sturdy endorsements from business leaders like Nvidia CEO Jensen Huang, who highlighted Infinia’s potential to redefine AI knowledge administration, emphasizing how metadata-driven architectures like Infinia remodel uncooked knowledge into actionable intelligence. Enterprise computing chief Lenovo additionally praised the platform, underscoring its potential to merge on-prem and cloud knowledge for extra environment friendly AI deployment.
Supermicro, one other DDN accomplice, additionally endorses Infinia: “At Supermicro, we’re proud to accomplice with DDN to rework how organizations leverage knowledge to drive enterprise success,” stated Charles Liang, founder, president, and CEO at Supermicro. “By combining Supermicro’s high-performance, energy-efficient {hardware} with DDN’s revolutionary Infinia platform, we empower clients to speed up AI workloads, maximize operational effectivity, and cut back prices. Infinia’s seamless knowledge unification throughout cloud, edge, and on-prem environments allows companies to make sooner, data-driven selections and obtain measurable outcomes, aligning completely with our dedication to delivering optimized, sustainable infrastructure options.”
On the Past Synthetic occasion, Bouzari and Huang sat down for a hearth chat to mirror on how a earlier thought, born from a 2017 assembly with Nvidia, developed into the Infinia platform.
DDN had been requested to assist construct a reference structure for AI computing, however Bouzari noticed a a lot greater alternative. If Huang’s imaginative and prescient for AI was going to materialize, the world would wish a basically new knowledge structure, one that would scale AI workloads, eradicate latency, and remodel uncooked info into actionable intelligence.
On the Past Synthetic occasion, Huang and Bouzari sit down for a hearth chat concerning the greater image of storage and AI.
Infinia is extra than simply storage, Bouzari says, and fuels AI techniques the way in which vitality fuels a mind. And in response to Huang, that distinction is essential.
“Probably the most essential issues individuals overlook is the significance of information that’s mandatory throughout utility, not simply throughout coaching,” Huang notes. “You wish to prepare on an unlimited quantity of information for pretraining, however throughout use, the AI has to entry info, and AI want to entry info, not in uncooked knowledge type, however in informational stream.”
This shift from conventional storage to AI-native knowledge intelligence has profound implications, the CEOs say. As a substitute of treating storage as a passive repository, DDN and Nvidia are turning it into an energetic layer of intelligence, enabling AI to retrieve insights immediately.
“That is the rationale why the reframing of storage of objects and uncooked knowledge into knowledge intelligence is that this new alternative for DDN, offering knowledge intelligence for the entire world’s enterprises as AIs run on high of this material of knowledge,” Huang says, calling it “a unprecedented reframing of computing and storage.”
Reframing definitely appears mandatory as AI continues to evolve as a result of the infrastructure supporting it should evolve as effectively. DDN’s Infinia 2.0 may symbolize a significant shift in how enterprises strategy AI storage, not as a passive archive, however as an energetic intelligence layer that fuels AI techniques in actual time. By eliminating conventional bottlenecks, unifying distributed knowledge, and integrating seamlessly with AI frameworks, Infinia 2.0 goals to reshape how AI functions entry, course of, and act on info.
With endorsements from business leaders like Nvidia, Supermicro, and Lenovo, and with its newest funding spherical of $300 million at a $5 billion valuation, DDN is positioning itself as a key participant within the AI panorama. Whether or not Infinia 2.0 delivers on its formidable guarantees stays to be seen, however one factor is obvious: AI’s subsequent frontier isn’t nearly fashions and compute however is about rethinking knowledge itself. And with this launch, DDN is making the case that the way forward for AI hinges on new paradigms for knowledge administration.
Be taught extra concerning the technical points of Infinia 2.0 at this hyperlink, or watch a replay of Past Synthetic right here.
Associated Objects:
Feeding the Virtuous Cycle of Discovery: HPC, Massive Information, and AI Acceleration
The AI Information Cycle: Understanding the Optimum Storage Combine for AI Workloads at Scale
DDN Cranks the Information Throughput with AI400X2 Turbo
Editor’s be aware: This text first appeared on AIWire.