Big Tech giants like OpenAI and Google might dominate the artificial intelligence industry with their industry-leading, multibillion-parameter large language models, but Decart aims to demonstrate there’s still plenty of scope for smaller firms to establish themselves as players in other use cases for AI.
The Silicon Valley-based unicorn startup, which is laser-focused on real-time video generation models, believes it has hit upon a winning combination. This week, the company has announced the pairing of its novel model architecture with cutting-edge computing infrastructure from Amazon Web Services to offer scalable generative AI video solutions that crush larger competitors with lower latency and superior performance.
In an announcement released this week ahead of a presentation at AWS’s annual re:Invent showcase, Decart says it has agreed a deal to run its most powerful video generation model, called Lucy, on Amazon’s most advanced AI processors. It’s leveraging the custom-designed AWS Trainium3 processors alongside Amazon’s cloud services to offer enterprises access to a full-stack video generation platform that delivers unrivaled low-latency and high throughput.
Decart’s announcement says that the company has been working for months to optimize Lucy to run on AWS’s Trainium processor chips, beginning with Trainium2 before undertaking the process of optimizing for Trainium3, the release of which was announced at re:Invent.
Decart’s leadership chose Amazon’s infrastructure because the team believes AWS can achieve better performance for video generation than Nvidia’s general-purpose GPUs. Trainium is a specialized type of processor that’s customized to run AI training and inference workloads, thanks in part to its systolic array design that enhances matrix operations.
“Before the breakthrough, we could only do it for 1.5 to 2 seconds before it turned to
noise on the screen,” Decart CEO Dean Leitersdorf told the Wall Street Journal. “The models would get too confused and everything would blur into oblivion…. Trainium really allowed us to run a bigger model which is smarter and doesn’t crash quickly.”
Disrupting the big boys
Decart’s partnership with Amazon is just the latest example of how startups are looking to compete with more established players in the AI industry by targeting verticals. This partnership demonstrates that it’s possible to disrupt the industry by developing bespoke models that solve highly specific problems.
For instance, AiHello is a small, 40-person startup launched in 2023 developed an AI advertising platform that’s focused exclusively on Amazon’s ad platform. Despite not raising any venture capital, it’s already pulling in seven-figure annual revenues, with its sales reportedly doubling each year. In an interview with Forbes, AiHello CEO Saif Elhager explained that the company has been laser-focused on a very specific problem from day one, rather than trying to compete with the vast resources of Meta, Microsoft, OpenAI, Google and others.
There are many other AI startups focused on solving niche challenges. SoundHound AI has established itself as a major player in the voice AI segment, creating a series of voice models widely regarded for their flexibility and superior accuracy.
While generative AI video is not a niche use case, generating AI videos in real time can be considered one for now, for it’s something that only a handful of companies have even tried, and the results have been somewhat questionable so far. The challenge stems from the fact that real-time video generation requires a performance trade-off to attain the extremely low latencies required to create instant outputs, and that inevitably means that graphics quality has to be sacrificed.
But Decart says Amazon’s highly-performant, customized Trainium infrastructure and its decision to build a streamlined model architecture focused solely on video allows it to avoid making this sacrifice. A recent Decart demo at Twitchcon certainly made the case that everyday content creators can now do a lot of interesting things with real-time AI video, and companies in game development, extended reality and entertainment media are taking note.
Trainium makes it possible
Decart said the performance boost it gets from Amazon is so significant that it can establish real-time generative AI video as a distinct market segment that enables various new applications in industries such as gaming and social media.
Lucy has already demonstrated impressive capabilities while running on AWS Trainium2, generating video outputs at up to 40 frames per second within milliseconds, but Decart’s leadership believes its models can do a lot better than this. Decart is now working to optimize its models for the new Trainium3 processors unveiled at re:Invent this week, with the goal of delivering real-time AI video at up to 100 FPS, a significant jump in quality.
Customers can access Lucy on AWS Trainium as of this week, with the model being distributed through the Amazon Bedrock platform, meaning developers can easily plug it into their existing or new applications. It promises what is essentially instantaneous AI video generation with unparalleled visual fidelity, paving the way for developers to easily integrate real-time AI-generated video experiences into third-party products.
In a statement, Leitersdorf said generative AI video is one of the toughest challenges of all in AI, due to its instance computation demands. And it gets even tougher when latency has the potential to ruin the viewer’s experience. That’s why he’s so enamored with AWS Trainium, he said.
“We’re making real-time video generation practical and cost-effective at scale,” he added. “Together with AWS, we’re enabling customers to create and stream new forms of interactive content, from personalized live experiences to large-scale simulations, faster and more efficiently than ever before.”
