The current discourse on AI largely centers around expanding cloud capacity and building large data centers for running models. Companies like Apple and Qualcomm are beginning to focus on making on-device AI more efficient. In this context, the 14-member technical team at London-based Mirai is dedicated to enhancing model performance on phones and laptops.
Established by Dima Shvets and Alexey Moiseenkov, Mirai has secured $10 million in seed funding led by Uncork Capital. Both founders bring expertise in developing scalable consumer applications. Shvets co-founded the face-swapping app Reface, supported by a16z, and later became a scout for the venture firm. Moiseenkov was the CEO and co-founder of Prisma, a popular AI filters app.
Both founders have long considered AI and machine learning on devices, even before the rise of generative AI, according to Shvets.
“When we got together in London, we began discussing technology and realized that amidst the hype of gen AI and increased AI adoption, the focus was on cloud and servers. However, on-device AI for consumer hardware was a missing aspect,” Shvets told TechCrunch.
Shvets and Moiseenkov aimed to use AI to streamline complex tasks on phones, leading them to establish Mirai. Conversations with other consumer app developers revealed a common desire for improved cost optimization and better margins per token usage.
Currently, Mirai is building a framework to enhance model performance on devices. The company has created an inference engine for Apple Silicon to optimize on-device throughput. Their upcoming SDK will allow developers to integrate the runtime with minimal code.
“Our vision was to provide developers with a seamless integration experience, much like Stripe. You visit our platform, input the key, and begin using the features like summarization or classification, depending on your needs,” Shvets explained.
The startup developed this engine in Rust, resulting in models generating content 37% faster, they claim. They assert that while optimizing models for specific platforms, they maintain the quality of the output by not modifying model weights.
Mirai’s current focus is on improving text and voice modalities, with plans to incorporate vision in the future. They are collaborating with frontier model providers to tune models for edge use and engaging in discussions with various chipmakers. The engine is also intended to be adapted for Android platforms later.
Additionally, Mirai plans to introduce on-device benchmarks for evaluating model performance. Shvets acknowledges that not all AI tasks can rely on on-device processing. To facilitate mixed operations, an orchestration layer is being developed to manage tasks that require cloud processing.
While Mirai isn’t directly partnering with apps presently, their engine has the potential to power on-device assistants, transcribers, translators, and chat applications.
Andy McLoughlin, managing partner at Uncork Capital, noted that he previously invested in an edge machine learning company that was eventually sold to Spotify. He suggests that market dynamics have changed since then.
“The cost of cloud inference necessitates a shift. Currently, VCs are willing to fund expansive cloud inference endeavors. However, focus will eventually turn to the business economics, necessitating change,” he said. “Model creators will likely want to conduct some inference workloads at the edge, and Mirai is well-positioned to meet this demand.”
Mirai’s seed funding round also attracted participation from various notable individuals, including Dreamer CEO David Singleton, YC Partner Francois Chaubard, Snowflake co-founder Marcin Żukowski, ElevenLabs co-founder Mati Staniszewski, former Google AdSense product manager and Coinbase board member Gokul Rajaram, investor Scooter Braun, Turing.com CTO Vijay Krishnan, and Theory Forge Ventures’ Ben Parr and Matt Schlicht, as well as ex-Netflix technical leader Aditya Jami.