Unify (YC W23) - Building LLM Router with Daniel Lenton
Make Your LLM API Calls Smarter
For more detailed discussion, listen to our podcast on Spotify: Unify (YC W23) - Building LLM Router with Daniel Lenton
Also on YouTube: Unify (YC W23) - Building LLM Router with Daniel Lenton
Don’t forget to subscribe to us on Spotify and YouTube
💡 Company Info:
Founder & CEO: Daniel Lenton, PhD in Deep Learning Dense Spatial Perception for Robotic Manipulation, Imperial College London
Founded in: 2022
Website: https://unify.ai
Part of YC W23 Batch
Combine All Models for Faster, Cheaper, and Better Responses Than Any Single Model
Keywords: #LLM #LLMRouter #Development
⏳ Beyond Accuracy: Optimizing Inference Speed/Cost/Other Metrics
Back in 2022/23, the main buzz on the street was about getting the best LLM out there and useing it to create new features. Then there’s a blossoming of different LLMs from big tech and startups alike. The attention (no pun intended) shifts from getting the model with the best metrics to the model that can actually support the use case economically.
Unify helps users create their own LLM router that can be trained to fit the typical inputs. Users can choose from a basket of models, and have the router optimize the inference, whether to reduce cost, increase time to first token, or a combination of different thresholds.
The “best” model is subjective for each use case, and even for each API call. Developers want this flexibility across models and cloud providers.
💬 Talking to Users: Lessons from Y Combinator as Deep Tech Founder
I received this email from Daniel just as I signed up for a trial account before their public release: an invitation to a 15-minute quick chat for feedback. It’s one of the most proactive experiences I’ve had with startups building MVP.
For Daniel, one of the standout lessons from YC was the importance of constantly engaging with users. Daniel admitted that in the early days, his team was so focused on building a sophisticated product that they neglected to validate it with real users. YC’s mantra of launching a Minimum Viable Product (MVP) quickly and iterating based on user feedback was a gamechanger. They even have a Discord community for users to contribute to the ecosystem.
What he learned from YC:
Launch incredibly quickly with the simplest version
Get feedback on problems, not feature suggestions
“I would say is that I was incredibly naive and a little bit, um, probably arrogant […] like I kind of knew it all anyway, which I didn't, um, we were kind of doing a deep tech thing.”
Before pivoting, they were “doing this in a silo without constantly talking to users, and it turned out that that was as YC would predict, like not the way to build things.”
This reminds me of a video from YC where Michael Seibel mentioned founders saying that the “standard to get a beta released at Google is 10x the MVP”. A lot of founders may have need to learn to trade technical excellence for being fast and scrappy.
🔄 Pivoting: From Ivy to Unify
Unify started by building Ivy, a transpiler and framework that allows people to try out new models across different ML frameworks. They had to shrink the team from 25 to seven. Daniel mentioned his immense pressure along the journey, to assume responsibility for it and not disappoint the people who trusted him.
“We weren't really like checking if this was like exactly what they wanted. We were kind of just like had this vision of like the perfect way of unifying all of AI and like, this was our mission.”
Coming out of the pivot, their product may be less technical than Ivy, but more focused on gathering feedback and building something that unlocks value in the AI ecosystem.
🌅 The Future
We’re entering a new era where the initial excitement about what AI can do is worn off. Now, the focus is shifting towards making the most out of these technologies in practical, tangible ways. Some of the large corporations that Daniel spoke to, are still trying to figure out how to effectively integrate LLM into their infrastructure.
In a world where LLMs are getting increasingly commoditized, developers feel the need for orchestration and interoperability between models. As developers experiment with the capabilities unlocked by LLMs, Unify can take the cost/speed/accuracy trade-off out of the development equation, and let them focus on unlocking the user experience and business value.
Join the Conversation!
We'd love to hear your thoughts on this exciting innovation in the wholesale industry. Share your feedback, questions, or experiences in the comments below!
Liked the newsletter today, insightful and easy to read