AI SLAs
The SLA Hierarchy For Enterprises: Quality > Latency
Hey friends! I’m Akash, an early stage investor at Earlybird Venture Capital, partnering with founders across Europe at the earliest stages.
Software Synthesis is where I connect the dots on software and startup strategy - you can always reach me at akash@earlybird.com if we can work together!
Current subscribers: 4,400
Last week, we hosted a dinner for early stage AI founders together with our friends at JP Morgan. With a now $17bn IT budget, JP Morgan’s views on evaluating third party vendors for their AI needs couldn’t have been better articulated by Jack Smith (Technology Strategy) and Allie Gillon-Livesey (AI Strategy in Commercial and Investment Bank).
Some takeaways:
RAG is key to the most prominent early use cases around knowledge retrieval
The goal is to be model agnostic over time
Sponsorship is key; navigating procurement in an enterprise like JPM requires diligent process management given how multi-threaded the sales process needs to be
It’s very early for discussions of agents in the enterprise - for now it’s about getting simple applications off the ground
This last point is key for the rest of this post.
Q1’s sobering earnings will likely temper predictions of a software reacceleration at the rate Q4 ‘23 would have had us believe.
“While bulls might be willing to look through the disappointment given ‘it’s just a Q1,’ we believe these results raise more meaningful questions around the adoption curve and ultimate monetization of genAI for seat-based SaaS companies”
Nearly 50% of AI budgets will be siphoned from other budgets in the enterprise, including existing IT budgets, underlining one of the longer-term headwinds to software earnings.
Discussions about the End of Software ensued, with retorts pointing to enterprises continuing to value enterprise support and SLAs.
For AI-native disruptors to effectively counterposition against seat-based ‘legacy’ enterprise software, the reliability with which their atomic unit of value is delivered is key.
It’s worth thinking about what SLAs will look like for AI-native companies that sell outcomes (SLAs are in c. 40% of B2B SaaS contracts).
In Beyond SaaS,
wrote about how AI-native companies could counterposition against incumbents with innovative SLA guarantees:Software companies often price discriminate with different tiers of reliability (e.g. “two-nines” versus “five-nines”). LLM companies could go a step further and construct pricing tiers around latency, accuracy, and relevance.
Philip admits the challenge for disruptors to outcompete incumbents on latency given the dependency on compute (e.g. ultra low-latency Groq doesn’t come cheap).
Accuracy and relevance are two dimensions that startups have more control over through intelligent system design; RAG, knowledge graphs, and agentic design patterns can significantly improve both. In fact, optimising system design for quality of accuracy and relevance will increase latency.
For the economy of 24/7 agent workers that are expected to eat into BPO, quality of outcomes (where latency is not a dimension of outcome quality) will more often than not outweigh inference speeds.
Outside of knowledge retrieval where latency is a more important consideration, the always-on nature of AI workers (and the higher throughput of work that this enables) places a greater emphasis on the quality of outcomes.
Let’s take a few examples:
AI SDRs delivering a higher quality top-of-funnel > low latency on outreach generation
SOC Analysts accurately investigating alerts > rendering possibly wrong conclusions faster
Compliance Analysts determining product compliance > wrong decisions leading to penalties
Just as it’s worth bifurcating the market for AI applications that can run on the edge (even with specialised hardware that has more memory) versus relying on the cloud, it’s also worth looking at the range of use cases where quality trumps latency for buying decisions.
When the early signs of margin expansion are as high as an 11% reduction in S&M spend, demand is undoubtedly going to accelerate for AI-native disruptors. At the same time, buyers will be applying rigour to scrutinise perceived outcome quality. As founders architect their products, they need to factor in the hierarchy of enterprise SLA guarantees.
Charts of the week
OpenAI is leading the way when it comes to striking partnerships with publishers
Amidst the gloom in software, CIO surveys would still suggest a back-half acceleration
Curated Content
SaaS Isn't Dying: It's Going Logarithmic by Jared Sleeper
We aren’t running out of training data, we are running out of open training data by
AI Integration and Modularization by Ben Thompson
Investing in the Cloud: From Gold Rush to Hunger Games and Beyond by Rory O’Driscoll
Quote of the week
‘Just like in the world of standardized testing, private benchmarks would need to establish themselves as standard-bearers for quality, which requires building up trust with the broad community. In a competitive market where end users of these models really do care about quality and performance, benchmark providers will sprout up and compete for their business, jostling for position as the ultimate arbiter of model performance. In the limit, the benchmarks will be benchmarked, in the same way that colleges are ranked for quality in explicit and implicit ways. Trusted “brands” will emerge.’
Thank you for reading. If you liked it, share it with your friends, colleagues, and anyone that wants to get smarter on startup strategy. Subscribe below and find me on LinkedIn or Twitter.