What is an AI Agent Framework?
It feels like everyone is building their own AI Agent Framework, from solo Open Source developers to the largest tech companies. If you’ve been following the news, Microsoft alone has launched at least four: the Microsoft Agent Framework, Semantic Kernel, AutoGen, and Microsoft Bot Framework. With so many frameworks appearing almost weekly, it’s hard to keep up. But what exactly is an Agent Framework? How do they stack up? And do you need one at all?
Do you even need an Agent Framework?
The first thing you’ll find when digging into Agent Frameworks is that there’s a split on whether you even need an Agent Framework in the first place. AI Agents are just “LLMs + tools, in a loop,” right? The reasoning goes that if that’s all there is to it, just make API calls to OpenAI yourself. They even cite the sheer volume of AI Agent Frameworks as proof for how trivial the problem space is.
But if you go down this route, you’ll quickly find people complaining that Agents don’t work. Fundamentally (and we all know this), LLMs make mistakes. Those mistakes feel unpredictable and somewhat random to us humans. Over long time horizons and many calls, even the small error rate of State of the Art, frontier models compound into almost certain failures.
So you’ll have to build in retries and checkpointing to recover from “obvious” (verifiable) errors. But more subtle errors in reasoning will remain. Over time, these subtle errors in logic and slowly accumulating context growth can cause the LLM to abandon its system prompt entirely. Try unleashing an LLM on something as simple as a login form and see what you get.
So “LLMs + tools, in a loop” is too much freedom. At this point, you’ll need to build state machines that leverage the power and flexibility of LLMs with the rigidity of consistent and logically correct workflows. You might find that you need checkpointing and partial execution to ensure your agent can run over longer time horizons. And just like any project that drastically underestimates the difficulty of the real-world, you’ve ended up with a mess of custom code and error handling. If you got here, congratulations you’ve built an Agent Framework.
What makes a good Agent Framework?
At LatentSpin, we build long-running AI Agents that can handle complex/real-world tasks. We believe that the fundamental barrier to unlocking these Agents is continuous learning. That is, the ability to learn from a changing environment, just like a person would. Without continuous learning, LLMs will always be bounded by a limited context window, unable to learn from past mistakes.
At the same time, given our team’s unique background in databases and distributed systems, we knew we needed a strong Agent Framework to serve as the foundation. We wanted a way to describe structured control flows, retry mechanisms, and checkpointing to build resilient self-improving Agents. We fundamentally believe that Agents need these features to be successful in production environments at scale. At its core, the process of solving long-term/sufficiently complicated tasks demands managing errors, we just tend to not think about this as humans. Finally, we hoped to be able to do this in a statically-typed language, where the compiler makes validation/correctness a first class citizen.
The other feature that makes a great framework is its ecosystem. The fundamental truth is that frameworks need real investment from a core team + a population of real enthusiasts who will build supporting libraries. The AI ecosystem is broad and moving quickly. The maintainers alone will be unable to sustain the current pace of innovation. They need a real community.
Our Story
In 2025, the AI Agent Framework landscape is a confusing mess of partial solutions. There are real advantages to using an Agent Framework but it’s far too early to say which frameworks will be the “winner.” With all of this in mind, we did a bunch of research/testing and landed on LangChain/LangGraph.
LangChain isn’t a shocking choice. It’s easily the most popular/well supported Agent Framework out there. It has a massive community and endless integrations. Combined with LangGraph, a durable graph execution framework, it imposes a real philosophy onto building Agents. And importantly to our system, there are TypeScript bindings which gives us access to a rich static typesystem.
But LangGraph isn’t perfect. Its graph system is far too flexible to always produce reliable Agents, and it makes little use of TypeScript’s powerful type system. To that end, it suffers from being a direct port of the company’s Python library of the same name.
So while we settled on LangGraph/LangChain, we built what we call the Flows DSL on top of it. Heavily Inspired by Akka Streams, it is rigidly typesafe with complete interop with LangGraph, even compiling directly down to LangGraph’s graph execution model. Instead of building graphs, we build flows/streams. Retries are conveniently expressed, context is managed through the flow, and the entire construction is typesafe.
The Flows DSL also supports our Execution Graph concept where end-users build Agents collaboratively with our AI Planner and iteratively define the “flow” that will be utilized to constrain the Agent’s behavior while providing automatic validations at each logical checkpoint. For this, we drew heavily on our experience building database query planners. And we also hijack LangGraph’s checkpointing features to support partial/repeatable Agent executions that slots into our serverless architecture, giving us the ability to elastically scale up to meet demand.
At LatentSpin, we were looking for an Agent Framework that would be to Agents as reactive streams are to Akka Streams. We wanted a framework that would help us build robust agents. Too frequently Agent Frameworks are focused on impressive demos (“an agent in 10 lines of code!”) and not on long term, large-scale agent deployments that are predictable, can recover from errors, and work for days not minutes. Fundamentally, the value of agents are in tasks that they can perform over hours or days, not minutes and at LatentSpin, we unlock this value.
