LLM orchestrators: how much code you actually need
We started with a framework and finished with 300 lines of our own. It doesn't always go that way, but it goes that way more than people admit.
Frameworks speed up the first prototype and get in the way during debugging, because they hide the final prompt being sent. You want to understand exactly what text hit the API. A framework that abstracts that away costs you hours of investigation when results go wrong.
We started two projects with LangChain, moved to custom code in both. The framework's opinion about chains and prompts felt right until the moment we needed to modify what was actually being sent. Then it became a liability.
The moment a project needs to see and version that exact prompt, the abstraction starts costing money. Prompts change. Sometimes it's to fix an issue, sometimes to improve quality. You need to commit that change to git and know exactly what shipped. Frameworks make that harder.
Custom code is verbose, but the verbosity is the point. Everything that happens is visible. No magic. When something goes wrong, you're debugging code you wrote, not code someone else abstracted away.
Our rule: framework to explore, own code for production, unless the client's team already lives in that framework. If the team knows LangChain and has built on it, we work within it. But we don't force a framework on a project that doesn't need one.
Three hundred lines of Python that you control beats a framework where you spend three hundred hours debugging someone else's choices.