One of the biggest lessons for me while riding the MCP hype was that if you're writing backend software, you don't actually need to do MCP. Architecturally, they don't make sense. Atleast not on Elixir anyway. One server per API? That actually sounds crazy if you're doing backend. That's 500 different microservices for 500 APIs. After working with 20 different MCP servers, it then finally dawned on me, good ole' function calling (which is what MCP is under the hood) works just fine. And each API can be just it's own module instead of a server. So, no need to keep yourself updated on the latest MCP spec nor need to update 100s of microservices because the spec changed. Needless complexity.
Unless the client speaks to each micro service independently without going through some kind of gateway or BFF, I'd expect you'd just plonk your MCP in front of it all and expose the same functionality clients call via API (or graphql or RPC or whatever), so it's basically just an LLM-specific interface.
No reason why you couldn't just use tool calls with OpenAPI specs though. Either way, making all your microservices talk to each other over MCP sounds wild.
I always saw MCPs as a plug-and-play integration for enabling function calling without incurring API costs when using Claude.
If you're using the API and not in a hurry, there's no need for it.
> One server per API? That actually sounds crazy if you're doing backend
Not familiar with elixir, but is there anything prohibiting you from just making a monolith MCP combining multiple disparate API's/backends/microservices as you were doing previously?
Further, you won't get the various client-application integrations with MCP merely using tool-calling; which to me is the "killer app" of MCP (as a sibling comment touches on).
(I do still have mixed feelings about MCP, but in this case MCP sorta wins for me)
> just making a monolith MCP combining multiple disparate API
This is what I ended up doing.
The reason I thought I must do it the "MCP way" was because of the tons of YouTube videos about MCP which just kept saying how much of an awesome protocol it is, an everyone should be using it, etc. Once I realized it's actually more consumer facing than the backend, it made much more sense as why it became so popular.
It really is a standard protocol for connecting clients to models and vice versa. It’s not there to just be a container for tool calls.
- [deleted]
Agree, one MCP server per API doesn’t scale.
With something like https://nango.dev you can get a single server that covers 400+ APIs.
Also handles auth, observability and offers other interfaces for direct tool calling.
(Full disclosure, I’m the founder)
Why do you even need to connect to 400 APIs?
In the end, MCP is just like Rest APIs, there isn't need for a paid service for me to connect to 400 Rest APIs now, why do I need a service to connect to 400 MCPs?
All I need for my users is to be able to connect to one or two really useful MCPs, which I can do myself. I don't need to pay for some multi REST API server or multi MCP server.
Agentic automation is almost always about operating multiple tools and doing something with them. So you invariably need to integrate with a bunch of APIs. Sure, you can write your own MCP and implement everything in it. Or you can save yourself the trouble and use the official one provided by the integrations you need.
My point is though, that you don't need some 3rd party service to integrate hundreds of MCPs, it doesn't make any sense at all.
An "agent" with access to 400 MCPs would perform terribly in real world situations, have you ever tried it? Agents would best with a tuned well written system prompt and access to a few, well tuned tools for that particular agents job.
There's a huge difference between a fun demo of an agent calling 20 different tools, and an actual valuable use-case which works RELIABLY. In fact, agents with tons of MCP tools are currently insanely UNRELIABLE, it's much better to just have a model + one or two tools combined with a strict input and output schema, even then, it's pretty unreliable for actual business use-cases.
I think most folks right now have never actually tried making a reliable, valuable business feature using MCPs so they think somehow having "400 MCPs" is a good thing. But they haven't spent a few minutes thinking, "wait, why does our business need an agent which can connect to Youtube and Spotify?"
People want to not think and throw the kitchen sink at problems instead of thinking what they actually need.
Nango is cool, but pricing is quite high at scale.
There are quite a few competitors in this space, trying to figure the best way about this. I've been recently playing with the Jentic MCP server[0] that seems to do it quite cleanly and appears to be entirely free for regular usage.
We offer volume discounts on all metrics.
Email me on robin @ <domain> and happy to find a solution for your use case
Looks pretty cool, thanks for sharing!
nothing prevents you from using a proxy server if you want it all from one entry point. idk of any clients hitting hundreds of MCPs anyways, that's a straw man.
unlike web dev where you client cab just write more components / API callers, AI has context limits. if you try to plug (i get it, exaggerated) 500 MCP servers each with 2-10 tools you will waste a ton of context in your sys prompt. you can use a tool proxy (one tool which routes to others) but that will add latency from the processing.
I'll go one further -
Forcing LLMs to output JSON is just silly. A lot of time and effort is being spent forcing models to output a format that is picky and that LLMs quite frankly just don't seem to like very much. A text based DSL with more restrictions on it would've been a far better choice.
Years ago I was able to trivially teach GPT 3.5 to reliably output an English like DSL with just a few in prompt examples. Meanwhile even today the latest models still have notes that they may occasionally ignore some parts of JSON schemas sent down.
Square peg, round hole, please stop hammering.
MCP doesn't force models to output JSON, quite the opposite. Tool call results in MCP are text, images, audio — the things models naturally output. The whole point of MCP is to make APIs digestable to LLMs
I think perhaps they're more referring to the tool descriptions...not sure why they said output
I'm not sure about that. Newer models are able to output structured outputs perfectly and infact, if you combine it with changesets, you can have insanely useful applications since changesets also provide type-checking.
For example, in Elixir, we have this library: https://hexdocs.pm/instructor/
It's massively useful for any structured output related work.
Any model can provide perfect JSON according to a schema if you discard non-conforming logits.
I imagine that validation as you go could slow things down though.
The technical term is constrained decoding. OpenAI has had this for almost a year now. They say it requires generating some artifacts to do efficiently, which slows down the first response but can be cached.
Expect this is a problem pattern that will be seen a lot with LLMs.
Do I look at whether the data format is easily output by my target LLM?
Or do I just validate clamp/discard non-conforming output?
Always using the latter seems pretty inefficient.
“each API can just be its own module instead of a server”
This is basically what MCP is. Before MCP, everyone was rolling their own function calling interfaces to every API. Now it’s (slowly) standardising.
If you search for MCP integrations, you will find tons of MCP "servers", which basically are entire servers for just one vendor's API (sometimes just for one of their products, eg. YouTube). This is the go to default right now, instead of just one server with 100 modules. The MCP protocol itself is just to make it easier to communicate with the LLM clients that users can use and install. But, if you're doing backend code, there is no need to use MCP for it.
There's also little reason not to. I have an app server in the works, and all the API endpoints will be exposed via MCP because it hardly requires writing any extra code since the app server already auto-generates the REST endpoints from a schema anyway and can do the same for MCP.
An "entire server" is also overplaying what an MCP server is - in the case where an MCP server is just wrapping a single API it can be absolutely tiny, and also can just be a binary that speaks to the MCP client over stdio - it doesn't need to be a standalone server you need to start separately. In which case the MCP server is effectively just a small module.
The problem with making it one server with 100 modules is doing that in a language agnostic way, and MCP solves that with the stdio option. You can make "one server with 100 modules" if you want, just those modules would themselves be MCP servers talking over stdio.
> The problem with making it one server with 100 modules is doing that in a language agnostic way
I agree with this. For my particular use-case, I'm completely into Elixir, so, for backend work, it doesn't provide much benefit for me.
> it can be absolutely tiny
Yes, but at the end of the day, it's still a server. Its size is immaterial - you still need to deal with the issues of maintaining a server - patching security vulnerabilities and making sure you don't get hacked and don't expose anything publicly you're not supposed to. It requires routine maintenance just like a real server. Mulitply that with 100, if you have 100 MCP "servers". It's just not a scalable model.
In a monolith with 100 modules, you just do all the security patching for ONE server.
You will still have the issues of maintaining and patching your modules as well.
I think you have an overcomplicated idea of what a "server" means here. For MCP that does not mean it needs to speak HTTP. It can be just a binary that reads from stdin and writes to stdout.
>I think you have an overcomplicated idea of what a "server"
That's actually true, because I'm always thinking from a cloud deployment perspective (which is my use case). What kind of architecture do you run this on, at scale on the cloud? You have very limited options if your monolith is on a serverless and is CPU/memory bound, too. So, that's where I was coming from.
You'd run it on any architecture that can spawn a process and attach to stdin/stdout.
The overhead of spawning a process is not the problem. The overhead if your runtime is huge and/or slow to start could be, in which case you simply wouldn't run it on a serverless system - which is ludicrously expensive at scale anyway (my dayjob is running a consultancy where the big money-earner is helping people cut their cloud costs, and moving them off serverless systems that are entirely wrong for them is often one of the big savings; it's cheap for tiny systems, but even then often the hassle isn't worth it)
.. what's the security patching you have to do for reading / writing on stdio on your local machine?
Hehe.
> each API can be just it's own module
That implies a language lock-in, which is undesirable.
It is only undesirable if you are to expose your APIs for others to use via a consumer facing client. As a backend developer, I'm writing a backend for my app to consume, not for consumers (like MCP is designed for). So, this is better for me since I'm a 100% Elixir shop anyway.