Show HN: GoModel – an open-source AI gateway in Go

Hi, I'm the author of GAI. I'm glad that you are happy with it! I'm using it a lot myself, both for own and client projects, but I wasn't sure if anyone else was. :D

I'm actually trying to build it out in a way so that gateways aren't necessarily necessary. Cost and token tracking happen through OpenTelemetry. Fallbacks and retries are handled through the new “robust” package, and I have other plans as well. You're always welcome to file issues in the repo for things you'd like to see but aren't there yet. :-)

If you're all in on Go and AI, you might want to take a look at: https://github.com/ewhauser/gbash

It's a just-bash like variant implemented in Go. Useful for giving a managed bash tool to your agents without a full sandboxing solution.

I built Fence (https://github.com/Use-Tusk/fence) in Go, a lightweight process sandbox for CLI agents (or any command really) with filesystem and network restrictions. It's also available as a Go library if you wish to add sandboxing to Shelley.

I'll take a closer look at it over the next few days.

However, it might be challenging, considering that Claude Code with a subscription no longer officially works with OpenClaw.

i use this for my personal projects. some features are gated behind a license but the basics like provider proxy, logs, metrics are covered in the free version. https://github.com/maximhq/bifrost

> Any other Go-based AI / LLM tools folks are happy with?

I can throw my hat into the ring, built on ADK, CUE, and Dagger (all also in Go); CLI, TUI, and VSCode interfaces. It's my personal / custom stack, still need to write up docs. My favorite features are powered by Dagger, sandbox with time travel, forking, jump into shell at any turn, diff between any points.

Good entrypoint folder: https://github.com/hofstadter-io/hof/tree/_next/lib/agent

Given the LiteLLM Supply Chain Incident, threat mitigation and governance are definitely major concerns.

[dead]

Maybe worth showing on SHOW HN

"... and I don't see if I would be able to track usage from individual end-users through a header".

Currently we have a unified concept of User-Paths. Once you add a specific header OR assign User-Path to an API key, you can track the usage based on this. The User-Path might be youe end-user, internal user or some service. Examples:

  /client1/app1
  /agents/agent1
  /team2/john
  /team2/adam

https://gomodel.enterpilot.io/docs/features/user-path

PS Thanks for the feedback on the Vault integration. Noted.

It's like fuel costs in a supply chain. When you buy apples at the store, you don't think about oil prices. But if trucks ran on something cheaper, more efficient, or less taxed, the apples on the shelf would be cheaper too.

Thank you. No config atm to declare which fields are "stable" for caching. I assume you mean cases where the user would provide a different session ID but still would prefer the key to match. Am I right?

1. Yes, we have OpenAI-compatible API and we develop GoModel with Postel’s law in mind: https://gomodel.enterpilot.io/docs/about/technical-philosoph... .

2. Regarding being open-source and the license, I've described our approach here transparently: https://gomodel.enterpilot.io/docs/about/license

I'm wroking on it full-time right now. It might be challenging, especially when it comes to interactions with video, audio, and image models. I'm just trying to stay on top of what's happening and add new things day by day.

An even bigger challenge might be integrating it tighter with the cloud platforms, Prometheus, OAuth, Datadog, vaults, and DLP software.

I've been maintaining an abstraction layer over multiple providers for a couple of years now - https://llm.datasette.io/

The best effort we have to defining a standard is OpenAI harmony/responses - https://developers.openai.com/cookbook/articles/openai-harmo... - but it's not seen much pickup. The older OpenAI Chat Completions thing is much more of an ad-hoc standard - almost every provider ends up serving up a clone of that, albeit with frustrating differences because there's no formal spec to work against.

The key problem is that providers are still inventing new stuff, so committing to a standard doesn't work for them because it may not cover the next set of features.

2025 was particularly turbulent because everyone was adding reasoning mechanisms to their APIs in subtly different shapes. Tool calls and response schemas (which are confusingly not always the same thing) have also had a lot of variance - some providers allow for multiple tool calls in the same response, for example.

My hunch is we'll need abstraction layers for quite a while longer, because the shape of these APIs is still too frothy to support a standard that everyone can get behind without restricting their options for future products too much.

The providers themselves can't keep this straight even within their own ecosystem. Plus everyone is running at a million miles/hour.

For example `Claude code` used to set 2 specific beta headers with some version numbers for their Max subscription to be supported.

Oauth tokens for Max plan is different from how their API keys looked. They kind of look similar, but has specific prefix that these tool pre-validate.

It is barely working at this point even within a single provider

It’s a complete mess, and the hardest part of this kind of tool is maintenance.

It’s not just about incompatible APIs, but also about how messages are structured. Even getting reliable tool calling requires a significant amount of work and testing for each individual model.

Just look at LiteLLM’s commit history and open issues/PRs. They’re still struggling with reliable multi-turn tool calling for Gemini, Kimi requires hardcoded rules (so K2.6 is currently unsupported because it’s not on the list), and so on.

Implementing the basic, generic OpenAI/Anthropic protocols is trivial, and at that point it almost feels like building an AI gateway is done. But it isn’t — that’s just the beginning of a long journey of constantly dealing with bugs, changes, and the quirks of each provider and model.

First, GoModel is designed to be flexible. If you add an extra field, it tries to pass it through in the appropriate place (Postel's law)

Therefore there's a good chance that if they make a minor API-level change, GoModel will handle it without any code changes.

Also, changes to providers' API formats might be less and less frequent. Keeping up typically means adding a few lines of code per month. I'm usually aware of those changes because I use LLMs daily and follow the news in a few places.

As a fallback, GoModel includes a passthrough API that forwards your request to the provider in its original format. That might be useful when an AI provider changes their contract significantly and we haven't caught up yet.

Also, official SDKs aren't bug-free either. Skipping that extra layer and hitting the API directly might actually be beneficial for GoModel.

Most APIs provide some sort of documentation. If it’s swagger you can just update the application from that.

Almost impossible without backing from some VC like litellm

LiteLLM proxy has to be about the worst codebase and performing Python code I've used in a long time. Incredibly poor performance, riddled with serious bugs and to be honest devs that don't seem to understand them.

Agree and thank you! Please let us know if you'd like to give it a try and if you miss any feature in GoModel.

It’s also badly implemented - everything is a global import. Had to stop using it

Hey, contributor here. That's right, GoModel embeds requests and does vector similarity lookup before proxying. Regarding the cache invalidation, there is no "purging" involved – the model is part of the namespace (params_hash includes the LLM model, path, guardrails hash, etc). TTL takes care of the cleanup later.

I've released a new version of GoModel (0.1.20) with explicit support for vllm. You can now use it even with a few vLLM instances. Like this:

  docker run --rm -p 8080:8080 \
    -e VLLM_BASE_URL=http://host.docker.internal:18000/v1 \
    -e VLLM_BASEMENT_BASE_URL=http://host.docker.internal:18000/v1 \
    enterpilot/gomodel:latest

This is way more interesting to me as well. I have projects that use small limited-purpose language models that run on local network servers and something like this project would be a lot simpler than manually configuring API clients for each model in each project.

This comment looks like AI-generated.

However IIUC what you're asking for - it's already in the dashboard! Check the Usage page.

You are not the first person who has asked about it.

It looks like a useful feature to have. Therefore, I'll dig into this topic more broadly over the next few days and let you know here whether, and possibly when, we plan to add it.

My thoughts about this:

Benchmarking AI gateways properly is harder than it looks. Feature sets differ meaningfully - exact vs semantic caching, cluster mode, guardrails, audit logging - and each carries its own latency cost. What actually matters for most users is end-to-end latency including provider overhead (200–2000ms), and in that frame Bifrost, LiteLLM, and GoModel are all perfectly fine.

I ran some comparisons but I'm not happy with the methodology, and I'd rather not spread misleading information. Once I have time to do it properly I'll write it up and share a link here. Honestly, I'd also love to see benchmarks done by someone other than the AI gateway builders. :)

Where GoModel actually differs today:

  - image size: 16.96 MB vs Bifrost's 69.84 MB. It matters for sidecar, edge, and cold-start scenarios.
  - per-tenant keys, guardrails, and audit logs are all in the OSS repo - not gated.
  - AI interaction visualization that makes debugging individual request/response flows much easier.

It’s a heavily vibe coded project with only proxy with terrible benchmarks design. Basically vibe coded benchmarks that lie through ignorance of mocked super fast endpoint without using full power of litellm in multiple processes.

Other than that almost useless it’s faster when this will be io bound and not cpu bound.

First of all, GoModel doesn't have a separate private repository behind a paywall/license.

It's more lightweight and simpler. The Bifrost docker image looks 4x larger, at least for now.

IMO GoModel is more convenient for debugging and for seeing how your request flows through different layers of AI Gateways in the Audit Logs.