Jan – Ollama alternative with local UI
github.com

・

197 points

・

maxloh

・

22 days ago

78 comments

apitman ・ 22 days ago

I really like Jan, especially the organization's principles: https://jan.ai/

Main deal breaker for me when I tried it was I couldn't talk to multiple models at once, even if they were remote models on OpenRouter. If I ask a question in one chat, then switch to another chat and ask a question, it will block until the first one is done.

Also Tauri apps feel pretty clunky on Linux for me.

  • diggan ・ 22 days ago

    > Also Tauri apps feel pretty clunky on Linux for me.

    All of them, or this one specifically? I've developed a bunch of tiny apps for my own usage (on Linux) with Tauri (maybe largest is just 5-6K LoC) and always felt snappy to me, mostly doing all the data processing with Rust then the UI part with ClojureScript+Reagent.

  • signbcc ・ 22 days ago

    > especially the organization's principles

    I met the team late last year. They’re based out of Singapore and Vietnam. They ghosted me after promising to have two follow-up meetings, and were unresponsive to any emails, like they just dropped dead.

    Principles and manifestos are a dime a dozen. It matters if you live by them or just have them as PR pieces. These folks are the latter.

    • dcreater ・ 22 days ago

      With a name like Menlo research, I assumed they were based in Menlo park. They probably intended that

  • _the_inflator ・ 22 days ago

    Yep. I really see them as an architecture blueprint with a reference implementation and not so much as a one size fits all app.

    I stumbled upon Jan.ai a couple of months ago when I was considering a similar app approach. I was curious because Jan.ai went way beyond what I considered to be limitations.

    I haven’t tried Jan.ai yet, I see it as an implementation not a solution.

  • inkyoto ・ 22 days ago

    > Main deal breaker for me when I tried it was I couldn't talk to multiple models at once […]

    … which seems particularly strange considering the size of the cloned GitHub repository to be 1.8GiB which swells up to 4.8GiB after running «make build» – I tried to build it locally (which failed anyway).

    It is startling that a relatively simple UI frontend can add 3Gb+ of build artefacts alone – that is the scale of a Linux kernel build.

  • c-hendricks ・ 22 days ago

    Yeah, webkit2gtk is a bit of a drag

roscas ・ 22 days ago

Tried to run Jan but it does not start llama server. It also tries to allocate 30gb that is the size of the model but my vram is only 10gb and machine is 32gb, so it does not make sense. Ollama works perfect with 30b models. Another thing that is not good is that it make constant connections to github and other sites.

  • hoppp ・ 22 days ago

    It probably loads the entire model into ram at once while ollama solves this and does not, it has a better loading strategy

    • blooalien ・ 22 days ago

      Yeah, if I remember correctly, Ollama loads models in "layers" and is capable of putting some layers in GPU RAM and the rest in regular system RAM.

  • trilogic ・ 22 days ago

    [flagged]

    • woadwarrior01 ・ 22 days ago

      > If you looking for privacy there is only 1 app in the whole wide internet right now, HugstonOne

      That's a tall claim.

      I've been selling a macOS and iOS private LLM app on the App Store for over two years now, that is:

      a) is fully native (not electron.js) b) not a llama.cpp / MLX wrapper c) fully sandboxed (none of Jan, Ollama, LM Studio are)

      I will not promote. Quite shameless of you to shill your electron.js based llama.cpp wrapper here.

      • threecheese ・ 22 days ago
        ・ 2 more

        Purchased, to show my support (and to play around ofc).

        • woadwarrior01 ・ 21 days ago

          Thanks! Also check out my other free privacy focussed app. :)

      • trilogic ・ 22 days ago
        ・ 6 more

        [flagged]

        • rovr138 ・ 22 days ago
          ・ 5 more

          Since they won't promote, here's the link, https://apps.apple.com/us/app/private-llm-local-ai-chat/id64...

          > I accept every challenge to prove that HugstonOne is worth the claim.

          I expect your review.

          • trilogic ・ 22 days ago
            ・ 4 more

            [flagged]

            • rovr138 ・ 22 days ago
              ・ 2 more

              I did? None of your points talk about privacy which was your original argument.

              I’ll remind you,

              > If you looking for privacy there is only 1 app in the whole wide internet right now, HugstonOne (I challenge everyone to find another local GUI with that privacy).

              Heck, if you look at the original comment, it clearly states it’s macOS and iOS native,

              > I've been selling a macOS and iOS private LLM app on the App Store for over two years now, that is: > a) is fully native (not electron.js) b) not a llama.cpp / MLX wrapper c) fully sandboxed (none of Jan, Ollama, LM Studio are)

              How do you expect it to be and cross platform? Isn’t hugstone windows only?

              So, what are your privacy arguments? Don’t move the goal post.

              • trilogic ・ 22 days ago

                I am honest, I don't know how your app works, speaking privacy. I can´t even try it. You are free (like literally free) to try mine. How do users of mac/ios own their data is unknown to me. I didn't want to make a point on that as I have already big techs against, and didn't want to hijack the session further.

                Now for real, I wish to meet more people like you, I admire your professional way of arguing, and I really wish you all the best :)

            • yjftsjthsd-h ・ 22 days ago

              > 7 It is only for mac for god sake, you need to pay to breath.

              And HugstonOne is for Windows; what of it?

    • imiric ・ 22 days ago

      You mean this[1]?

      It's not open source, has no license, runs on Windows only, and requires an activation code to use.

      Also, the privacy policy on their website is missing[2].

      Anyone remotely concerned about privacy wouldn't come near this thing.

      Ah, you're the author, no wonder you're shilling for it.

      [1]: https://github.com/Mainframework/HugstonOne

      [2]: https://hugston.com/privacy

      • trilogic ・ 22 days ago
        ・ 3 more

        The app has a licence well visible when you install the app. The rest is written in the website and in github. Then about: requires an activation code to use, ofc it is made for ethical research purposes, so yes I am distributing it responsibly. And you can see the videos in the youtube channel for how it works. But the most important point is that you can try it easily with a firewall to see that it do not leak bytes like all the rest there. That´s what i call privacy, It has a button that cut all connection. You can say what you want but that´s it that´s all.

        • do_not_redeem ・ 22 days ago

          > But the most important point is that you can try it easily with a firewall to see that it do not leak bytes like all the rest there.

          Great to hear! Since you care so much about privacy, how can I get an activation code without sending any bytes over a network or revealing my email address?

        • riquito ・ 22 days ago

          Closed source, without 3rd party independent review and people should just trust you? As if your app cannot start sending data away in a month or attempt to detect monitoring software, to name a couple

    • kgeist ・ 22 days ago

      >I challenge everyone to find another local GUI with that privacy

      Llama.cpp's built-in web UI.

      • trilogic ・ 22 days ago
        ・ 10 more

        This is from webui website docs: Once saved, Open WebUI will begin using your local Llama.cpp server as a backend! So you see Llama server not CLI. That´s a big flag there. I repeat no app in the whole world takes seriously privacy like HugstonOne. This is not advertisement, I am just making a point.

        • kgeist ・ 22 days ago
          ・ 9 more

          I'm not sure what you're talking about. Llama.cpp is an inference server which runs LLMs locally. It has a built-in web UI. You can't get more private than the inference server itself.

          I tried downloading your app, and it's a whopping 500 MB. What takes up the most disk space? The llama-server binary with the built-in web UI is like a couple MBs.

          • trilogic ・ 22 days ago
            ・ 8 more

            With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http. And yes the app is a bit heavy as is loading llm models using llama.cpp cli and multimodal which in itself are quite heavy, also just the dlls for cpu/gpu are huge, (just the one for the nvidial gpu is 500mb if I don't go wrong).

            • kgeist ・ 22 days ago

              Unless you expose random ports on the local machine to the Internet, running apps on localhost is pretty safe. Llama-server's UI stores conversations in the browser's localStorage so it's not retrievable even if you expose your port. To me, downloading 500 MB from some random site feels far less safe :)

              >the app is a bit heavy as is loading llm models using llama.cpp cli

              So it adds an unnecessary overhead of reloading all the weights to VRAM on each message? On some larger models it can take up to a minute. Or you somehow stream input/output from an attached CLI process without restarting it?

            • rcakebread ・ 22 days ago
              ・ 2 more

              Says the guy with a link to a broken privacy policy on their website.

              • trilogic ・ 22 days ago

                I accept critics, and I thank you for it. It will be fixed ASAP.

            • giantrobot ・ 22 days ago
              ・ 4 more

              > With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http.

              What in the world are you trying to say here? llama.cpp can run completely locally and web access can be limited to localhost only. That's entirely private and offline (after downloading a model). I can't tell if you're spreading FUD about llama.cpp or are just generally misinformed about how it works. You certainly have some motivated reasoning trying to promote your app which makes your replies seem very disingenuous.

              • trilogic ・ 22 days ago
                ・ 3 more

                I am not here to teach cybersecurity Tcp/ip protocols or ML. HTTP = HyperText Transfer Protocol The standard protocol for transferring data over the web. CLI = Command-Line Interface. Try again after endless nights of informatic work please.

                • kgeist ・ 22 days ago

                  HTTP can be 100% local without involving the web.

                • giantrobot ・ 21 days ago

                  Wow. I honestly can't tell if you're trying to troll people are sitting on top of some Dunning-Kruger peak. Shine on you delusional diamond.

  • SilverRubicon ・ 22 days ago

    Did you see the feature list? It does not deny that makes connections to other sites.

    - Cloud Integration: Connect to OpenAI, Anthropic, Mistral, Groq, and others

    - Privacy First: Everything runs locally when you want it to

mathfailure ・ 22 days ago

Is this an alternative to OpenWebUI?

  • apitman ・ 22 days ago

    Not exactly. OWUI is a server with a web app frontend. Jan is a desktop app you install. But it does have the ability to run a server for other apps like OWUI to talk to.

    • ekianjo ・ 22 days ago

      Openweb-ui does not include a server.

      • cristoperb ・ 22 days ago

        It starts a webserver to serve its UI, which is what your comment parent meant. It doesn't provide its own openai-style API, which I guess is what you meant.

      • apitman ・ 22 days ago

        I was referring to Jan.

  • PeterStuer ・ 22 days ago

    More an alternative to LM Studio I think from the description.

    • apitman ・ 22 days ago

      Jan also supports connecting to remote APIs (like OpenRouter), which I don't think LM Studio does

jwildeboer ・ 22 days ago

My name is Jan and I am not an AI thingy. Just FTR. :)

  • underlines ・ 22 days ago

    Jan here too, and I work with LLMs full time and I'm a speaker about these topics. Annoying how many times people ask me if Jan.ai is me lol

    • dsp_person ・ 22 days ago

      We need a steve.ai

      • ithkuil ・ 22 days ago
        ・ 3 more

        I want a Robert Duck AI

        • tough ・ 22 days ago
          ・ 2 more

          We're the AI's Robert's Ducks

          • ithkuil ・ 20 days ago

            nice one.

            we're AI's fitness function

klausa ・ 22 days ago

So this is how women named Siri felt in 2011.

biinjo ・ 22 days ago

Im confused. Isn’t the whole premise of Ollama that its locallt ran? What’s the difference or USP when comparing the two.

  • moron4hire ・ 22 days ago

    That's not the actual tagline being used in the repo. The repo calls itself an alternative to ChatGPT. Whoever submitted the link changed it.

  • hoppp ・ 22 days ago

    I think its an alternative because ollama has no UI and its hard to use for non-developers who will never touch the CLI

    • simonw ・ 22 days ago

      Ollama added a chat UI to their desktop apps a week ago: https://ollama.com/blog/new-app

      • apitman ・ 22 days ago
        ・ 5 more

        Their new app is closed source right?

        • simonw ・ 22 days ago
          ・ 4 more

          Huh, yeah it looks like the GUI component is closed source. Their GitHub version only has the CLI.

          • diggan ・ 22 days ago
            ・ 2 more

            I think at this point it's fair to say that most of the stuff Ollama does, is closed source. AFAIK, only the CLI is open source, everything else isn't.

            • conradev ・ 22 days ago

              Yeah, and they’re also on a forked llama.cpp

          • kanestreet ・ 22 days ago

            Yup. They have not even acknowledged the fact that it’s closed, despite a ton of questions. Ppl are downloading it assuming it’s open source only to get a nasty surprise. No mention of it in their blog post announcing the GUI. Plus no new license for it. And no privacy policy. Feels deceptive.

      • accrual ・ 22 days ago
        ・ 3 more

        I have been using the Ollama GUI on Windows since release and appreciated its simplicity. It recently received an update that puts a large "Turbo" button in the message box that links to a sign-in page.

        I'm trying Jan now and am really liking it - it feels friendlier than the Ollama GUI.

        • dcreater ・ 22 days ago
          ・ 2 more

          And ollamas founder was on here posting that they are still focused on local inference... I don't see ollama as anything more than a funnel for their subscription now

          • numpad0 ・ 21 days ago

            I truly don't understand why it's supposed to be the end of the world. They need to monetize eventually, and simultaneously its userbase desireg good inference. It looks a complete win-win to me. Anyone can fork it in case they actually turn evil once it'd happen.

            I mean, it's not like people enjoy lovely smell of cash burning and bias opinions heavily towards it, or is it like that?

reader9274 ・ 22 days ago

Tried to run the gpt-oss:20b in ollama (runs perfectly) and tried to connect ollama to jan but it didn't work.

  • accrual ・ 22 days ago

    I got Jan working with Ollama today. Jan reported it couldn't connect to my Ollama instance on the same host despite it working fine for other apps.

    I captured loopback and noticed Ollama returning an HTTP 403 forbidden message to Jan.

    The solution was set environment variables:

        OLLAMA_HOST=0.0.0.0
    OLLAMA_ORIGINS=*
    Here's the rest of the steps:

    - Jan > Settings > Model Providers

    - Add new provider called "Ollama"

    - Set API key to "ollama" and point to http://localhost:11434/v1

    - Ensure variables above are set

    - Click "Refresh" and the models should load

    Note: Even though an API key is not required for local Ollama, Jan apparently doesn't consider it a valid endpoint unless a key is provided. I set mine to "ollama" and then it allowed me to start a chat.

semessier ・ 22 days ago

still looking for vLLM to support Mac ARM Metal GPUs

  • baggiponte ・ 22 days ago

    Yeah. The docs tell you that you should build it yourself, but…

    • tough ・ 22 days ago

      but unlike cuda there's no custom kernels for inference in vllm repo...

      I think

bogdart ・ 22 days ago

I tried Jan last year, but the UI was quite buggy. But maybe they fixed it.

  • diggan ・ 22 days ago

    Please do try it out again, if things used to be broken but they no longer are, it's a good signal that they're gaining stability :) And if it's still broken, even better signal that they're not addressing bugs which would be worse.

    • esafak ・ 22 days ago

      So you're saying bugs are good?!

      • diggan ・ 22 days ago

        No, but maybe that their shared opinion will be a lot more insightful if they provide a comparison between then and now, instead of leaving it at "it was like that before, now I don't know".

venkyvb ・ 22 days ago

How does this compare to LM studio ?

  • rmonvfer ・ 22 days ago

    I use both and Jan is basically the OSS version of LM Studio with some added features (e.g, you can use remote providers)

    I first used Jan some time ago and didn’t really like it but it has improved a lot so I encourage everyone to try it, it’s a great project

  • angelmm ・ 22 days ago

    For me, the main difference is that LM Studio main app is not OSS. But they are similar in terms of features, although I didn't use LM Studio that much.

azyc ・ 22 days ago

[dead]