Observable Notebooks 2.0 Technology Preview
observablehq.com

・

232 points

・

mbostock

・

3 days ago

63 comments

mourner ・ 3 days ago

The updated syntax looks brilliant — can't wait to try it out by upgrading some of my older notebooks! I was worried about the future of Observable since the Canvas announcement — it felt like notebooks were being slowly phased out, but I'm glad to be proven wrong.

Any timeline on bringing the new format to the Web editor? For me, the biggest draw of Observable is being able to instantly start exploring and visualizing some data without ANY setup (creating some local files, installing dependencies, launching a dev server etc.) — just click "New notebook" and off you go, frictionless. I hope this remains a priority.

Also worth mentioning the data viz community aspect, which I used to enjoy a ton — it's a bummer that things like being able to follow certain users and see their new notebooks in a feed were phased out, and that trending notebooks are now hidden under the Help menu instead of being featured on the user's homepage. This might be one of the reasons I stopped coming back to the platform as often, although I realize building a community is very difficult and hard to monetize — just thought I'd share.

  • mbostock ・ 3 days ago

    Thank you. I don’t want to commit to a public timeline on the web editor yet, but rest assured it is a top priority, and we also value the convenience of a web editor, instant collaboration, and community!

1dom ・ 3 days ago

I'm not sure I understand what the offering is here. I read the website, and watched the video, but it doesn't really make sense.

Is Notebook 2.0 just a HTML subset that starts with e.g. <notebook> instead of <html>?

Then what's the notebook kit npm package?

Then there's an editor?

I've come away from the page a little irritated, because starts with hyperbole like "the future of notebooks is here", and then the "hello world" in the new notebook 2.0 format looks like HTML, and a video of what could have just been a generic AI chat window.

Maybe I'm too hungry/tired to make sense of it right now. Will be interesting to read what others think.

  • skybrian ・ 3 days ago

    They probably could have explained better what hasn’t changed. Observable Notebooks work like a spreadsheet or makefile. The cells aren’t executed top to bottom (like a Jupyter Notebook). Instead they’re executed in dependency order.

    Looks like this is still true with the new system:

    > The Build API analyzes cell source code to find unbound references, such that the appropriate variable graph can be initialized using the Observable Runtime. For non-JavaScript cells (such as Markdown, HTML, and SQL), the Build API also transpiles the source into a tagged template literal expression.

    This is unlike other document formats. They made the cells look like standard JavaScript now (versus the JavaScript variant they had before) but the file format is necessarily different, because the execution model is different. There is a build step to ensure that the cells will be executed in the right order on page load.

  • chaps ・ 3 days ago

    Have a look at https://observablehq.com/@observablehq and just poke around.

    It's a neat tool that solves a lot of specific sharing/visualization problems (think of newsroom data analysis).

    I like observable but I hate how much it feels like the service has been taken over by product managers looking for that next bump.

dr__mario ・ 3 days ago

I'd love to love them but I don't: I can't seem to justify moving away from Python to use this. What advantages does JS offer for this use case? I' ve never felt that I couldn't do any visualization with Python (but I don't do nice newspaper figures).

  • MantisShrimp90 ・ 3 days ago

    Im in the same boat. In theory JavaScript holds more potential to make finely crafted visuals. But you're right, the ecosystem is so mature I still find other ecosystems lacking.

    But if anything would change my opinion this has the right set of values

    • chaps ・ 3 days ago

      I'm not really a JavaScript person either, but that hasn't been the worst problem with observable. Tools like vega-lite and importable tools from other notebooks really reduces the amount of code I need to write. Like, I'm not writing d3 code for hours for a simple choropleth. What makes it easier is that I can query objects using SQL rather than lengthy JavaScript code.

  • wonger_ ・ 3 days ago

    Exporting as client-side webpages, maybe? I liked using it to prototype some interactive D3.js charts that I would later move to my website. Also some people just are more comfortable with JS than Python

  • jeffbee ・ 3 days ago

    Maybe I just don't know enough python libraries but I can get to a good-looking interactive visual in Observable in almost no time at all.

  • skybrian ・ 3 days ago

    It depends where you're coming from, I suppose. The web is pretty popular. Web developers are more familiar with HTML, CSS, and JavaScript, and with various visualization libraries in the JavaScript ecosystem. I quite like Observable Plot [1].

    I'm not familiar enough with Python or Jupyter to know how you would build similar visualizations with them. What would you use?

    [1] https://observablehq.com/@observablehq/plot-gallery

    • ZeroCool2u ・ 2 days ago

      You'd have to do some styling, but Plotly is what I'd use to recreate the graphs on the gallery page there. It would obviously require some work to match the exact styling, but it should all be doable and TTFP would probably be much lower assuming you're starting with a Plotly express template.

      https://plotly.com/python/

  • Hasnep ・ 3 days ago

    I think marimo is a python version of the same idea: https://marimo.io/

    • threecheese ・ 2 days ago

      I am loving Marimo, I think they are doing a good job of balancing ease of use with the ability to drop into a virtualenv and run the code. The UI is much nicer than colab, from a visual and feature perspective. Key for me though is repeatability and ability to collaborate at the branch level, shared notebooks have issues we all know about.

      What’s missing for me is a local desktop app like Jupyter Studio, but that’s an easy thing for the community to build.

  • nsonha ・ 2 days ago

    > never felt that I couldn't do any visualization with Python

    Yes but, probably what they are shooting for are mini apps that you can just copy pasted into a front-end codebase and not some weird-ass python dsl

jitl ・ 3 days ago

I used the older online notebooks for a bit and really loved the immediacy but I’m always hurting for TypeScript syntax support. The is this a thing w 2.0? Something I’ll need to add later with Vite and only available locally?

It was a bunmer to not be able to copy paste code from my projects into observable. With local notebooks I see more potential for homegrown workflow but for me the benefit was always the platform and how starting a new thing was as simple as going to a webpage and clicking plus.

Anyways excited to play with this once it ends up in the online editor

  • mbostock ・ 3 days ago

    We’re interested in adding a TypeScript cell mode, probably first with type stripping rather than validating types. I was hoping for a pure JavaScript implementation; maybe we’ll try https://github.com/bloomberg/ts-blank-space? I’ve also been looking at https://github.com/val-town/codemirror-ts for the editor, but it’s a lot of machinery.

    • jitl ・ 2 days ago

      I would only expect type stripping, although it seems you already have the TypeScript language server hanging out in the editor for completion and docs and such, it’s not much extra challenge to get the VFS hooked up enough to show red squiggles in CodeMirror. I built a CodeMirror + typescript for the React docs site and it’s definitely machinery but not too bad maybe 600-800 LoC (never got merged, sad)

nixpulvis ・ 3 days ago

The video demonstrating the AI powered desktop app perfectly demonstrates my discomfort with AI. When asked to fix the mapping between county codes, it generated a lookup table itself.

HAVE FUN TESTING THAT. And when you don't, enjoy random countries being mixed up but it mostly working.

  • MrManatee ・ 3 days ago

    Yeah. I can believe that this will improve in the future, and this is a technology preview and all. But my takeaway from the video is that it is still too vibey for me to trust it. If I was coaching a junior data analyst at my company, and they wrote this code, I would happily give some feedback so that they can improve. With AI, I'd probably just do the analysis myself.

    In addition to the hardcoded lookup table, here are some other notes on the generated code:

    1. Silently assuming that the first page of results contains all of the data feels a bit fragile. If we're not going to bother with paging otherwise, I'd at least assert that the first page contains everything.

    2. Unlike the code comment claims, 2022 is not the latest year that has data available, 2023 is. The reason this is worrisome is not that the difference is massive, but because of methodological implications. It looks like the year 2022 came from "remembering" details from some other code that was analyzing this same data set instead of just looking at current version of the data set we're supposed to be analyzing.

    3. The code for removing the world aggregate doesn't actually work (although it doesn't matter for the map). The place where it says d.country.id !== "WLD" should be either d.country.id !== "1W" or d.countryiso3code !== "WLD" instead. Also, if it would actually be important to filter this then presumably it would also be important to filter out a bunch of other aggregates as well.

    4. The text says "darker colors indicating higher life expectancy", which is pretty much the opposite of how I would describe this color scheme.

    5. The analysis given is: "Notice how certain regions tend to cluster together in terms of life expectancy, reflecting similar economic, healthcare, and social conditions". This is such a bland take. It feels like something I could have written before seeing the data. I would try to encourage the analyst to look for something more concrete and more interesting.

    6. The actual thing that immediately pops out from the map is that the Central African Republic is a massive outlier with a life expectancy of 18.8 years, whereas every other country is over 50. This doesn't seem plausible. I would do something about that just so that it doesn't mess up the colors of the entire map.

simonw ・ 3 days ago

The Observable Desktop macOS app is a 11.8MB download which decompresses to a 23.9MB application. In a world filled with 200+MB Electron apps this is a very refreshing change!

lvl155 ・ 3 days ago

I wish Bostock et al got paid. This is literally the pain of doing open source for some. You basically watch as other people get rich off of your life’s work.

That rambling aside, I like the idea of having a desktop app. Not sure where this notebook fits in with all their other products. They have the Framework which is actually pretty useful.

  • gaws ・ 3 days ago

    > I wish Bostock et al got paid.

    They do get paid. Observable sells team and enterprise licenses: https://observablehq.com/pricing

    • simantel ・ 3 days ago

      Yeah Observable has raised $45M, I imagine the ELT is paid well.

  • jeffbee ・ 3 days ago

    Who parks a Porsche in front of a fancy Pacific Heights (San Francisco) mansion with the license plate "D3JS"?

    • lvl155 ・ 3 days ago

      This is why I come to HN. Good for Bostock if true!

    • swyx ・ 3 days ago

      source pls?

  • chaps ・ 3 days ago

    Say more?

kragen ・ 3 days ago

I've found Bostock Observable to be a fascinating and inspiring platform since the outset in my limited experiments with it. I wish they'd give it a name that isn't a well-known design pattern that it implements, because I end up having to call it "you know, Bostock's thing" in conversation.

However, I definitely don't want to have to type "<script id="2" type="module" pinned>" in order to calculate 1 + 2, and I don't want to have to scan past that in order to find the actual calculations. So I guess I'll have to keep being inspired by Bostock Observable Notebooks 1.0, which I can keep writing in GitHub-flavored Markdown, because it's free software and already checked into Git.

stared ・ 3 days ago

It is interesting to see different formats for notebooks - Jupyter uses JSON, RMarkdown (as you may have guessed) Markdown, and Observable - HTML.

th0ma5 ・ 3 days ago

They have or at least used to make their notebook format public, and I always thought if I ever see someone else adopt it then maybe the project will be worth looking at again. I think there could be better ways to capitalize on d3 but this one seems self limiting.

mritchie712 ・ 3 days ago

Observable has always demo'd well and looked cool, but I never understood the business application. Does anyone use it for work?

  • bsimpson ・ 3 days ago

    I work on a quantitative user research team, and we've found notebooks to be really useful for crunching and visualizing the data from experiments we've run.

    I work at Google and my UXR colleagues are more comfortable with Python than JS, so we use Google Colab, but I'd use Observable if those weren't true.

    • mritchie712 ・ 8 hours ago

      yeah, I know notebooks are used broadly (used them for years myself). I just haven't run into companies using Observable specifically in the wild.

svieira ・ 3 days ago

Out of curiosity, why `<notebook>` as an element and not a standard web-component `<observable-notebook>` or something like that if the goal is to go all-in on "normal" web development?

  • skybrian ・ 3 days ago

    They made the syntax look familiar so that standard editors (and LLM’s) will work, but this is still a custom file format that requires a build step to convert into an HTML page. See my other comment.

    The build tool is open source and unlike a Jupyter notebook, it’s easy to edit without a custom editor. So you don’t need their editor, but it will still be nicer to edit with their editor.

  • mbo ・ 3 days ago

    self-plug that you might be interested in: I built a thin wrapper around the Observable Runtime that allows you to more easily use it in a pure HTML context: https://maxbo.me/celine/

  • pphysch ・ 3 days ago

    I think the goal is explicitly not to go all-in on normal web development, to maintain some opportunity to monetize.

    D3.js started as/still is just a (collection of) JS libraries. Then Observable came along, and while it is a nice tool for tinkering with D3, it was not at all obvious how to then move your finished Observable D3 viz to a fully self-hosted, regular website (without paying a subscription to use their custom runtime or servers or whatever).

    Now I guess they realize they strayed way too far from "the web" and are back-tracking.

ModernMech ・ 3 days ago

It's pretty clear the future is not really here yet when the LLM can't even debug a simple table join issue on its own. The code to produce the table and chart should reasonably be like 2 lines of code with proper tool design. It looks like the LLM accomplishes it in 100 LOC and still can't manage to do it without significant human intervention.

And this is the take they published to sell the thing!

jwilber ・ 3 days ago

By far the biggest complaint I hear from Observable users is the non-standard JS syntax, so great to see them moving to regular JS.

RobinL ・ 3 days ago

This looks great. I love the idea behind notebooks and for a long time it was my favourite environment to program in. But slowly I stopped using them because it never quite felt like the code was entirely mine, and alternatives became easier due to llms. This looks like exactly the remedy I was hoping for. I'm excited to start using them again.

ujkhsjkdhf234 ・ 3 days ago

Observable's pricing is bit out of bounds for what they offer. It feels like a lot of these data startups are just looking for large companies that will unsuspectingly spend a ton of money on something they don't need.

  • gaws ・ 3 days ago

    Or get the golden ticket through an acquisition.

xyzzy_plugh ・ 3 days ago

This is really, really incredible stuff. I can't wait to replace a bunch of terrible BI dashboards.

I've seen and used so many amazing in-house dashboarding and reporting tools, and this suite blows them all out of the water. Can't believe this is all open source too.

I think this is a great way to slice up the product/feature set from a monetization perspective: hosting w/ collaboration is paid, but the tools are all free to use.

dunham ・ 3 days ago

I'd love to see support for inline TeX in the markdown. Back when I used observable on the web, I hacked up a `md2` string template to replace /\$(.*?)\$/g with the katex equivalent before calling `md`, but I don't know if there is a workaround in the desktop notebook application.

amdivia ・ 3 days ago

Would this be open source? I would really like to run this locally

For the time being I hacked something using Observable Framework [1], but still, it isn't the full set

1: https://observablehq.com/framework/

Edit: nvm just saw the notebook kit

blef ・ 3 days ago

I'm curious to understand why did you decided to develop a local app?

  • jeffbee ・ 3 days ago

    I don't know but it's nice if the local app doesn't have the 50MB file size limit that the hosted notebooks have.

kickout ・ 3 days ago

Big fan of the Notebooks. Hope the offline verison is hardened and available on windows soon enough.

These data viz are great dashboard builders but need a local/offline solution for my needs.

swyx ・ 3 days ago

i dont understand why Observable insists on topological execution. isn't out of order execution one of the biggest complaints about notebooks?

  • skybrian ・ 3 days ago

    Not exactly. Jupyter notebooks execute top-to-bottom at startup and then you can manually execute cells in whatever order you choose after that. I believe it’s the manual execution that gets people, because it can have inconsistent results. But just rerunning the entire file automatically (like reloading a web page) might be slow if some build steps are slow (they can read or write huge files), and some people have slow steps in their notebooks. They could use manual cell regeneration to intentionally avoid rebuilding cells, when stale data is acceptable.

    On the other hand, I don’t know if Observable Notebooks 2.0 even does incremental builds anymore? If not, it’s just about presentation order, similar to literate programming.

    Observable Framework had data loaders for creating static snapshots of data at build time. I don’t see any mention of how to handle data loading in Observable Notebooks 2.0.

  • jeffbee ・ 3 days ago

    Why should irrelevant nerd junk be at the top of my notebook? If the point of the notebook is to inform and educate the reader, that's a distracting limitation.

  • gampleman ・ 3 days ago

    Because it makes the reactive model work - and work quite efficiently. Cells don't re-run just because of edits, but also if the reader interacts with any of the interactive inputs.

chaps ・ 3 days ago

Wait is this different from canvas?