A better streams API is possible for JavaScript

> Obviously I'm gonna be biased, but I'm pretty sure my version is also objectively superior:

> - I can easily make mine from theirs

That... doesn't make it superior? On the contrary, theirs can't be easily made out of yours, except by either returning trivial 1-byte chunks, or by arbitrary buffering. So their proposal is a superior primitive.

On the whole, I/O-oriented iterators probably should return chunks of T, otherwise you get buffer bloat for free. The readv/writev were introduced for a reason, you know.

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.

If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior that stopped happening, such that the setup and execution steps started firing in the wrong order, due to a mock returning a number instead of a Promise.

I wonder if I still have that code somewhere…

I think the more generic stream concept is interesting, but their proposal is based on different underlying assumptions.

From what it looks like, they want their streams to be compatible with AsyncIterator so it'd fit into existing ecosystem of iterators.

And I believe the Uint8Array is there for matching OS streams as they tend to move batches of bytes without having knowledge about the data inside. It's probably not intended as an entirely new concept of a stream, but something that C/C++ or other language that can provide functionality for JS, can do underneath.

For example my personal pet project of a graph database written in C has observers/observables that are similar to the AsyncIterator streams (except one observable can be listened to by more than one observer) moving about batches of Uint8Array (or rather uint8_t* buffer with capacity/count), because it's one of the fastest and easiest thing to do in C.

It'd be a lot more work to use anything other than uint8_t* batches for streaming data. What I mean by that, is that any other protocol that is aware of the type information would be built on top of the streams, rather than being part of the stream protocol itself for this reason.

This is similar to how Clojure transducers are implemented: "give me the next thing plz." – https://clojure.org/reference/transducers

There is no such thing as Uint8Array<T>. Uint8Array is a primitive for a bunch of bytes, because that is what data is in a stream.

Adding types on top of that isn't a protocol concern but an application-level one.

I think the context that some other responders are missing is that in some functional languages, like Elixir, streams and iterators are used idiomatically to do staged transforms of data without necessitating accumulation at each step.

They are those languages versions of goroutines, and JavaScript doesn’t have one. Generators sort of, but people don’t use them much, and they don’t compose them with each other.

So if we are going to fix Streams, an implementation that is tuned only for IO-bound workflows at the expense of transform workflows would be a lost opportunity.

Other angles of critique & consideration already covered well by sibling commenters. One extra consideration (unrelated to streams, more general) is the API design & dev UX/DX:

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

  type Stream<T> = {
    next(): { done, value: T }
  }
  type Stream<T> = {
    next(): Promise<{ done, value: T }>
  }

> My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.

Writing all the code twice is cleaner in every implementation scenario I can envisage. It's very rare I want generalised flexibility on an API call - that leads to a lot of confusion & ambiguity when reading/reviewing code, & also when adding to/editing code. Any repetitiveness in handling both use-cases (separately) can easily be handled with well thought-out composition.

Your idea is flatten the UInt8Array into the stream.

While I understand the logic, that's a terrible idea.

* The overhead is massive. Now every 1KiB turns into 1024 objects. And terrible locality.

* Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway.

In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.

How do you send multiple sub-streams in parallel?

There's one more interesting consequence: you rid yourself of the feedback problem.

To see the problem let's create a stream with feedback. Lets say we have an assembly line that produces muffins from ingredients, and the recipe says that every third muffin we produce must be mushed up and used as an ingredient for further muffins. This works OK until someone adds a final stage to the assembly line, which puts muffins in boxes of 12. Now the line gets completely stuck! It can't get a muffin to use on the start of the line because it hasn't made a full box of muffins yet, and it can't make a full box of muffins because it's starved for ingredients after 3.

If we're mandated to clump the items together we're implicitly assuming that there's no feedback, yet there's also no reason that feedback shouldn't be a first-class ability of streams.

In the language I've been working on for a couple months, Eidos, streams are achieved through iterators as well. It's dead simple. And lazy for loops are iterators, and there is piping syntax. This means you can do this (REPL code):

  >> fn double(iter: $iterator<i32>) {
    return *for x in iter { $yield( x * 2 )}
  }

  >> fn add_ten(iter: $iterator<i32>) {
    return *for x in iter { $yield( x + 10 )}
  }

  >> fn print_all(iter: $iterator<i32>) {
    for x in iter { $print( x )}
  }

  >> const source = *for x in [1, 2, 3] { $yield( x )}

  >> source |> double |> add_ten |> print_all
  12
  14
  16

Yeah that problem you have is pretty much what I'm offering a solution to. It's the same thing you're already doing but more robust.

Also I'm curious why you say that generators are far too slow. Were you using async generators perhaps? Here's what I cooked up using sync generators: https://github.com/bablr-lang/stream-iterator/blob/trunk/lib...

This is the magic bit:

  return step.value.then((value) => {
    return this.next(value);
  });

I liked conartist6's proposal,

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

Engineers had a collective freak out panic back in 2013 over Do not unleash Zalgo, a worry about using callbacks with different activation patterns. Theres wisdom there, for callbacks especially; it's confusing if sometime the callback fires right away, sometimes is in fact async. https://blog.izs.me/2013/08/designing-apis-for-asynchrony/

And this sort of narrow specific control has been with us since. It's generally not cool to use MaybeAsync<T> = T | Promise<T>, for similar "it's better to be uniform" reasons. We've been so afraid of Zalgo for so long now.

That fear just seems so overblown and it feels like it hurts us so much that we can't do nice fast things. And go async when we need to.

Regarding the pulling multiple, it really depends doesn't it? It wouldn't be hard to make a utility function that lets you pull as many as you want queueing deferrables, allowing one at a time to flow. But I suspect at least some stream sources would be just fine yielding multiple results without waiting. They can internally wait for the previous promise, use that as a cursor.

I wasn't aware that generators were far too slow. It feels like we are using the main bit of the generator interface here, which is good enough.

It's news to me that anyone actually uses the web streams in node. I thought they were just for interoperability, for code that needs to run on both client and server.

In the repeater callback, you're both calling the stop argument and awaiting it. Is it somehow both a function and a promise? Is this possible in JS?

edit: I found where stop is created[1]. I can't say I've seen this pattern before, and the traditionalist in me wants to dislike the API for contradicting conventions, but I'm wondering if this was designed carefully for ergonomic benefits that outweigh the cost of violating conventions. Or if this was just toy code to try out new patterns, which is totally legit also

[1]: https://github.com/repeaterjs/repeater/blob/638a53f2729f5197...

Off topic - But just wanna say - Love the cheat code! 30 Lives added :-) Nostalgia runs deep with that code. So deep - in fact, that I sign many of my emails off with "Sent by hitting Up, Up, Down, Down, Left, Right, Left, Right, B, A"

I'm fairly sure it's not Promises that are actually the heavy part but the `await` keyword as used in the `for await` loop. That's because await tries to preserve the call stack for debugging, making it a relatively high-level expensive construct from a perf perspective where a promise is a relatively low-level cheap one.

So if you're going to flatten everything into one stream then you can't have a for loop implementation that defensively awaits on every step, or else it'll be slooooooooow. That's my proposal for the change to the language is a syntax like

  for await? (value of stream) {
  }

It avoids the overhead of Promises, so I can imagine that this would be quite useful if you know that blocking the thread is fine for a little while (e.g. in a worker).

Observables has moved to WHATWG [1] and been implemented in Chrome, although I don't know if the other browsers have expressed any interest (and there's still some issues [2] to be worked through).

But Observables really do not solve the problems being talked about in this post.

[1] https://github.com/WICG/observable [2] https://github.com/WICG/observable/issues/216

There's always a comment like this in most discussions about javascript.

> WebAssembly failed to keep some of its promises here

classic case of not using an await before your promise

Where can I find these not kept promises?

As wonky as JS is I really like it. Typescript has done such a good job at making it fun to use.

I wonder if you can get most of the benefit BYOB with a much simpler API:

    for await (const chunk of stream) {
        // process the chunk
        stream.returnChunk(chunk);
    }

(Lately I’ve been thinking that a really nice stream or receive API would return an object with a linear type so that you must consume it and possibly even return it. This would make it impossible to write code where task cancellation causes you to lose received data. Sadly, mainstream languages can’t do this directly.)

Heh, I was using emdashes and tricolons long before LLMs appropriated the style but I did let the agent handle some of the details on this. Honestly, it really is just easier sometimes... Especially for blogs posts like this when I've also got a book I'm writing, code to maintain etc. Use tools available to make life easier.

What was it specifically about the style that stood out as incongruous, or that hindered comprehension? What was it that made you stumble and start paying close attention to the style rather than to the message? I am looking at the two examples, and I can't see anything wrong with them, especially in the context of the article. They both employ the same rhetorical technique of antithesis, a juxtaposition of contrasting ideas. Surely people wrote like this before? Surely no-one complained?

The idea is well articulated and comes across clear. What’s the issue? Taking a magnifying glass to the whole article to find sentence structure you think is “LLM-slop” is an odd way to dismiss the article entirely.

I’ve read my fair share of LLM slop. This doesn’t qualify.

cloudflare does seem to love ai written everything

You’ve got it backwards: LLMs were trained on human writing and appropriated our style.

> high-performance data processing tools in JS

I may be naive in asking this, but what leads someone to building high perf data tools in JS? JS doesn't seem to me like it would be the tool of choice for such things

but instead of trying to solve that, this api is just like “too hard no one uses it let’s forget about it”.

right now when i need to wrangle bytes, i switch languages to Golang. it’s easy gc language, and all its IO is built around BYOB api:

interface Reader { read(b: Uint8Array): [number, Error?] }

you pass in your own Uint8Array allocation (in go terms, []byte), the reader fills at most the entire thing, and returns (bytes filled, error). it’s a fully pull stream API with one method at its core. now, the api gets to be that simple because it’s always sync, and blocks until the reader can fill data into the buffer or returns an error indicating no data available right now.

go has a TeeReader with no buffering - it too just blocks until it can write to the forked stream.

https://pkg.go.dev/io#TeeReader

we can’t do the same api in JS, because go gets to insert `await` wherever it wants with its coroutine/goroutine runtime. but we can dream of such simplicity combined with zero allocation performance.

UDP is a protocol, not an API

We're too busy building products while waiting for the perfect system to arrive.