(Let me start clarifying that this is not at all a criticism of the author)
I am usually amused by the way really competent people judge other's context.
This post assumes understanding of:
- emacs (what it is, and terminology like buffers)
- strace
- linux directories and "everything is a file"
- environment variables
- grep and similar
- what git is
- the fact that 'git whatever' works to run a custom script if git-whatever exists in the path (this one was a TIL for me!)
- irc
- CVEs
- dynamic loaders
- file priviledges
but then feels important to explain to the audience that:
>A socket is a facility that enables interprocess communication
That feels like part of why some juniors are so confident while more senior engineers are plagued with self-doubt.
Juniors know how much they have learned whereas a 10+ year senior (like the author) forget most people don't know all this stuff intuitively.
I still will say stuff like "yeah it's just a string" forgetting everyone else thinks a "string" is a bit of thread/cord.
Well coached juniors run through brick walls
Scrapping them off the wall is not pleasant, though. But throw enough of them at it, I guess the wall eventually goes down too.
I forget that people think strings are different from sequences of bytes.
I didn't know that at some point, then I knew that and found it obvious, and now I don't know it again.
Strings are very very not sequences of bytes. Strings are a semantic thing. There may be a sequence of bytes in some representation of a particular string, but even then those bytes are not enough to define a string without other stuff. An encoding, at the very least. But even then, there are many things that could be described as a "string". A sequence of code points, perhaps? Or scalar values? Grapheme clusters?
Not to mention that you may not even have a linear sequence of bytes at the bottom level. You might have a rope (cons cell), or an intern pointer, or...
This is a profoundly stupid kind of argument. There isn't even an objective truth you could conceivably convince someone of. There's just how you're choosing to use the word in conflict with a preexisting convention, which marks you as part of some social group, just like "this slaps", "skibidi", "rad", or "whenever". The preexisting convention isn't some apprehension of objective truth either. It's just an arbitrary tradition, like the meaning of any word.
People who are using the word in the older sense are usually not mistaken. At worst, they're your political enemies, but often they aren't even that; they just have experiences you don't. Attempting to persuade them, as you are doing, can only have the effect of further narrowing your intellectual horizons—even in the unlikely case that you are successful, but especially in the far more common case where they try to avoid you after that.
I recommend more curiosity and less crusading.
(In the rare case where someone is mistaken, it's sufficient to say "I meant a Unicode string" or "but we're iterating over codepoints, not bytes," but such mere clarification is not what you're up to.)
Check out both how Python strings are implemented and the string type’s semantics in the language.
Strings are sequences of bytes only in the sense that everything stored in memory is a sequence of bytes. The semantics matter far more, and they aren’t the same as a sequence of bytes.
Also many languages make strings immutable and byte arrays mutable.
I think it's bad to attempt to redefine an established term in this way, but anyway people who use that established definition are not merely fools who lack your wisdom; see https://news.ycombinator.com/item?id=46086919.
> I recommend more curiosity and less crusading.
That is wonderfully ironic.
Anyway, coming from a C background, sure, strings are kind of just sequences of bytes. For people coming from other backgrounds, they'll have different understandings of what a string is (probably more based on semantics of the language they learnt first than on the underlying representation in memory). I'm not trying to persuade you of one definition or another. Nor am I redefining the meaning of a string, as it's clearly subjective by background and/or by context.
To that end, take my point as merely "you need to know the context", and I happen to believe the context that matters is the semantics of the programming language you're using (as opposed to the underlying representation of an instance of the type in memory).
My comments are also for the benefit of the many folks (particularly junior members of our community) that perhaps don't have exposure to this way of looking at things.
I mean, all data is just binary in the end.
In most programming languages strings contains more semantics than just sequences of bytes.
For example in Rust all Strings are utf-8, so Rust strings are binary sequences but not all binary sequences can be Rust strings.
As a blogger who makes similar assumptions, I think we depend on how a lot of us from that time "grew up" similarly. Sockets came to relevance later in my career compared to everything else listed here.
That might be part of it, yes.
As someone younger, ports and sockets appeared very early in my learning. I'd say they appeared in passing before programming even, as we had to deal with router issues to get some online games or p2p programs to work.
And conversely, some of the other topics are in the 'completely optional' category. Many of my colleagues work on IDEs from the start, and some may not even have used git in its command line form at all, though I think that extreme is more rare.
git was released in 2005.
>The term socket dates to the publication of RFC 147 in 1971, when it was used in the ARPANET. Most modern implementations of sockets are based on Berkeley sockets (1983), and other stacks such as Winsock (1991).
I read RFC 147 the other day, and it turns out that by "socket" it means "port number", more or less (though maybe they were proposing to also include the host number in the 32-bit "socket", which was quietly dropped within the next few months). Also Berkeley sockets are from about 01979, which is a huge difference from 01983.
Wikipedia is pretty convinced Berkeley sockets are from 1983[1]. Here's another site saying the same thing[2]. Do you have a source saying 1979?
[1] https://en.wikipedia.org/wiki/Berkeley_sockets
[2] https://medium.com/theconsole/40-years-of-berkeley-sockets-8...
https://gunkies.org/wiki/4.1_BSD says 01981. I was wrong; 3BSD in 01979 did not have sockets, and neither did 4BSD in 01980. Joy and Fabry's proposal to DARPA in August 01981 https://www.jslite.net/notes/joy2.pdf proposes:
> Initially we intend to add the facilities described here to UNIX. We will then begin to implement portions of UNIX itself using the IPC [inter-process communication] as an implementation tool. This will involve layering structure on top of the IPC facilities. The eventual result will be a distributed UNIX kernel based on the IPC framework.
> The IPC mechanism is based on an abstraction of a space of communicating entities communicating through one or more sockets. Each socket has a type and an address. Information is transmitted between sockets by send and receive operations. Sockets of specific type may provide other control operations related to the specific protocol of the socket.
They did deliver sockets more or less as described in 4.1BSD later that year, but distributing the Unix kernel never materialized. The closest thing was what Joy would later bring about at Sun, NFS and YP (later NIS). They clarify that they had a prototype working already:
> A more complete description of the IPC architecture described here, measurements of a prototype implementation, comparisons with other work and a complete bibliography are given in CSRG TR/3: "An IPC Architecture for UNIX."
And they give a definition for struct in_addr, though not today's definition. Similarly they use SOCK_DG and SOCK_VC rather than today's SOCK_DGRAM and SOCK_STREAM, offering this sample bit of source:
CSRG TR/3 does not seem to have been promoted to an EECS TR, because I cannot find anything similar in https://www2.eecs.berkeley.edu/Pubs/TechRpts/. And they evidently didn't check their "prototype" socket implementation in to source control until November 01981: https://github.com/robohack/ucb-csrg-bsd/commit/9a54bb7a2aa0...s = socket(SOCK_DG, &addr, &pref);In theory that's four months after the 4.1BSD release in http://bitsavers.trailing-edge.com/bits/UCB_CSRG/4.1_BSD_198..., linked from https://gunkies.org/wiki/4.1_BSD, which does seem to have sockets in some minimal form. I don't understand the tape image format, but the string "socket" occurs: "Protocol wrong type for socket^@Protocol not available^@Protocol not supported^@Socket type not supported^@Operation not supported on socket^@Protocol family not supported^@Address family not supported by protocol family^@Address already in use^@Can't assign requested address^@".
This is presumably compiled from lib/libc/gen/errlst.c or its moral equivalent (e.g., there was an earlier version that was part of the ex editor source code). But those messages were not added to the checked-in version of that file until Charlie Root checked in "get rid of mpx stuff" in February of 01982: https://github.com/robohack/ucb-csrg-bsd/commit/96df46d72642...
The 4.1 tape image I linked above does not contain man pages for sockets. Evidently those weren't added until 4.2! The file listings in burst/00002.txt mention finger and biff, but those could have been non-networked versions (although Finger was a documented service on the ARPANet for several years at that point, with no sign of growing into a networked hypertext platform with mobile code). Delivermail, the predecessor of sendmail, evidently had cmd/delivermail/arpa-mailer.8, cmd/delivermail/arpa.c, etc.
That release was actually the month before Joy and Fabry's proposal, so perhaps sockets were still a "prototype" in that release?
The current sockaddr_in structure was checked in to source control as a patch to sys/netinet/in.h on November 18, 01981: https://github.com/robohack/ucb-csrg-bsd/commit/b5bb9400a15e...
Kirk McCusick's "Twenty Years of Berkeley Unix" https://www.oreilly.com/openbook/opensources/book/kirkmck.ht... says:
> When Rob Gurwitz released an early implementation of the TCP/IP protocols to Berkeley, Joy integrated it into the system and tuned its performance. During this work, it became clear to Joy and Leffler that the new system would need to provide support for more than just the DARPA standard network protocols. Thus, they redesigned the internal structuring of the software, refining the interfaces so that multiple network protocols could be used simultaneously.
> With the internal restructuring completed and the TCP/IP protocols integrated with the prototype IPC facilities, several simple applications were created to provide local users access to remote resources. These programs, rcp, rsh, rlogin, and rwho were intended to be temporary tools that would eventually be replaced by more reasonable facilities (hence the use of the distinguishing "r" prefix). This system, called 4.1a, was first distributed in April 1982 for local use; it was never intended that it would have wide circulation, though bootleg copies of the system proliferated as sites grew impatient waiting for the 4.2 release.
rcmd, rexec, rsh, rlogin, and rlogind were checked into SCCS on April 2, 01982. At first glance, this socket code looks like it would compile today: https://github.com/robohack/ucb-csrg-bsd/commit/58a2fc8197d0...
Telnet, also using sockets, had been checked in earlier on February 28: https://github.com/robohack/ucb-csrg-bsd/commit/0dd802d6a649...
Wow, thanks for the deep dive!
I haven't even realized that while I was reading the article, but it is amusing!
Though one explanation is that I think for the other stuff that the writer doesn't explain, one can just guess and be half right, and even if the reader guesses wrong, isn't critical to the bug — but sockets and capabilities are the concepts that are required to understand the post.
It still is amusing and I wouldn't have even realized that until you pointed that out.
I found it interesting that they know how to use strace, but not how to list open files held by a process which to me seems simpler. Again, not criticism just an observation and I enjoyed the article
Given the "(hi Julia!)" immediately after the strace shenanigans, I interpreted this as a third-party hint; the author most likely had not used strace before.
The author is both an example of and an example for how we can get caught in "bubbles" of tools/things we know and use and don't, and blog posts like this are great for discovery (I didn't know about git invoking a binary in the path like his "git re-edit", for example, until today).
I discovered that by accident, I had a script called git-pr that opened a pull request with github using the last commit message and then pushed it to slack for approval. I was trying to rewrite it to add a description and wondered why "git pr" pushed an empty message to slack
I assure you that the author has been using strace since it was invented in the early 1990s.
When are you planning to run lsof? Emacsclient exits immediately when it can't find the socket.
I found that specific clarification useful while everything else was easy to follow.
It’s not that I was unaware that’s how Unix worked here, just that I rarely think of sockets in that context.
It's just poor editing. The author started with one idea about the audience, before he realized that it might not be possible to write this article for that audience.
Then he didn't go back to clean it up afterwards.
- [deleted]
All of the things you listed are ops topics. But sockets are a programming concept.
I would expect a person with 10+ years of Unix sysadmin experience — but who has never programmed directly against any OS APIs, “merely” scripting together invocations of userland CLI tools — to have exactly this kind of lopsided knowledge.
(And that pattern is more common than you might think; if you remember installing early SuSE or Slackware on a random beige box, it probably applies to you!)
congratulations on being the highest emdash users of hacker news, I totally get why now.
Most people these days are using http and don’t need to touch sockets. (Except for the people implementing http of course).
To be fair, it does link the CVE, so if you don't know what a CVE is, you can click the link.
I agree that it's amusing.
Yep. The Curse of Knowledge is a real thing.
I mean, the title is a quote from Buckaroo Banzai. Lack of context is part of the fun!