Show HN: Boulette - Protect you from yourself (even as root).

I’ve tried using iTerm2’s automatic profile switching feature to adjust the theme depending on the connection, but I’ve never been able to get it to work reliably.

Double deep environment protection. True prod servers can't be deployed to except from special terminal which only appears visible to the ProdGlasses(TM). If you are wearing ProdGlass(TM) you receive mild brain wave stimulation which is collected for "product control purposes"(TM) future product development(patent pending).

I have iTerm set up to change the background colour of a shell based on whether it's local, docker, staging or production, and on top of that prompt colours change depending on how privileged the user is. Without all this, I'm sure I'd be making terrible mistakes on a regular basis.

If it's Linux, you can bind-mount the rootfs read-only and chroot into it:

    mahler ~ # mkdir /roroot
    mahler ~ # mount -o bind,ro / /roroot
    mahler ~ # chroot /roroot
    mahler / # passwd
    Enter new password: 
    Re-type new password: 
    passwd: Authentication token lock busy
    passwd: password unchanged

I've used this command to achieve that in the past:

    sudo systemd-nspawn --directory=/ --read-only --ephemeral --volatile=yes

I believe this specific set of options relies on (BTRFS) file system snapshots for performance. It's possible that you can get it to work on non-BTRFS systems by providing another combination of command line variables, but the default setting is to copy the file system tree to a temporary path.

You can also pass parameters like --volatile=state (so you can write to /var) and --volatile=overlay (so you can "write" to state, but all changes are discarded after the container exits). --volatile=state is useful for extracting data from a temporary read-only system, --volatile=overlay is useful for running tools that crash if they run on a read-only filesystem.

I've used seccomp in the past to create a read-only root.

I created a seccomp DSL to make this kind of stuff easier [0] (an example of dropping network access is at [1])

[0] https://chiselapp.com/user/rkeene/repository/bash-drop-netwo...

[1] https://chiselapp.com/user/rkeene/repository/bash-drop-netwo...

Can you create a group and set a sticky bit for ACLs for your root for the filesystem with read permissions for that group? That way all files created under the filesystem root will have read permissions for that user, but you cannot modify anything.

Can you create a second read-only mount of root?

Maybe this would help:

https://serverfault.com/questions/136515/read-only-bind-moun...

MicroOS has something in that spirit I think ? Maybe I'm wrong tough, don't even know if MicroOS is usable in host

> Having proper access control would be better.

Some of us live on the "granted" side of that access control. I'm an SRE; I have access to a fair number of things. I worry about making dumb mistakes like that in the article due to sleep deprivation. The warning here might very well be a warning that you're about to do something that you are allowed to do.

In the specific example … sometimes rebooting a production machine is what needs doing. Ideally, there's a second VM, ideally, the services on the machine would gracefully drain, ideally, it wouldn't matter. Ideally. But the reality is that often they don't, whether because of developer laziness, incompetence, some ugly confluence of bullshit bugs that conspire to prevent sanity, etc., more often than not I want to be careful even when the process/theory would like to say "this is a fully automated & completely safe operation" — and half-asleep is not careful, and accidental allegedly safe commands while half asleep is where Murphy likes to rear his head.

I've definitely used techniques like this. Red prompts that scream "You're SSH'd into PROD dummy" also come to mind. I keep my kubectl context fixed to "local laptop" so that I am forced to always type out --context the_production_cluster when I want that.

Temet nosce.

The idea is to protect you from yourself. You already have root access to the production server, because you are the admin and you need it to do your job.

Fun followup: the image is from the French movie "Le dîner de cons", where the character depicted says "Oh la boulette", after making a gaffe.

When you work at a bank and have multiple production boxes, this is pretty typical

I mean, you're probably going to need to give the reboot rights to someone. For instance, the people in charge of applying kernel updates. And those people will, at this point, be prone to making a boulette.

I use that in combination with base16-shell [0] to set different color profiles as a protection.

[0] https://github.com/chriskempson/base16-shell

You're not really supposed to type it everytime. Just declare an alias prefixed with boulette replacing the actual risky commands.

If you're in a mental state where you can shutdown a prod server while thinking you're on a dev one, you certainly won't think to type "wtf command", no matter "wtf" actually is.

For what it’s worth, the name is funny and easy to remember to French-speaking people.

Probably to avoid mistakes such as `boulette some command || some other command`, which would not happen with `boulette "some command || some other command"`.

It just didn't cross my mind! You've got a point here! I'll rethink the argument parsing.

Boulette isn't only for safeguarding "shutdown", it is for safeguarding what needs to be. This is to prevent yourself from some harmeful reflexes.

I my case, I use atuin a lot and automatically retype long commands and sometimes as root.

The regularity of this actions tend to lower my attention, but the fact that many users depend on what I do on big machines and with lots of priviledges isn't less true.

Danger comes from getting used to it.

This for the case where you think you type it locally but are actually typing it remotely. But I’d personally never type this locally so this use case is not convincing at all for me.

Not enough, because those are just wrapper scripts that call systemd.

To be fair, the specific use case mentioned on the github is typing "shutdown -h now" into the wrong terminal window, and the way it's solved is quite clever: it asks you to enter the hostname of the system you're actually trying to shutdown.

This is something that could conceivably happen to people who are properly trained. This also means it doesn't fall prey to the usual "do you really want to do this? yes - no" prompt, where you just get used to automatically hitting "yes." Even if you habitually enter the hostname, it'll be wrong if you execute the command in the wrong terminal.

Framing it as an "idiots vs not idiots" is the wrong way to think about it.

When you are working with the PTO you know you are working with the PTO. You are standing next to the tractor. The PTO should always be handled with the understanding that it can be dangerous. Here people are doing something mundane, everyday and not dangerous (shutting down their own laptop) and suddenly it becomes dangerous because they mixed up the terminal they typed their command in.

It is as if sometimes your alarm clock would be replaced with a PTO. Your alarm beeps, you reach out to turn it off as always and bam your hand is gone. That is the situation we are talking about here.

> I think because we have internalized an owner/management point of view that workers are fungible and training is a waste

This is not something you can train out of people. Your best trained, and most skillful operator can do this mixup. This is not that someone doesn't know what "sudo shutdown -h now" does. They do very much know it. Like the palm of their hand. What they don't notice is that they are typing it into the wrong terminal.

It’s the “Do you want to share XXX” popup all over again. Once you get used to see it, it will be a part of the shutdown command (to take the example in the readme).

As a counterpoint, aircraft designers have learnt through bitter experience that interface design is extremely important, as in stressful situations, or even due to a momentary lapse in attention, even a very highly trained, careful individual can make a fatal mistake. Quite frankly the only reason tractors don't get the same treatment is that generally the operator only kills themselves, not a few hundred other people.

(This isn't to say that training is useless, just that more training isn't the best and only solution to all problems, nor will a lack of careful interface design magically create more capable operators)

> The number of software systems I've seen designed to allow the clueless to bumble their way through operations is much higher than the number of companies I've seen that attempts to train better operators.

I agree, and that's a problem. Now with LLM, we're training everybody to not know how to find information by themselves (we call that "prompt engineering" so it cannot be bad, right? /s).

For anyone wondering how this works, molly-guard uses package diversions to actually rename the shutdown/reboot/... binaries and replace them with molly-guard: https://git.launchpad.net/ubuntu/+source/molly-guard/tree/de...

This is a cool ability of dpkg that is not well known. You can use it without making a package as well.

Yes! Molly guard is genius! Unfortunately I couldn't get it to work on nixos which is not FHS-complient!

You can set the challenge type to hostname as well and some others. The only pb with Molly-guard is that I can't have it working properly on my system.

I don’t understand: you say one can do this without an external thing, and then promote your own external thing?

> i don't need anything reminding me that i'm in a remote shell session: i know because it lags.

It doesn’t if you use a tool like Mosh [1] (I’m not affiliated)

[1]: https://mosh.org/