Happened to me once. Had a little Pi at my parent's house and that was a nice excuse to visit them.
post
even worse. I regularly have to get up out of my chair and go down 2 stairs.
Also this took a while to find, but : https://sourceforge.net/p/shorewall/svn/HEAD/tree/branches/4.2/Samples/one-interface/shorewall.conf
ADMINISABSENTMINDED=Yes
Is an actual setting in the config for the (now apparently unmaintained) Shorewall Firewall software/tool for linux.
If I remember correctly, it always checks on firewall rule changes if there is an active connection on port 22, and adds a special rule at the end to maintain that connection.
They don't build them like they used to anymore.
Before you make a change, do this in a screen-session:
sleep 300 && iptables-restore old_fw_rules.bak
permission denied
fuuuu
Found the debian user.
Yeah except it would be iptables-restore < old_fw_rules.bak
Fun fact: When you do iptables-save, you have to redirect the output if you want to save it to a file. But when you use iptables-restore, you don't need to pipe it back in, you can just use the filename!
Almost the same thing happened to me. I accidentally fucked up the internet connection in my home while in Japan, and I had to video call my mom to have her fix it. It was a pain for both of us, but thankfully it went rather smoothly. Thank you mom!
Do you mind explaining the details? I’m trying to learn as much as possible!
Most corporate network devices like Cisco will reset their config to the one written in memory when they lose power.
So in that case, just unplug and replug them to restore to previous config.
Just make sure you write your new config to memory or it will reset when there is ever a power failure.
So I connected through ssh back home to fiddle with the router settings, and in the PPPoE settings (where you set a pair of username and password that your router sends to the ISP such that the ISP knows you and knows what IP to assign to you) I made a typo, and apparently that instantly killed the internet connection at home and also for me. I had to call my mom to instruct her to fix the typo in the username. TBH I don't know that much about PPPoE either, I only do it so that the ISP assigns us the same IP address every time.
What's really fun is hearing "oh shit" from the UPS maintenance tech followed by darkness and silence.
Doing this is a right of passage.
Classic.
Love Hetzner. If something like that were to happen to me they can hook up a remote console accessible through their web interface.
Since that happens to the best of us, I envision writing a wrapper script around {n,}pfctl that asks for confirmation upon detecting that you're logged in via ssh through a specific port AND detecting that the new rules would block that port.
I'll always be grateful for the firewalls like OpenWRT that will automatically revert any changes if you don't log back in after a few minutes (at least on the web interface). I'm not proud of how many times that's saved me.
This is the NetAdmin's problem. And he's got 3 ways to get into the datacenter, so he goddamn well better have an answer that doesn't involve airfare. Worst case, he's gotta use remote hands, but that would be embarrassing, and I'd not let him forget it. Nobody forgives me when I screw up a server cluster, so he gets no latitude when he takes a datacenter offline.
That the slrpnk.net admins in the picture?
They had a hardware failure but close enough
Would misusing the dd command be considered a hardware failure?
I try to remember to always open two SSH connections when altering iptables or the ssh config - just in case
this sounds like something chip from sales would do
Does it actually happen to people? All servers I worked with both had a back door (or two), and someone at the data centre (during work hours at least) you could contact in an emergency.
I guess some smaller companies might have simpler setups they self-host
Most data centers have some kind of service where you can request a KVM to be connected to the server. It's not instant as an actual human has to do so but a lot sooner than another human driving long distance. I guess in this case, it's a mid size company that is big enough to have multiple locations yet small enough to still manage to use on-premise infra instead of data centers.
iptables default DENY and flush the rules. Done by at least two people I know (then me) at the same company. Led to them moving the servers in-house and virtualizing some services to connect to the hypervisor. It does happen though.
This is precisely the problem that deploy-rs solves!
why is everything in rust now
It's easy to write, easy to build, produces lightweight and fast executables, and the type system is great. Why not rust?
Rust does not have an ABI. Everything is linked into the executables. I would not call them lightweight.
A standard Docker container with a NodeJS/PHP/Python app is usually around 200-300 MB (yes really), the OpenJDK JVM is around a hundred MB, but a fully statically compiled rust binary that doesn't even depend on libc is just a couple MB and can be deployed as a tiny distroless Docker container.
It's a lot heavier than your 8kb C++ executable but it's nothing compared to what is required to deploy anything else.
Oh, so it's inconvenient for GPL-circumventers, too? That just sounds better and better.
To me, it is mostly a real blocker for using it in some embedded Linux devices due to size constraints, otherwise I personally would be using it extensively.
Lol.
Just tailscale it and this will never happen again.
(Set the whole interface of tailscale0 as a trusted network)

all 65 comments