CrowdStrike effectively bricked windows, Mac and Linux today.
Windows machines won’t boot, and Mac and Linux work is abandoned because all their users are on twitter making memes.
Incredible work.
this post was submitted on 19 Jul 2024
2024 points (99.1% liked)
linuxmemes
21355 readers
1322 users here now
Hint: :q!
Sister communities:
Community rules (click to expand)
1. Follow the site-wide rules
- Instance-wide TOS: https://legal.lemmy.world/tos/
- Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
- Understand the difference between a joke and an insult.
- Do not harrass or attack members of the community for any reason.
- Leave remarks of "peasantry" to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
- Bigotry will not be tolerated.
- These rules are somewhat loosened when the subject is a public figure. Still, do not attack their person or incite harrassment.
3. Post Linux-related content
- Including Unix and BSD.
- Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of
sudoin Windows. - No porn. Even if you watch it on a Linux machine.
4. No recent reposts
- Everybody uses Arch btw, can't quit Vim, and wants to interject for a moment. You can stop now.
Please report posts and comments that break these rules!
Important: never execute code or follow advice that you don't understand or can't verify, especially here. The word of the day is credibility. This is a meme community -- even the most helpful comments might just be shitposts that can damage your system. Be aware, be smart, don't fork-bomb your computer.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Gonna try my best here:
Crowdstrike is an anti-virus program that everyone in the corporate world uses for their windows machines. They released a update that made the program fail badly enough that windows crashes. When it crashes like this, it tries to restart in case it fixes the issue, but here it doesn't, and computers get stuck in a loop of restarting.
Because anti-virus programs are there to prevent bad things from happening, you can't just automatically disable the program when it crashes. This means a lot of computers cannot start properly, which means you also cannot tell the computers to fix the problem remotely like you usually would.
The end result is a bunch of low level techs are spending their weekends manually going to each computer individually, and swapping out the bad update file so the computer can boot. It's a massive failure on crowdstrikes part, and a good reason you shouldn't outsource all your IT like people have been doing.
It's also a strong indicator that companies are not doing enough to protect their own infrastructure. Production servers shouldn't have third party software that auto-updates without going through a test environment. It's one thing to push emergency updates if there is a timely concern or vulnerability, but routine maintenance should go through testing before being promoted to prod.
It's because this got pushed as a virus definition update and not a client update bypassing even customer staging rules that should prevent issues like this. Makes it a little more understandable because you'd want to be protected against current threats. But, yeah should still hit testing first if possible.
If a company disguises a software update as a virus definition update, that be a huge scandal and no serious company should ever work with them again…are you sure that’s what happened?
It wasn't a virus definitions update. It was a driver update. The driver is used to identify and block threats incoming from wifi and wired internet.
The "Outage" section of the Wikipedia article goes into more detail: https://en.wikipedia.org/wiki/2024_CrowdStrike_incident#Outage
Ah, was a bit off. The update disregarded update controls per reddit and I must have misunderstood what exactly the channel update did. I know for the sensors you can set how closely you want to track current releases but I guess the driver update is not considered under those rules. I use CrowdStrike in my day to day but not from the administrative side, sorry for the misinformation. Thanks for the details Gestrid.
100% agree. I haven't been on the backend of managing crowdstrike so I don't know if this is a option, but running a wsuz server and manually weeding out bad updates was such an improvement over rawdogging windows updates.
Yeah but testing costs money and CEO needs new private island, his old one is too small.
And the kids on the island are too old now