all 3 comments

sorted by: hot top controversial new old
[–] 15 points 1 day ago
[–] 2 points 20 hours ago

My grandma is more informed and on the ball than this clown.

  • source
  • [+] 1 point 20 hours ago*
    90 comments in c/politics@lemmy.world view that post
    [–] 125 points 2 days ago* (last edited 2 days ago) (19 children)

    So. One of the things DOGE got rid of was the CAB - Change Advisory Board. I'm pretty sure there were established Change Windows with approvals required and a documented test plan and rollback plan. Don't need those! That just gets in the way of business! Let's get rid of it and see how that goes!

    Yeah. We saw how it went, alright.

    Edit: I should have said it's apparent that DOGE got rid of CAB. Because this isn't the kind of thing you do with an Information System if you have a CAB.

  • source
  • hideshow 19 child comments
  • [–] 33 points 2 days ago (8 children)

    I work at a Fortune 500 company and we have CAB meetings all the time. It's a bit of a pain but a necessary part of maintaining stability in Production systems.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 6 points 1 day ago (5 children)

    I wonder if Tesla has one of these. I suspect not.

    CAB is the opposite of mine fat and break things.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 6 points 1 day ago (4 children)

    mine fat and break things

    Typo or not, I'm on board.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 4 points 1 day ago (3 children)

    I swear autocorrect is changing things after I type following words.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 1 day ago (1 child)

    I work for a health insurance company. We kept old systems going way longer than we should because costs were predictable. We reduced volatility and could throw more money at patient care. (Due to how the company was built, there’s no shareholders so we can’t hide the money there, and in our state excesses get refunded to payers every year).

    Change of guard, new boss says everything must be modern. The legacy maintenance costs are killing us. Not wrong but missing the point. They were predictable. We can manage predictable.

    They move everything to cloud shit with Microsoft dynamics and it fucking sucks. Shit doesn’t work half the time, calls keep dropping, systems are slow and unreliable, AI is everywhere it doesn’t need to be and work is of lower quality, higher cost, and takes more time/money. costs WILL explode once it’s finally working because we have willingly entered into a hostage situation with the vendor and what are we gonna do about it, move again? Security is horrifyingly bad. If you mention the words “packet capture” IT stops talking to you. They just keep buying new products to staple onto this mess that promise to solve their problems instead of hiring people who know what they are fucking doing and can say the word “no” to the c-suites.

    Spoke to someone the other day from a similar company, griping about the upgrades going very badly. They said they had upgrades they had to do too when the edict everything must be win11 came down. His companies IT dept actually fought the board and refused. They said they absolutely will not service win11 machines handling patient data due to security concerns. The board caved and funded switching that entire hospital system to Mac and upgrading on-prem systems where possible instead of clouding everything.

    Remember how everything Apple does is on a time delay? The ram cost surge didn’t clobber then as soon as everyone else. They got a killer deal on those minis.

    Once the migration was done, they spent about 30% less than expected and their downtime was minimal. IT was saying no because of the security implications. The board saw lower costs and not having to replace the entire IT dept. everybody is happy.

    I mentioned this to a supervisor. They were interested and asked what industry they were in. When I said healthcare, her eyes went wide.

    We wasted so much money on needless risk. That could have gone to fucking insulin or cancer treatments.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 22 hours ago

    Sounds a bit like my company (thankfully we rarely hold someone's life in our hands though). New leadership is obsessed with removing redundancies. In theory it's a nice idea but it often falls apart when the rubber meets the road. We're going all in on Microsoft products (started before the latest outage) and we're not just annoyed with the migrations we're quite pissed off at the features we're losing from our previous products. There's also data, propriety company information, that's straight up going to be purged from existence because most of us have neither the time or motivation to convert artifacts into the Microsoft equivalent. 6 months later when we get asked about that research effort from 2 years ago we'll just have to say, "What research?"

  • source
  • parent
  • [–] 21 points 2 days ago (6 children)

    It seems blowing up things for progress doesn't work all so well outside the rocket industry. I guess military, but only certain things should blow up there.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 13 points 2 days ago (5 children)

    It doesn't even work that well in the Rocket Industry, honestly. And Elon knows that! The Falcon 9 rocket is a stable 'release' that's not experimented on. Sure, Starship isn't, but Starship isn't yet a production rocket. He can ask Vulcan Centaur what happens when you have a glitch in a production model rocket. That's gonna be grounded until they find out what broke on the launch yesterday and how to fix it going forward!

  • source
  • parent
  • hideshow 5 child comments
  • [–] 8 points 2 days ago (4 children)

    The thing missing was that with new rocket designs you expect some booms, but you take that data and you find out why so the next boom isn't from the same thing. Which in the case of anything they've touched in the FAA or other places, is not what's happening. It's a free for all sponsored by the prevailing administration. There was not a goal to fix and improve, but to make a show of it (at a minimum, I'm sure conspiracies abound on the real purposes).

    I often see SpaceX as a separate thing from Musk, mainly because he's been smart enough to let them run themselves more than the other companies. And it shows.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 1 day ago (3 children)

    You actually want to do component-level testing instead of an all-up, so theoretically you find the thing that makes it go boom without it going boom.

    Failure to Launch is doing a large multi-part on how NASA insisting on doing all-ups with the shuttle (because it's easier to explain to media and budget dudes than component testing) was part of what led to Challenger happening. The issue with an all-up in rocketry without the component testing before means that when something goes boom, you're now doing a huge deep dive on production data from subcontractors of subcontractors of contractors, and whatever you can piece together from the exploded components.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 1 day ago (1 child)

    Good points, full building to fill isn't great, but the assumption is that heavy testing is done before that point, so the failure isn't a known one, but to learn a new factor to design for. The advantage now is the data retrieval, everything has a sensor or camera so it's far easier (still difficult) to find the cause. Whereas a big part of the Challenger aftermath was going off of media footage to find the spot that led to looking closer at the o-rings. There is also the issue that some were already telling them to not launch, but that was quieted down because of the focus and they didn't want to drag it out more (without looking, it had been delayed a few times I think). So that was more a production run that pushed past recommended limits.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 day ago

    There’s always pressure to ship! And it’s not like there’s a sudden cutoff between safe and not safe, nor was the o-ring failure an expected scenario. Do you really hold a critical launch because someone is being paranoid about temperature?

    Being somewhat facetious with that but it’s easy to see how the wrong decision was made and I know in would not have done better

  • source
  • parent
  • [–] 2 points 1 day ago

    With the shuttle, you had less than half a dozen ships, all hand built. Component testing would not have been as likely

    The SpaceeX approach is all about “mass production”. Everything is componentized, everything is made to scale. They can easily component test an engine, for example, because there are always more and it is somewhat independent of the ship

  • source
  • parent
  • [–] 4 points 2 days ago (1 child)

    Do you have references for that? I can't find anything that says DOGE eliminated CABs.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 12 points 2 days ago

    Oh you got me. I don't have any proof that CAB was gotten rid of...

    ...except the fact that they made an impactful change during working hours and spent hours trying to back it out with the system being down. I should have said that one of the things DOGE apparently got rid of was the CAB. Because CAB would NOT have let this happen without LOUD protestations.

  • source
  • parent
  • [–] 40 points 2 days ago (2 children)

    INCOMPETENCE. The answer is incompetence.

  • source
  • hideshow 2 child comments
  • [–] [S] 37 points 2 days ago (1 child)

    Apparently the plan was to modernize air traffic control by testing upgrades during Friday rush hour, then modernize accountability by blaming the contractor. If nobody can explain why the work happened then, the biggest system failure might be the decision making above the control tower.

  • source
  • hideshow 1 child comment
  • [–] 21 points 2 days ago (7 children)

    Wasn't it even on a Friday afternoon?

    Definitely FNGs. Nobody that's been in IT for 10 minutes updates shit on a Friday afternoon.

  • source
  • hideshow 7 child comments
  • [–] 7 points 2 days ago (3 children)

    You would also probably want to advise the pilots currently in the air before takeoff..

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 1 day ago

    Depends. I'd say that logic applies to the old IT people, ones who came in / had high salaries and valued their personal time with family etc. The people who implemented things like read only fridays.

    Younger workers, or people chasing money due to lower salaries/job insecurity though.... an issue translates as OT, and as an easy marker to point at for "what you do here" to people trying to cut jobs. Even the older workers know, when shit works smoothly people wonder what you do -- but when it breaks, they're all thankful you fix it.

    Similar mentality as I've heard with cops. They'll bust a kid on an inflated minor charge near the end of their shift, because the arresting officer is required to fill in the paper work, which translates as hours of OT pay.

    The whole point of read only Fridays, is that you know shit can go wrong, and you don't generally want to have a bunch of people doing unplanned OT on the weekend. It's based on a shared understanding that personal time is important, and that no one desperately needs/wants unplanned OT. So if enough people in the mix don't care so much about personal time, and/or want that extra cash... so long as you can deflect the problem as a fault with "decades old routers"....

  • source
  • parent
  • [–] 20 points 2 days ago (1 child)

    Watch him eventually admit.

    "grok thought it was a good idea to test on heavy traffic".

  • source
  • hideshow 1 child comment
  • [–] 1 point 1 day ago

    I guarantee, if Grok isn't lobotomized, it would have recommend a cautious and sensible approach. My local AI is smart enough to point out problems, as is Kagi's assistant.

    The members of the Trump Regime simply have the patience of a toddler, and none of the forbearance to handle being told "No."

  • source
  • parent
  • [–] 17 points 2 days ago

    Just another notch on the failure belt of this idiotic pedophile regime. They're already clear up to the buckle of this fuckin belt. Incompetent child molesters.

  • source
  • [–] 15 points 1 day ago

    "I'm just surface lying here. I never gave it too much thought, as I don't really know the patterns of "working for a living". ".

  • source
  • [–] 15 points 2 days ago

    Saw a post shortly after where Donnie specified 'I claim no responsibility for it', which probably means he told them to get it done before the weekend and blames the tech people for it failing.

  • source
  • [–] 15 points 1 day ago

    It's because he has no goddamn idea what he's doing.

  • source
  • [–] 14 points 1 day ago (1 child)

    If it were me, I would have started with the least used routes for rolling out, confirm that they are in good order after a couple months, slowly expand the rollout locations over time. All the while being administered and monitored by experts whose sole purpose is to assess everything - cost, tradeoffs, problems, improvements, personnel training, the whole enchilada.

    My approach isn't nearly good enough, yet Duffy is even below that questionable bar.

    🤨

  • source
  • hideshow 1 child comment
  • [–] 15 points 1 day ago

    There are so many ways to migrate. Migrating systems is very standard practice in software and there are many good options.

    "Make all traffic use the new system in the middle of the day" is among the worst available options.

  • source
  • parent
  • [–] 12 points 1 day ago (1 child)

    Maybe conservatives shouldn't be making important decisions for the public.

  • source
  • hideshow 1 child comment
  • [–] 12 points 14 hours ago

    This is what happens when you select employees by loyalty instead of proven competence

  • source
  • [–] 10 points 2 days ago

    But its the old system's fault. We need money to upgrade ... to the new things we installed that caused the problem.

  • source
  • [–] 10 points 1 day ago

    Because every aspect of the current American system is a clownshow entirely peopled by unqualified boot-lickers....

    Does this make me Sherlock?

  • source
  • [–] 9 points 1 day ago (3 children)
    As democracy is perfected, the office of president represents, more and more closely, the inner soul of the people. On some great and glorious day the plain folks of the land will reach their heart's desire at last and the White House will be adorned by a downright moron.
    

    H.L. Mencken, On Politics: A Carnival of Buncombe

  • source
  • hideshow 3 child comments
  • [–] 2 points 14 hours ago (2 children)

    "Democracy is by the people, of the people, for the people."

    Does that mean people support stupidity and pedophilia for electing Trump?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 13 hours ago
    If you can convince the lowest white man he's better than the best colored man, he won't notice you're picking his pocket. Hell, give him somebody to look down on, and he'll empty his pockets for you.
    

    Lyndon B. Johnson

    I can't say I understand Trump voters. I assume that most of them are just completely ignorant of a LOT of things. First, because they don't pay attention to politics, and second because the news sources they do occasionally catch are captured and closer to propaganda. I'm also reminded of Hanlons razor: Never attribute to malice that which can be sufficiently explained by stupidity. While I support trials and convictions and incarceration for those responsible for this massive con job, I also think it's vital to provide a path of forgiveness and grace to let the folks who fell for the con. It's socially devastating to admit that you fell for it, tons of people never report falling victim to confidence scams because of embarrassment. So somehow we have to give that THIRD of Americans some way to diplomatically save face before they will change their mind.

  • source
  • parent
  • [–] 8 points 1 day ago (5 children)

    Lol. Only fools test in production.

  • source
  • hideshow 5 child comments
  • [–] 14 points 1 day ago (3 children)

    Everyone has a test environment. Some people are lucky enough to have a separate environment for production...

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 2 days ago

    Smart people are not in charge should be correct answer.

  • source
  • [–] 5 points 1 day ago (1 child)

    I'm sure Claude thought it was a good idea. Everyone who knew better has likely been fired or "retired" long ago.

  • source
  • hideshow 1 child comment
  • [–] 4 points 2 days ago

    We just let a contractor have full access to a single point of failure. See? Not our fault!

    Fucking incompetent idiots.

  • source
  • [–] 9 points 2 days ago (1 child)

    The USA voted for mediocrity, this is what mediocrity looks like.

    A bunch of absolute idiots that are not even aware of how stupid they are, in positions of power because they were born with a silver spoon up their ass.

    This is exactly what MAGAtards complain about DEI not realizing DEI avoids this while their white-evangelical-men first policy is exactly what they supposedly hate so much

  • source
  • hideshow 1 child comment
  • [–] 6 points 1 day ago (2 children)

    Given the global nature of the industry, is it even realistic to wait for a 'down' period? I suppose for local traffic, which would be the majority at international hubs. Would be interesting to see the graphs.

  • source
  • hideshow 2 child comments
  • [–] 1 point 1 day ago

    If you don’t run your change on Friday, at the very least you have people available to deal with any issues, without paying overtime.

    But yes, I’m sure sometimes are quieter than ithers

  • source
  • parent
  • [–] 3 points 1 day ago

    He's another one of the know-nothings maga loves, since they sound as misinformed and clueless as they are, very tribal.

  • source
  • [–] 3 points 2 days ago (1 child)

    Maybe their AI told them that?

  • source
  • hideshow 1 child comment
  • [–] 6 points 2 days ago (4 children)

    One would think that very few people have access to the master update files and even less have full access to direct it to every computer. They're probably all based on sccm, so I'm kinda flabbergasted that enough computers took the plunge at once to close all traffic. Seems like maybe more market manipulation as a much easier answer as to how and why.

  • source
  • hideshow 4 child comments
  • [–] 3 points 1 day ago*

    Nah, my bet is on it being a combination of old hardware/infrastructure, and newly rolled out AI systems.

    The new ai systems they rolled out most likely had built in 'fail safes' to trigger ground stops in a variety of scenarios. I can easily imagine a 20-something year old vibe coder setting one of those conditions along the lines of "If there's a significant rise in data gaps/integrity loss, such that you are being 'blinded', trigger a ground stop". Rationale could've been something along the lines of it being an indicator of a potential hack/breach, alongside the general preference of protecting lives vs gambling on an AI with blind spots. That tools integrated into all the airports, with the same logic, and likely connected via cloud channels to centralized AI farms.

    If you accept that as plausible, then I'd go a step further and say it's plausible that the comments about network devices is potentially legit as a contributing factor. Older networking hardware would struggle to push through the amount of data a centrally-based AI control type system would be seeking. The way data gets routed through those infrastructure devices, would be as chopped up packets of data -- and the more overloaded a system gets, the more it drops individual packets of data. This either results in the computer needing to re-request the lost packet (increasing the load on the network even further), or it just tries to make do without the packet of data. Those lost/delayed packets add up over time, and start causing system faults.

    To the second note, airport networks are generally already pretty overloaded in this sense from what I understand, adding in AI-level data harvesting would absolutely cripple them -- and to the first point, if there's a 'threshold' of packet loss / data integrity issues that'd trigger a ground stop, you'd hit that threshold at a semi unpredictable time (though it'd happen as the load increases, which sorta makes sense that it happened at a busier time after the AI tool was integrated).

    The above seems to sorta 'fit' with most of the information I've heard so far at least.

    Tech companies are heavily leveraged in AI, and they're highly involved with the US Administration. So any fault related to AI will be downplayed significantly. Sorta like that Iranian schoolgirl bombing incident, which was apparently carried out due to bad information provided by Palantir's AI, paired with the US cutting a bunch of safeguards around military strikes -- if palantir says fire, press red button! Hegseth Strongk! .... those sorts of AI stories get downplayed, as they'd have a negative impact on stocks. AI killing us all is only a good investor pitch so long as people aren't seeing the corpses the AI companies are already making. So if you can blame anything else, be it "Maybe the Iranians bombed themselves?" or "copper wires", so long as it shifts blame away from AI, you can keep that hype train going.

  • source
  • parent
  • [–] 2 points 1 day ago

    Going by what they said, since that’s all we have, a new router can easily be misconfigured to take out a network, whether it’s publishing bad routes, flooding the net with verbose data, or not throttling the fiber side down to what the copper side could accept. Thats why you have redundancy and very conservative process.

  • source
  • parent
  • [–] 2 points 1 day ago

    It's the exact same answer as to the previous question: "why deploy a data hungry system (SMART) on an infrastructure that was guaranteed not able to support the load (old copper network)?"

    Because incompetent fucking idiots make loads of stupid decision all the time.

  • source
  • [–] 2 points 1 day ago

    I’m tired of everything being broken or about to be broken boss.

  • source
  • [–] 1 point 13 hours ago

    Virtuosic Incompetence

  • source
  • [–] 0 points 2 days ago (3 children)

    I'm going to put on my conspiracy theorist hat to say someone is lying because this looks like an attack.

  • source
  • hideshow 3 child comments
  • [+] -6 points 1 day ago (10 children)

    I mean, isn't visibility better during the day?

    Seems like you'd rather it go down on a clear day than at night or during a winter storm.

  • source
  • hideshow 10 child comments
  • [–] 10 points 1 day ago (8 children)

    You'd rather it go down with as few flights in the air as possible. Ideally no flights in the air...but ideally you can do this without taking the system down entirely.

  • source
  • parent
  • hideshow 8 child comments
  • [–] -4 points 1 day ago (7 children)

    I would say the number of flights at nightis not lower enough to matter, especially because other countries with different time zones are sending planes at different times.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 3 points 1 day ago (5 children)

    You would be wrong. Go look at flights out of any national airport between 11pm - 5am. They're significantly cheaper and have less options for a reason. Because demand is much lower during those hours and it provides a window for other work to happen to keep the airport safe and operational.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 0 points 1 day ago (4 children)

    That's leaving. What about arriving?

    Air travel, after all, generally makes the most sense when you cross oceans

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 14 hours ago (3 children)

    You can look it up, it's significantly less. It's a fact you can check. Literally just go into an airport at midnight and count the people. It's about 90% less than are there at noon.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 14 hours ago (2 children)

    You make an ass out of yourself when you assume others live in the US.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 11 hours ago (1 child)

    This whole thread is about flights in the United States being grounded. I did assume someone so invested in the topic might live in the country. For what it's worth, I'd bet money there are less flights in most countries at night as well, for the same reasons I'm mentioning in my other comments.

    I don't appreciate being called an ass, we were having a discussion. I think I've been cordial with you, not sure why you're upset. I'd rather not continue this, I made my point and you've ignored it several times.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 day ago

    We don’t really have any details but presumably they were able to continue with manual processes. We used to do that all the time and it still works. Just doesn’t scale, so you’d want as little traffic as possible.

  • source
  • parent