Elon Musk’s management bombing on Twitter has so thinned the ranks of the software engineers who power the world’s de facto public sphere that industry insiders and programmers who have been fired or resigned this week agree: Twitter could soon fray as badly as it might actually crash.
Musk ended a very public row with nearly two dozen programmers over his microblogging platform revamp earlier this week by ordering their firing. Hundreds of engineers and other workers then quit after he demanded they commit to “extremely hard” work or quit with severance pay by Thursday night.
The latest departures mean the platform is shedding staff as it prepares for the 2022 FIFA World Cup, which opens on Sunday. It’s one of Twitter’s busiest events when tweet surges tax its systems.
“It looks like he’s going to blow up Twitter,” said Robert Graham, a veteran cybersecurity entrepreneur. “I can’t imagine the lights not going out any moment” — although many recently deceased Twitter employees predicted a more gradual death.
Hundreds of employees signaled they were leaving before Thursday’s deadline and posted farewell messages, a greeting emoji or other familiar symbols on the company’s internal Slack messaging board, according to employees who still have access. Dozens also took to Twitter publicly to announce their departure.
Earlier in the week, some were so angry at Musk’s perceived recklessness that they took to Twitter to insult the Tesla and Space X CEO. “Kiss my ass Elon,” said one engineer, adding traces of lipstick. She had been fired.
Twitter leadership sent an unsigned email after Thursday’s deadline, saying offices were closed through Monday and access to employee badges was disabled. No reason was given, according to two employees who received the email – one who took the severance pay, one who is still on the payroll. They spoke on condition of anonymity for fear of retribution.
Having a trusted phalanx of Tesla programmers by his side as he browsed a formerly sociable workspace didn’t seem to bother Musk.
“The best people stay so I’m not too worried,” he tweeted Thursday night. But it soon became clear that some key programming teams had been thoroughly gutted.
To show how close he is to programmers, Musk sent all-hands emails on Friday saying “anyone who actually writes software” at 2 p.m. to his command post at 10 a.m. on Thursday, but still received Company Emails.
After acquiring Twitter less than three weeks ago, Musk laid off half of the company’s 7,500 full-time employees and a untold number of contractors responsible for moderating content and other key duties. Then came this week’s ultimatum.
Three techs who left this week have described to The Associated Press why they expect significant inconvenience to Twitter’s more than 230 million users now that well over two-thirds of Twitter’s core service techs are gone before Musk. While they don’t anticipate a near-term collapse, Twitter could get very rough around the edges — particularly if Musk makes major changes without much off-platform testing.
Signs of fraying could be seen ahead of Thursday’s mass exit. People reported seeing more spam and scams on their feeds and in their direct messages. Engineers reported dropped tweets. People were getting strange error messages.
Nevertheless, nothing important was broken. Still.
“There’s a pool of bets on when that happens,” said one of the engineers, who all spoke on condition of anonymity for fear of retaliation from Musk, which could impact their careers and finances.
Another said that when Twitter shut down servers and “high volume suddenly hits, it could crash”.
“The World Cup is the biggest event for Twitter. That’s the first thing you learn when you jump into Twitter,” he said.
Twitter’s trending pages have already suffered from previous curation staff layoffs. The tech fireworks began Tuesday when Musk announced on Twitter that he had begun shutting down “microservices” that he believed were unnecessary “bloatware.”
“Less than 20% is actually needed to make Twitter work!” he tweeted.
That drew objections from engineers, who told Musk he had no idea what he was talking about.
“Microservices are what most of today’s major web services use to organize their code so software engineers can work quickly and efficiently,” said Gergely Orosz, author of the blog Pragmatic Engineer and former Uber programmer. There are dozens of such services and each one manages a different function. Instead of testing microservices removal in a simulated real-world environment, Musk’s team apparently updated Twitter live on all machines.
Indeed, a microservice broke briefly – the one people use to verify their identity on Twitter via SMS when they log in. It’s called two-factor authentication.
“You’ve reached your SMS code limit. Try again in 24 hours,” Twitter advised when a reporter tried to download his microblogging history archive. Luckily, the email verification alternative worked.
One of Twitter’s newly separated engineers, who had worked in core services, told the AP that engineering team clusters had gone from about 15 people before Musk — not counting team leaders, all of whom were fired — to three or four before Thursday’s resignations.
Then more institutional knowledge that cannot be replaced overnight left the door.
“Everything could break,” said the programmer.
According to the engineers, it takes six months to train someone to work on-call for some services. Such rotations require programmers to be available 24/7. But if the person on duty is unfamiliar with the codebase, errors can cascade as they frantically trawl through reference guides.
“If I had stayed, I would have been on call indefinitely with little support on several additional complex systems that I had no experience with,” tweeted Peter Clowes, an engineer who accepted the severance package.
“To run even relatively boring systems, you need people who know where to go when something breaks,” said Blaine Cook, Twitter’s founding engineer, who left the company in 2008.
“It’s like saying, ‘These firefighters aren’t doing anything. So we’re just going to fire them all.’”
Engineers also worry that Musk will shut down tools for moderating content and removing illegal material uploaded to Twitter — or that there simply won’t be enough staff to run them properly.
Another problem is hackers. If they’ve entered the system in the past, the damage depends on spotting them quickly and kicking them out.
It’s not clear how Musk’s Twitter housecleaning has affected his cybersecurity team, which suffered a major PR black eye in August when the highly respected security chief who was fired by the company earlier this year, Peiter Zatko, filed a whistleblower complaint filed claiming the platform is a cybersecurity wreck.
“A lot of the security infrastructure of a large organization like Twitter resides in people’s minds,” said Graham, the cybersecurity veteran. “And when they’re gone, you know, everything goes with them.”
AP Technology Writer Matt O’Brien contributed to this report.
© Copyright 2022 The Associated Press. All rights reserved. This material may not be published, broadcast, transcribed or redistributed without permission.