Connect with us

Tech

She risked everything to expose Facebook. Now she’s telling her story.

Published

on

She risked everything to expose Facebook. Now she’s telling her story.


The world first learned of Sophie Zhang in September 2020, when BuzzFeed News obtained and published highlights from an abridged version of her nearly 8,000-word exit memo from Facebook.

Before she was fired for poor performance, Zhang was officially employed as a low-level data scientist at the company. But she had become consumed by a task she deemed more important: finding and taking down fake accounts and likes on the platform that were being used to sway elections globally.

Her memo revealed how she’d identified dozens of countries, including India, Mexico, Afghanistan, and South Korea, where this type of abuse was enabling politicians to mislead the public and gain power. It also revealed how little the company had done to mitigate it, despite Zhang’s repeated efforts to bring it to the attention of leadership.

“I know that I have blood on my hands by now,” she wrote.

On the eve of her departure, Zhang was still debating whether to write the memo at all. It was perhaps her last chance to create enough internal pressure on leadership to start taking the problems seriously. In anticipation of writing it, she had turned down a nearly $64,000 severance package to avoid signing a nondisparagement agreement and retain the freedom to speak critically about the company.

But she was disturbed by the idea that, just two months out from the 2020 US election, the memo could erode the public’s trust in the electoral process if prematurely released to the press. “I was terrified of somehow becoming the James Comey of 2020,” she says, referring to the former FBI director who told Congress the agency had reopened an investigation into Hillary Clinton’s use of a private email server days before the election. Clinton went on to blame Comey for losing her the presidency.

To Zhang’s great relief, that didn’t happen. And after the election passed, she proceeded with her original plan. In April, she came forward in two Guardian articles with her face, name, and even more detailed documentation on the political manipulation she’d uncovered as well as Facebook’s negligence.

Her account supplied concrete evidence to support what critics had long been saying on the outside: Facebook makes election interference easy, and that unless such activity hurts the company’s business interests, it can’t be bothered to fix the problem.

By going public and eschewing anonymity, Zhang also risked legal action from the company, her future career prospects, and perhaps even action from the politicians she exposed in the process. “What she did is very brave,” says Julia Carrie Wong, the Guardian reporter who published her revelations.

In a statement Joe Osborn, a Facebook spokesperson, vehemently denied Zhang’s characterization. “For the countless press interviews she’s done since leaving Facebook, we have fundamentally disagreed with Ms. Zhang’s characterization of our priorities and efforts to root out abuse on our platform,” he said. “We aggressively go after abuse around the world and have specialized teams focused on this work. As a result, we’ve already taken down more than 150 networks of coordinated inauthentic behavior…Combatting coordinated inauthentic behavior is our priority.”

After nearly a year of avoiding personal questions, Zhang is now ready to tell her story. She wants the world to understand how she became so entwined in trying to protect democracy worldwide and why she cared so deeply. She’s also tired of being in the closet: Zhang is a transgender woman, a core aspect of her identity that informed her actions at and after Facebook.

Her story reveals that it is really pure luck that we now know so much about how Facebook enables election interference globally. Zhang was not just the only person fighting an entire swath of political manipulation, it also wasn’t her job. She had discovered the problem because of a unique confluence of skills and passion, then taken it upon herself, driven by an extraordinary sense of moral responsibility.

To regulators around the world considering how to rein in the company, this should be a wakeup call.

Zhang never planned to be in this position. She’s deeply introverted and hates being in the limelight. She’d joined Facebook in 2018 after the financial strain of living in the Bay Area on part-time contract work had worn her down. When she received Facebook’s offer, she was upfront with her recruiter: she didn’t think the company was making the world better, but she would join to help fix it.

“They told me, ‘You’d be surprised how many people at Facebook say that,’” she remembers.

But the task was easier said than done. Like many new hires, she joined without being assigned to a specific team. She wanted to work on election integrity, which works to mitigate election-related platform abuse, but her skills didn’t match their openings. She settled for a new team tackling fake engagement instead.

Fake engagement refers to things such as likes, shares, and comments that have been bought or otherwise inauthentically generated on the platform. The focus of the new team’s work was narrower, on so-called “scripted inauthentic activity”—fake likes and shares produced by automated bots, used to drive up someone’s popularity.

The vast majority of such cases were people obtaining likes for vanity. But half a year in, Zhang intuited that politicians could do the same to increase their influence and reach on the platform. It didn’t take long for her to find examples in Brazil and India, in the lead up to general elections.

But in the process of searching for scripted activity, she found something far more worrying. The Facebook page administrator of the Honduran president, Juan Orlando Hernández, had created hundreds of pages with fake names and profile pictures to look just like users, and was using them to flood the president’s posts with likes, comments, and shares. (Facebook bars users from making multiple profiles but doesn’t apply the same restriction to pages, which are usually meant for businesses and public figures.)

The activity didn’t count as scripted, but the effect was the same. It could not only mislead the casual observer into believing Orlando Hernández was more well-liked and popular than he was. It was also boosting his posts higher up in people’s newsfeed. For a politician whose 2017 re-election campaign was widely believed to be fraudulent, the brazenness—and implications—were alarming.

“Everyone agreed that it was terrible. No one could agree who should be responsible, or even what should be done.”

But when Zhang raised the issue, she says she received a lukewarm reception. The pages integrity team, which handles abuse of and on Facebook pages, wouldn’t block the mass-manufacture of pages to look like users. The newsfeed integrity team, which tries to improve the quality of what appears in user’s newsfeeds, wouldn’t remove the fake likes and comments from the ranking algorithm’s consideration. “Everyone agreed that it was terrible,” Zhang says. “No one could agree who should be responsible, or even what should be done.”

After a year of Zhang applying pressure, the network of fake pages was finally removed. A few months later, Facebook created a new “inauthentic behavior policy” to ban fake pages marauding as users. But this policy change didn’t address a more fundamental problem: no one was being asked to enforce it.

So Zhang took it upon herself. When she wasn’t working to scrub away vanity likes, she diligently combed through streams of data, searching for the use of fake pages, fake accounts, and other forms of coordinated fake activity on politicians’ pages. She found cases across dozens of countries, most egregiously in Azerbaijan where the pages technique was being used to harass the opposition.

But finding and flagging new cases wasn’t enough. In order to get any networks of fake pages or accounts removed, Zhang found she had to persistently lobby the relevant teams. In countries where such activity posed little PR risk to the company, enforcement could be put off repeatedly. (Facebook disputes this characterization.) The responsibility weighed on her heavily. Was it more important to push for a case in Bolivia with a population of 11.6 million, or in Rajasthan, India, with a population close to 70 million?

Then in the fall of 2019, weeks of deadly civil protest broke out in Bolivia after the public contested the results of its presidential election. Only a few weeks earlier, Zhang had indeed deprioritized the country to take care of more urgent cases. The news consumed her with guilt. Intellectually, she knew there was no way to draw a direct connection between her decision and the events. The fake engagement had been so small the effect was likely negligible. But psychologically and emotionally, it didn’t matter. “That’s when I started losing sleep,” she says.

Whereas someone else may have chosen to leave such a taxing job or perhaps abdicate themselves of the responsibility as a means of coping, Zhang leaned in, at great personal cost, in an attempt to single-handedly right a wrong.

Over the year between the events in Bolivia and her firing, the exertion sent her health into sharp decline. She already suffered from anxiety and depression, but it grew significantly—and dangerously—worse. Always a voracious reader of world news, she could no longer distance herself from the political turmoil of other countries. The pressure pushed her away from friends and loved ones. She grew increasingly isolated and broke up with her girlfriend. She upped her anxiety and antidepressant medication until her dose had increased by six-fold.

For Zhang, the explanation of why she cared so much is tied up in her identity. She grew up in Ann Arbor, Michigan, the daughter of parents who’d immigrated from mainland China. From an early age, she was held to high academic standards and proved a precocious scholar. At six or seven, she read an introductory physics book and grew fascinated by the building blocks of the universe. Her passion would lead her to study cosmology at the University of Michigan, where she published two research papers, including one as a single author.

“She was blazing smart. She may be the smartest undergrad student I’ve ever worked with,” recalls Dragan Huterer, her undergraduate advisor. “I would say she was more advanced than a graduate student.”

But her childhood was also marked by severe trauma. As early as five years old, she began to realize she was different. She read a children’s book about a boy whose friends told him that if he kissed his elbow he would turn into a girl. “I spent a long time after that trying to kiss my elbow,” she says.

She did her best to hide it, understanding that her parents would find her transgender identity intolerable. But she vividly remembers the moment her father found out. It was spring of 8th grade. It had just rained. And she was cowered in the bathroom, contemplating whether to jump out the window, as he beat down the door.

In the end, she chose not to jump and let him hit her until she was bloody, she says. “Ultimately, I decided that I was the person who stayed in imperfect situations to try and fix them.” The next day, she wore a long sleeve shirt to cover up the bruises and prepared an excuse in case a teacher noticed. None did, she says.

(When reached by email, her father denied the allegations. “I am sad that she alleges that I beat her as a child after I discovered her transgender identity, which is completely false,” he wrote. But multiple people who knew Zhang through high school to present day corroborated her account of her father’s abusive behavior.)

“To give up on them and abandon them would be a betrayal of the very core of my identity.”

In college, she decided to transition, after which her father disowned her. But she soon discovered that finally being perceived correctly as a woman came with its own consequences. “I knew precisely how people treated me when they thought that I was a dude. It was very different,” she says.

After being accepted to all the top PhD programs for physics, she chose to attend Princeton University. During orientation, the person giving a tour of the machine shop repeatedly singled her out in front of the group with false assumptions about her incompetence. “It was my official introduction to Princeton and a very appropriate one,” she says.

From there the sexism only got worse. Almost immediately a male grad student began to stalk and sexually harass her. To cope, she picked a thesis advisor in the biophysics department, which allowed her to escape her harasser by conducting research in another building. The trouble was she wasn’t actually interested in biophysics. And whether for this or other reasons, her interest in physics slowly dissolved.

Three years in, deeply unhappy, she decided to leave the program, though not without finally reporting the harassment to the university. “They were like, ‘It’s your word against his,’” she remembers. “You can probably guess now why I extensively documented everything I gave to Julia,” referring to Julia Carrie Wong at the Guardian. “I didn’t want to be in another ‘he said she said’ situation.”

(A Princeton spokesperson said he was unable to comment on individual situations but stated the university’s commitment to “providing an inclusive and welcoming educational and working environment.” “Princeton seeks to support any member of the campus community who has experienced sexual misconduct, including sexual harassment,” he said.)

“What these experiences have in common is the fact that I’ve experienced repeatedly falling through the cracks of responsibility,” Zhang wrote in her memo, after summarizing these experiences. “I never received the support from the authority figures I needed…In each case, they completed the letter of their duty but failed the spirit, and I paid the price of their decisions.”

“Perhaps then you can understand why this was so personal for myself from the very start, why I fought so hard to keep the people of Honduras and Azerbaijan from slipping through those cracks,” she wrote. “To give up on them and abandon them would be a betrayal of the very core of my identity.”

It was during the start of her physical and mental decline in the fall of 2019 that Zhang began thinking about whether to come forward. She wanted to give Facebook’s official systems a chance to work. But she worried about being a single point of failure. “What if I got hit by a bus the next day?” she says. She needed someone else to have access to the same information.

By coincidence, she received an email from a journalist in her inbox. Wong, then a senior tech reporter at the Guardian, had been messaging Facebook employees looking to cultivate new sources. Zhang took the chance and agreed to meet for an off-the-record conversation. That day, she dropped her company-issued phone and computer off at a former housemate’s place as a precaution, knowing that Facebook had the ability to track her location. When she returned, she looked a little more relieved, the former housemate Ness Io Kain remembers. “You could tell that she felt like she’d accomplished something. It’s pretty silent, but it’s definitely palpable.”

For a moment things at Facebook seemed to make progress. She saw the policy change and takedown of the Honduran president’s fake network as forward momentum. She was called upon repeatedly to help handle emergencies and praised for her work, which she was told was valued and important.

But despite her repeated attempts to push for more resources, leadership cited different priorities. They also dismissed Zhang’s suggestions for a more sustainable solution, such as suspending or otherwise penalizing politicians who were repeat offenders. It left her to face a never-ending firehose: The manipulation networks she took down quickly came back, often only hours or days later. “It increasingly felt like I was trying to empty the ocean with a colander,” she says. “Two steps back, two steps forward.”

“I have never hated my autism more than when I joined Facebook.”

Then in January of 2020, the tide turned. Both her manager and manager’s manager told her to stop her political work and stick to her assigned job. If she didn’t, her services at the company would no longer be needed, she remembers the latter saying. But without a team assigned to continue her work, Zhang kept doing some in secret.

As the pressure of her work and her health worsened, Zhang realized she would ultimately need to leave. She made a plan to depart after the US election, considering it the last and most important duty she needed to perform. But leadership had other plans. In August, she was informed that she would be fired due to poor performance.

On her last day, within hours of her posting her memo internally, Facebook deleted it (though it later restored an edited version after widespread employee anger). A few hours later, an HR person called her, asking her to also remove a password-protected copy she had posted on her personal website. She tried to bargain: she would do so if they restored the internal version. The next day, instead, she received a notice from her hosting server that it’d taken down her entire website after a complaint from Facebook. A few days later, it took down her domain as well.

Even after all that Facebook put her through, Zhang defaults to blaming herself. In her memo, she apologized to colleagues for any trouble she may have caused them and for leaving them without achieving more. In a Reddit AMA months later, she apologized to the citizens of different countries for not acting fast enough and for failing to reach a long-term solution.

To me, Zhang, who is autistic, wonders aloud what she could have accomplished if she were not. “I have no talent for persuasion and convincing,” she says. “If I were someone born with a silver tongue, perhaps I could have made changes.”

“I have never hated my autism more than when I joined Facebook.”

In preparation for going public, Zhang made one final sacrifice: to conceal her trans identity, not for fear of harassment, but for fear that it would distract from her message. In the US, where transgender rights are highly politicized, she didn’t want protecting democracy to become a partisan issue. Abroad, where some countries treat being transgender as a crime punishable by prison time or even death, she didn’t want people to stop listening.

It was a continuation of a sacrifice she’d repeatedly made when policing election interference globally. She treated all politicians equally, even when removing the fake activity of one in Azerbaijan inevitably boosted an opponent who espoused homophobia. “I did my best to protect democracy and rule of law globally for people, regardless of whether they believed me to be human,” she says with a deep sigh. “But I don’t think anyone should have to make that choice.”

The night the Guardian articles published, she anxiously awaited the public reaction, worried about whether she’d be able to handle the media attention. “I think she actually surprised herself at how good she was in interviews,” says her girlfriend Lisa Danz, who Zhang got back together with after leaving Facebook. “She found that when there’s material that she knows very well and she’s just getting asked questions about it, she can answer.”

The attention ultimately fell short of what Zhang had hoped for. Several media outlets in the US did follow-up pieces as did foreign outlets from countries impacted by the manipulation activity. But as far as she’s aware, it didn’t achieve what she was ultimately hoping for: enough of a PR scandal for Facebook to finally prioritize the work she left behind.

Facebook once again disputes this characterization, saying the fake engagement team has continued Zhang’s work. But Zhang points to other evidence: the network of fake pages in Azerbaijan is still there. “It’s clear they haven’t been successful,” she says.

Nonetheless Zhang doesn’t regret her decision to come forward. To her, it was a foregone conclusion. “I was the only one in this position of responsibility from the start,” she says, “and someone had to take the responsibility and do the utmost to protect people.”

Without skipping a beat, she then rattles off the consequences that others have faced for going up against the powerful in more hostile countries: journalists being murdered for investigating government corruption, protestors being gunned down for showing their dissent.

“Compared to them, I’m small potatoes,” she says.

Tech

Why I became a TechTrekker

Published

on

group jumps into the air with snowy mountains in the background


My senior spring in high school, I decided to defer my MIT enrollment by a year. I had always planned to take a gap year, but after receiving the silver tube in the mail and seeing all my college-bound friends plan out their classes and dorm decor, I got cold feet. Every time I mentioned my plans, I was met with questions like “But what about school?” and “MIT is cool with this?”

Yeah. MIT totally is. Postponing your MIT start date is as simple as clicking a checkbox. 

Sofia Pronina (right) was among those who hiked to the Katla Glacier during this year’s TechTrek to Iceland.

COURTESY PHOTO

Now, having finished my first year of classes, I’m really grateful that I stuck with my decision to delay MIT, as I realized that having a full year of unstructured time is a gift. I could let my creative juices run. Pick up hobbies for fun. Do cool things like work at an AI startup and teach myself how to create latte art. My favorite part of the year, however, was backpacking across Europe. I traveled through Austria, Slovakia, Russia, Spain, France, the UK, Greece, Italy, Germany, Poland, Romania, and Hungary. 

Moreover, despite my fear that I’d be losing a valuable year, traveling turned out to be the most productive thing I could have done with my time. I got to explore different cultures, meet new people from all over the world, and gain unique perspectives that I couldn’t have gotten otherwise. My travels throughout Europe allowed me to leave my comfort zone and expand my understanding of the greater human experience. 

“In Iceland there’s less focus on hustle culture, and this relaxed approach to work-life balance ends up fostering creativity. This was a wild revelation to a bunch of MIT students.”

When I became a full-time student last fall, I realized that StartLabs, the premier undergraduate entrepreneurship club on campus, gives MIT undergrads a similar opportunity to expand their horizons and experience new things. I immediately signed up. At StartLabs, we host fireside chats and ideathons throughout the year. But our flagship event is our annual TechTrek over spring break. In previous years, StartLabs has gone on TechTrek trips to Germany, Switzerland, and Israel. On these fully funded trips, StartLabs members have visited and collaborated with industry leaders, incubators, startups, and academic institutions. They take these treks both to connect with the global startup sphere and to build closer relationships within the club itself.

Most important, however, the process of organizing the TechTrek is itself an expedited introduction to entrepreneurship. The trip is entirely planned by StartLabs members; we figure out travel logistics, find sponsors, and then discover ways to optimize our funding. 

two students soaking in a hot spring in Iceland

COURTESY PHOTO

In organizing this year’s trip to Iceland, we had to learn how to delegate roles to all the planners and how to maintain morale when making this trip a reality seemed to be an impossible task. We woke up extra early to take 6 a.m. calls with Icelandic founders and sponsors. We came up with options for different levels of sponsorship, used pattern recognition to deduce the email addresses of hundreds of potential contacts at organizations we wanted to visit, and all got scrappy with utilizing our LinkedIn connections.

And as any good entrepreneur must, we had to learn how to be lean and maximize our resources. To stretch our food budget, we planned all our incubator and company visits around lunchtime in hopes of getting fed, played human Tetris as we fit 16 people into a six-person Airbnb, and emailed grocery stores to get their nearly expired foods for a discount. We even made a deal with the local bus company to give us free tickets in exchange for a story post on our Instagram account. 

Continue Reading

Tech

The Download: spying keyboard software, and why boring AI is best

Published

on

🧠


This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

How ubiquitous keyboard software puts hundreds of millions of Chinese users at risk

For millions of Chinese people, the first software they download onto devices is always the same: a keyboard app. Yet few of them are aware that it may make everything they type vulnerable to spying eyes. 

QWERTY keyboards are inefficient as many Chinese characters share the same latinized spelling. As a result, many switch to smart, localized keyboard apps to save time and frustration. Today, over 800 million Chinese people use third-party keyboard apps on their PCs, laptops, and mobile phones. 

But a recent report by the Citizen Lab, a University of Toronto–affiliated research group, revealed that Sogou, one of the most popular Chinese keyboard apps, had a massive security loophole. Read the full story. 

—Zeyi Yang

Why we should all be rooting for boring AI

Earlier this month, the US Department of Defense announced it is setting up a Generative AI Task Force, aimed at “analyzing and integrating” AI tools such as large language models across the department. It hopes they could improve intelligence and operational planning. 

But those might not be the right use cases, writes our senior AI reporter Melissa Heikkila. Generative AI tools, such as language models, are glitchy and unpredictable, and they make things up. They also have massive security vulnerabilities, privacy problems, and deeply ingrained biases. 

Applying these technologies in high-stakes settings could lead to deadly accidents where it’s unclear who or what should be held responsible, or even why the problem occurred. The DoD’s best bet is to apply generative AI to more mundane things like Excel, email, or word processing. Read the full story. 

This story is from The Algorithm, Melissa’s weekly newsletter giving you the inside track on all things AI. Sign up to receive it in your inbox every Monday.

The ice cores that will let us look 1.5 million years into the past

To better understand the role atmospheric carbon dioxide plays in Earth’s climate cycles, scientists have long turned to ice cores drilled in Antarctica, where snow layers accumulate and compact over hundreds of thousands of years, trapping samples of ancient air in a lattice of bubbles that serve as tiny time capsules. 

By analyzing those cores, scientists can connect greenhouse-gas concentrations with temperatures going back 800,000 years. Now, a new European-led initiative hopes to eventually retrieve the oldest core yet, dating back 1.5 million years. But that impressive feat is still only the first step. Once they’ve done that, they’ll have to figure out how they’re going to extract the air from the ice. Read the full story.

—Christian Elliott

This story is from the latest edition of our print magazine, set to go live tomorrow. Subscribe today for as low as $8/month to ensure you receive full access to the new Ethics issue and in-depth stories on experimental drugs, AI assisted warfare, microfinance, and more.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 How AI got dragged into the culture wars
Fears about ‘woke’ AI fundamentally misunderstand how it works. Yet they’re gaining traction. (The Guardian
+ Why it’s impossible to build an unbiased AI language model. (MIT Technology Review)
 
2 Researchers are racing to understand a new coronavirus variant 
It’s unlikely to be cause for concern, but it shows this virus still has plenty of tricks up its sleeve. (Nature)
Covid hasn’t entirely gone away—here’s where we stand. (MIT Technology Review)
+ Why we can’t afford to stop monitoring it. (Ars Technica)
 
3 How Hilary became such a monster storm
Much of it is down to unusually hot sea surface temperatures. (Wired $)
+ The era of simultaneous climate disasters is here to stay. (Axios)
People are donning cooling vests so they can work through the heat. (Wired $)
 
4 Brain privacy is set to become important 
Scientists are getting better at decoding our brain data. It’s surely only a matter of time before others want a peek. (The Atlantic $)
How your brain data could be used against you. (MIT Technology Review)
 
5 How Nvidia built such a big competitive advantage in AI chips
Today it accounts for 70% of all AI chip sales—and an even greater share for training generative models. (NYT $)
The chips it’s selling to China are less effective due to US export controls. (Ars Technica)
+ These simple design rules could turn the chip industry on its head. (MIT Technology Review)
 
6 Inside the complex world of dissociative identity disorder on TikTok 
Reducing stigma is great, but doctors fear people are self-diagnosing or even imitating the disorder. (The Verge)
 
7 What TikTok might have to give up to keep operating in the US
This shows just how hollow the authorities’ purported data-collection concerns really are. (Forbes)
 
8 Soldiers in Ukraine are playing World of Tanks on their phones
It’s eerily similar to the war they are themselves fighting, but they say it helps them to dissociate from the horror. (NYT $)
 
9 Conspiracy theorists are sharing mad ideas on what causes wildfires
But it’s all just a convoluted way to try to avoid having to tackle climate change. (Slate $)
 
10 Christie’s accidentally leaked the location of tons of valuable art 🖼📍
Seemingly thanks to the metadata that often automatically attaches to smartphone photos. (WP $)

Quote of the day

“Is it going to take people dying for something to move forward?”

—An anonymous air traffic controller warns that staffing shortages in their industry, plus other factors, are starting to threaten passenger safety, the New York Times reports.

The big story

Inside effective altruism, where the far future counts a lot more than the present

" "

VICTOR KERLOW

October 2022

Since its birth in the late 2000s, effective altruism has aimed to answer the question “How can those with means have the most impact on the world in a quantifiable way?”—and supplied methods for calculating the answer.

It’s no surprise that effective altruisms’ ideas have long faced criticism for reflecting white Western saviorism, alongside an avoidance of structural problems in favor of abstract math. And as believers pour even greater amounts of money into the movement’s increasingly sci-fi ideals, such charges are only intensifying. Read the full story.

—Rebecca Ackermann

We can still have nice things

A place for comfort, fun and distraction in these weird times. (Got any ideas? Drop me a line or tweet ’em at me.)

+ Watch Andrew Scott’s electrifying reading of the 1965 commencement address ‘Choose One of Five’ by Edith Sampson.
+ Here’s how Metallica makes sure its live performances ROCK. ($)
+ Cannot deal with this utterly ludicrous wooden vehicle
+ Learn about a weird and wonderful new instrument called a harpejji.



Continue Reading

Tech

Why we should all be rooting for boring AI

Published

on

Why we should all be rooting for boring AI


This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

I’m back from a wholesome week off picking blueberries in a forest. So this story we published last week about the messy ethics of AI in warfare is just the antidote, bringing my blood pressure right back up again. 

Arthur Holland Michel does a great job looking at the complicated and nuanced ethical questions around warfare and the military’s increasing use of artificial-intelligence tools. There are myriad ways AI could fail catastrophically or be abused in conflict situations, and there don’t seem to be any real rules constraining it yet. Holland Michel’s story illustrates how little there is to hold people accountable when things go wrong.  

Last year I wrote about how the war in Ukraine kick-started a new boom in business for defense AI startups. The latest hype cycle has only added to that, as companies—and now the military too—race to embed generative AI in products and services. 

Earlier this month, the US Department of Defense announced it is setting up a Generative AI Task Force, aimed at “analyzing and integrating” AI tools such as large language models across the department. 

The department sees tons of potential to “improve intelligence, operational planning, and administrative and business processes.” 

But Holland Michel’s story highlights why the first two use cases might be a bad idea. Generative AI tools, such as language models, are glitchy and unpredictable, and they make things up. They also have massive security vulnerabilities, privacy problems, and deeply ingrained biases.  

Applying these technologies in high-stakes settings could lead to deadly accidents where it’s unclear who or what should be held responsible, or even why the problem occurred. Everyone agrees that humans should make the final call, but that is made harder by technology that acts unpredictably, especially in fast-moving conflict situations. 

Some worry that the people lowest on the hierarchy will pay the highest price when things go wrong: “In the event of an accident—regardless of whether the human was wrong, the computer was wrong, or they were wrong together—the person who made the ‘decision’ will absorb the blame and protect everyone else along the chain of command from the full impact of accountability,” Holland Michel writes. 

The only ones who seem likely to face no consequences when AI fails in war are the companies supplying the technology.

It helps companies when the rules the US has set to govern AI in warfare are mere recommendations, not laws. That makes it really hard to hold anyone accountable. Even the AI Act, the EU’s sweeping upcoming regulation for high-risk AI systems, exempts military uses, which arguably are the highest-risk applications of them all. 

While everyone is looking for exciting new uses for generative AI, I personally can’t wait for it to become boring. 

Amid early signs that people are starting to lose interest in the technology, companies might find that these sorts of tools are better suited for mundane, low-risk applications than solving humanity’s biggest problems.

Applying AI in, for example, productivity software such as Excel, email, or word processing might not be the sexiest idea, but compared to warfare it’s a relatively low-stakes application, and simple enough to have the potential to actually work as advertised. It could help us do the tedious bits of our jobs faster and better.

Boring AI is unlikely to break as easily and, most important, won’t kill anyone. Hopefully, soon we’ll forget we’re interacting with AI at all. (It wasn’t that long ago when machine translation was an exciting new thing in AI. Now most people don’t even think about its role in powering Google Translate.) 

That’s why I’m more confident that organizations like the DoD will find success applying generative AI in administrative and business processes. 

Boring AI is not morally complex. It’s not magic. But it works. 

Deeper Learning

AI isn’t great at decoding human emotions. So why are regulators targeting the tech?

Amid all the chatter about ChatGPT, artificial general intelligence, and the prospect of robots taking people’s jobs, regulators in the EU and the US have been ramping up warnings against AI and emotion recognition. Emotion recognition is the attempt to identify a person’s feelings or state of mind using AI analysis of video, facial images, or audio recordings. 

But why is this a top concern? Western regulators are particularly concerned about China’s use of the technology, and its potential to enable social control. And there’s also evidence that it simply does not work properly. Tate Ryan-Mosley dissected the thorny questions around the technology in last week’s edition of The Technocrat, our weekly newsletter on tech policy.

Bits and Bytes

Meta is preparing to launch free code-generating software
A version of its new LLaMA 2 language model that is able to generate programming code will pose a stiff challenge to similar proprietary code-generating programs from rivals such as OpenAI, Microsoft, and Google. The open-source program is called Code Llama, and its launch is imminent, according to The Information. (The Information

OpenAI is testing GPT-4 for content moderation
Using the language model to moderate online content could really help alleviate the mental toll content moderation takes on humans. OpenAI says it’s seen some promising first results, although the tech does not outperform highly trained humans. A lot of big, open questions remain, such as whether the tool can be attuned to different cultures and pick up context and nuance. (OpenAI)

Google is working on an AI assistant that offers life advice
The generative AI tools could function as a life coach, offering up ideas, planning instructions, and tutoring tips. (The New York Times)

Two tech luminaries have quit their jobs to build AI systems inspired by bees
Sakana, a new AI research lab, draws inspiration from the animal kingdom. Founded by two prominent industry researchers and former Googlers, the company plans to make multiple smaller AI models that work together, the idea being that a “swarm” of programs could be as powerful as a single large AI model. (Bloomberg)

Continue Reading

Copyright © 2021 Vitamin Patches Online.