Tech
How it feels to be sexually objectified by an AI
This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
My social media feeds this week have been dominated by two hot topics: OpenAI’s latest chatbot, ChatGPT, and the viral AI avatar app Lensa. I love playing around with new technology, so I gave Lensa a go.
I was hoping to get results similar to my colleagues at MIT Technology Review. The app generated realistic and flattering avatars for them—think astronauts, warriors, and electronic music album covers.
Instead, I got tons of nudes. Out of 100 avatars I generated, 16 were topless, and another 14 had me in extremely skimpy clothes and overtly sexualized poses. You can read my story here.
Lensa creates its avatars using Stable Diffusion, an open-source AI model that generates images based on text prompts. Stable Diffusion is trained on LAION-5B, a massive open-source data set that has been compiled by scraping images from the internet.
And because the internet is overflowing with images of naked or barely dressed women, and pictures reflecting sexist, racist stereotypes, the data set is also skewed toward these kinds of images.
As an Asian woman, I thought I’d seen it all. I’ve felt icky after realizing a former date only dated Asian women. I’ve been in fights with men who think Asian women make great housewives. I’ve heard crude comments about my genitals. I’ve been mixed up with the other Asian person in the room.
Being sexualized by an AI was not something I expected, although it is not surprising. Frankly, it was crushingly disappointing. My colleagues and friends got the privilege of being stylized into artful representations of themselves. They were recognizable in their avatars! I was not. I got images of generic Asian women clearly modeled on anime characters or video games.
Funnily enough, I found more realistic portrayals of myself when I told the app I was male. This probably applied a different set of prompts to images. The differences are stark. In the images generated using male filters, I have clothes on, I look assertive, and—most important—I can recognize myself in the pictures.
“Women are associated with sexual content, whereas men are associated with professional, career-related content in any important domain such as medicine, science, business, and so on,” says Aylin Caliskan, an assistant professor at the University of Washington who studies biases and representation in AI systems.
This sort of stereotyping can be easily spotted with a new tool built by researcher Sasha Luccioni, who works at AI startup Hugging Face, that allows anyone to explore the different biases in Stable Diffusion.
The tool shows how the AI model offers pictures of white men as doctors, architects, and designers while women are depicted as hairdressers and maids.
But it’s not just the training data that is to blame. The companies developing these models and apps make active choices about how they use the data, says Ryan Steed, a PhD student at Carnegie Mellon University, who has studied biases in image-generation algorithms.
“Someone has to choose the training data, decide to build the model, decide to take certain steps to mitigate those biases or not,” he says.
Prisma Labs, the company behind Lensa, says all genders face “sporadic sexualization.” But to me, that’s not good enough. Somebody made the conscious decision to apply certain color schemes and scenarios and highlight certain body parts.
In the short term, some obvious harms could result from these decisions, such as easy access to deepfake generators that create nonconsensual nude images of women or children.
But Aylin Caliskan sees even bigger longer-term problems ahead. As AI-generated images with their embedded biases flood the internet, they will eventually become training data for future AI models. “Are we going to create a future where we keep amplifying these biases and marginalizing populations?” she says.
That’s a truly frightening thought, and I for one hope we give these issues due time and consideration before the problem gets even bigger and more embedded.
Deeper Learning
Grant money meant to help cities prepare for terror attacks is being spent on “massive purchases of surveillance technology” for US police departments, a new report by the advocacy organizations Action Center on Race and Economy (ACRE), LittleSis, MediaJustice, and the Immigrant Defense Project shows.
Shopping for AI-powered spytech: For example, the Los Angeles Police Department used funding intended for counterterrorism to buy automated license plate readers worth at least $1.27 million, radio equipment worth upwards of $24 million, Palantir data fusion platforms (often used for AI-powered predictive policing), and social media surveillance software.
Why this matters: For various reasons, a lot of problematic tech ends up in high-stake sectors such as policing with little to no oversight. For example, the facial recognition company Clearview AI offers “free trials” of its tech to police departments, which allows them to use it without a purchasing agreement or budget approval. Federal grants for counterterrorism don’t require as much public transparency and oversight. The report’s findings are yet another example of a growing pattern in which citizens are increasingly kept in the dark about police tech procurement. Read more from Tate Ryan-Mosley here.
Bits and Bytes
hatGPT, Galactica, and the progress trap
AI researchers Abeba Birhane and Deborah Raji write that the “lackadaisical approaches to model release” (as seen with Meta’s Galactica) and the extremely defensive response to critical feedback constitute a “deeply concerning” trend in AI right now. They argue that when models don’t “meet the expectations of those most likely to be harmed by them,” then “their products are not ready to serve these communities and do not deserve widespread release.” (Wired)
The new chatbots could change the world. Can you trust them?
People have been blown away by how coherent ChatGPT is. The trouble is, a significant amount of what it spews is nonsense. Large language models are no more than confident bullshitters, and we’d be wise to approach them with that in mind.
(The New York Times)
Stumbling with their words, some people let AI do the talking
Despite the tech’s flaws, some people—such as those with learning difficulties—are still finding large language models useful as a way to help express themselves.
(The Washington Post)
EU countries’ stance on AI rules draws criticism from lawmakers and activists
The EU’s AI law, the AI Act, is edging closer to being finalized. EU countries have approved their position on what the regulation should look like, but critics say many important issues, such as the use of facial recognition by companies in public places, were not addressed, and many safeguards were watered down. (Reuters)
Investors seek to profit from generative-AI startups
It’s not just you. Venture capitalists also think generative-AI startups such as Stability.AI, which created the popular text-to-image model Stable Diffusion, are the hottest things in tech right now. And they’re throwing stacks of money at them. (The Financial Times)
My senior spring in high school, I decided to defer my MIT enrollment by a year. I had always planned to take a gap year, but after receiving the silver tube in the mail and seeing all my college-bound friends plan out their classes and dorm decor, I got cold feet. Every time I mentioned my plans, I was met with questions like “But what about school?” and “MIT is cool with this?”
Yeah. MIT totally is. Postponing your MIT start date is as simple as clicking a checkbox.
COURTESY PHOTO
Now, having finished my first year of classes, I’m really grateful that I stuck with my decision to delay MIT, as I realized that having a full year of unstructured time is a gift. I could let my creative juices run. Pick up hobbies for fun. Do cool things like work at an AI startup and teach myself how to create latte art. My favorite part of the year, however, was backpacking across Europe. I traveled through Austria, Slovakia, Russia, Spain, France, the UK, Greece, Italy, Germany, Poland, Romania, and Hungary.
Moreover, despite my fear that I’d be losing a valuable year, traveling turned out to be the most productive thing I could have done with my time. I got to explore different cultures, meet new people from all over the world, and gain unique perspectives that I couldn’t have gotten otherwise. My travels throughout Europe allowed me to leave my comfort zone and expand my understanding of the greater human experience.
“In Iceland there’s less focus on hustle culture, and this relaxed approach to work-life balance ends up fostering creativity. This was a wild revelation to a bunch of MIT students.”
When I became a full-time student last fall, I realized that StartLabs, the premier undergraduate entrepreneurship club on campus, gives MIT undergrads a similar opportunity to expand their horizons and experience new things. I immediately signed up. At StartLabs, we host fireside chats and ideathons throughout the year. But our flagship event is our annual TechTrek over spring break. In previous years, StartLabs has gone on TechTrek trips to Germany, Switzerland, and Israel. On these fully funded trips, StartLabs members have visited and collaborated with industry leaders, incubators, startups, and academic institutions. They take these treks both to connect with the global startup sphere and to build closer relationships within the club itself.
Most important, however, the process of organizing the TechTrek is itself an expedited introduction to entrepreneurship. The trip is entirely planned by StartLabs members; we figure out travel logistics, find sponsors, and then discover ways to optimize our funding.
COURTESY PHOTO
In organizing this year’s trip to Iceland, we had to learn how to delegate roles to all the planners and how to maintain morale when making this trip a reality seemed to be an impossible task. We woke up extra early to take 6 a.m. calls with Icelandic founders and sponsors. We came up with options for different levels of sponsorship, used pattern recognition to deduce the email addresses of hundreds of potential contacts at organizations we wanted to visit, and all got scrappy with utilizing our LinkedIn connections.
And as any good entrepreneur must, we had to learn how to be lean and maximize our resources. To stretch our food budget, we planned all our incubator and company visits around lunchtime in hopes of getting fed, played human Tetris as we fit 16 people into a six-person Airbnb, and emailed grocery stores to get their nearly expired foods for a discount. We even made a deal with the local bus company to give us free tickets in exchange for a story post on our Instagram account.
This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.
How ubiquitous keyboard software puts hundreds of millions of Chinese users at risk
For millions of Chinese people, the first software they download onto devices is always the same: a keyboard app. Yet few of them are aware that it may make everything they type vulnerable to spying eyes.
QWERTY keyboards are inefficient as many Chinese characters share the same latinized spelling. As a result, many switch to smart, localized keyboard apps to save time and frustration. Today, over 800 million Chinese people use third-party keyboard apps on their PCs, laptops, and mobile phones.
But a recent report by the Citizen Lab, a University of Toronto–affiliated research group, revealed that Sogou, one of the most popular Chinese keyboard apps, had a massive security loophole. Read the full story.
—Zeyi Yang
Why we should all be rooting for boring AI
Earlier this month, the US Department of Defense announced it is setting up a Generative AI Task Force, aimed at “analyzing and integrating” AI tools such as large language models across the department. It hopes they could improve intelligence and operational planning.
But those might not be the right use cases, writes our senior AI reporter Melissa Heikkila. Generative AI tools, such as language models, are glitchy and unpredictable, and they make things up. They also have massive security vulnerabilities, privacy problems, and deeply ingrained biases.
Applying these technologies in high-stakes settings could lead to deadly accidents where it’s unclear who or what should be held responsible, or even why the problem occurred. The DoD’s best bet is to apply generative AI to more mundane things like Excel, email, or word processing. Read the full story.
This story is from The Algorithm, Melissa’s weekly newsletter giving you the inside track on all things AI. Sign up to receive it in your inbox every Monday.
The ice cores that will let us look 1.5 million years into the past
To better understand the role atmospheric carbon dioxide plays in Earth’s climate cycles, scientists have long turned to ice cores drilled in Antarctica, where snow layers accumulate and compact over hundreds of thousands of years, trapping samples of ancient air in a lattice of bubbles that serve as tiny time capsules.
By analyzing those cores, scientists can connect greenhouse-gas concentrations with temperatures going back 800,000 years. Now, a new European-led initiative hopes to eventually retrieve the oldest core yet, dating back 1.5 million years. But that impressive feat is still only the first step. Once they’ve done that, they’ll have to figure out how they’re going to extract the air from the ice. Read the full story.
—Christian Elliott
This story is from the latest edition of our print magazine, set to go live tomorrow. Subscribe today for as low as $8/month to ensure you receive full access to the new Ethics issue and in-depth stories on experimental drugs, AI assisted warfare, microfinance, and more.
The must-reads
I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.
1 How AI got dragged into the culture wars
Fears about ‘woke’ AI fundamentally misunderstand how it works. Yet they’re gaining traction. (The Guardian)
+ Why it’s impossible to build an unbiased AI language model. (MIT Technology Review)
2 Researchers are racing to understand a new coronavirus variant
It’s unlikely to be cause for concern, but it shows this virus still has plenty of tricks up its sleeve. (Nature)
+ Covid hasn’t entirely gone away—here’s where we stand. (MIT Technology Review)
+ Why we can’t afford to stop monitoring it. (Ars Technica)
3 How Hilary became such a monster storm
Much of it is down to unusually hot sea surface temperatures. (Wired $)
+ The era of simultaneous climate disasters is here to stay. (Axios)
+ People are donning cooling vests so they can work through the heat. (Wired $)
4 Brain privacy is set to become important
Scientists are getting better at decoding our brain data. It’s surely only a matter of time before others want a peek. (The Atlantic $)
+ How your brain data could be used against you. (MIT Technology Review)
5 How Nvidia built such a big competitive advantage in AI chips
Today it accounts for 70% of all AI chip sales—and an even greater share for training generative models. (NYT $)
+ The chips it’s selling to China are less effective due to US export controls. (Ars Technica)
+ These simple design rules could turn the chip industry on its head. (MIT Technology Review)
6 Inside the complex world of dissociative identity disorder on TikTok
Reducing stigma is great, but doctors fear people are self-diagnosing or even imitating the disorder. (The Verge)
7 What TikTok might have to give up to keep operating in the US
This shows just how hollow the authorities’ purported data-collection concerns really are. (Forbes)
8 Soldiers in Ukraine are playing World of Tanks on their phones
It’s eerily similar to the war they are themselves fighting, but they say it helps them to dissociate from the horror. (NYT $)
9 Conspiracy theorists are sharing mad ideas on what causes wildfires
But it’s all just a convoluted way to try to avoid having to tackle climate change. (Slate $)
10 Christie’s accidentally leaked the location of tons of valuable art 
Seemingly thanks to the metadata that often automatically attaches to smartphone photos. (WP $)
Quote of the day
“Is it going to take people dying for something to move forward?”
—An anonymous air traffic controller warns that staffing shortages in their industry, plus other factors, are starting to threaten passenger safety, the New York Times reports.
The big story
Inside effective altruism, where the far future counts a lot more than the present
October 2022
Since its birth in the late 2000s, effective altruism has aimed to answer the question “How can those with means have the most impact on the world in a quantifiable way?”—and supplied methods for calculating the answer.
It’s no surprise that effective altruisms’ ideas have long faced criticism for reflecting white Western saviorism, alongside an avoidance of structural problems in favor of abstract math. And as believers pour even greater amounts of money into the movement’s increasingly sci-fi ideals, such charges are only intensifying. Read the full story.
—Rebecca Ackermann
We can still have nice things
A place for comfort, fun and distraction in these weird times. (Got any ideas? Drop me a line or tweet ’em at me.)
+ Watch Andrew Scott’s electrifying reading of the 1965 commencement address ‘Choose One of Five’ by Edith Sampson.
+ Here’s how Metallica makes sure its live performances ROCK. ($)
+ Cannot deal with this utterly ludicrous wooden vehicle.
+ Learn about a weird and wonderful new instrument called a harpejji.
