Tech
Podcast: What’s AI doing in your wallet?
Published
4 years agoon
By
Terry Power
Our entire financial system is built on trust. We can exchange otherwise worthless paper bills for fresh groceries, or swipe a piece of plastic for new clothes. But this trust—typically in a central government-backed bank—is changing. As our financial lives are rapidly digitized, the resulting data turns into fodder for AI. Companies like Apple, Facebook and Google see it as an opportunity to disrupt the entire experience of how people think about and engage with their money. But will we as consumers really get more control over our finances? In this first of a series on automation and our wallets, we explore a digital revolution in how we pay for things.
We meet:
- Umar Farooq, CEO of Onyx by J.P. Morgan Chase
- Josh Woodward, Director of product management for Google Pay
- Ed McLaughlin, President of operations and technology for MasterCard
- Craig Vosburg, Chief product officer for MasterCard
Credits
This episode was produced by Anthony Green, with help from Jennifer Strong, Karen Hao, Will Douglas Heaven and Emma Cillekens. We’re edited by Michael Reilly. Special thanks to our events team for recording part of this episode at our AI conference, Emtech Digital.
Transcript
[TR ID]
Strong: For as long as people have needed things, we’ve… also needed a way to pay for them. From bartering and trading… to the invention of money… and eventually, credit cards… which these days we often use through apps on our phones.
Farooq: No one, 10 years ago, no one thought that, you know, you’d be just getting up from a dinner table and using Zelle or Venmo to send five bucks to your friend. And now you do.
Strong: The act of paying for something might seem simple. But trading paper for groceries…or swiping a piece of plastic for new clothes is built on a few powerful ideas that allow us to represent and exchange things of value.
Our entire financial system is built on this agreement… (and trust).
But this model is changing… and banks are no longer the only players in town.
[Sounds from an advertisement for Apple Card]
[Ad music fades in]
Announcer: This is Apple Card. A credit card created by Apple—not a bank. So it’s simple, transparent, and private. It works with Apple Pay. So buying something as easy as: *iPhone ding*.
Strong: It’s not just Apple. Many other tech giants are moving into our wallets… including Google… and Facebook…
[Sounds from Facebook’s developer conference]
Mark Zuckerberg: I believe it should be as easy to send money to someone as it is to send a photo.
Strong: Facebook Pay works through it’s social apps—including Instagram and Whatsapp—and executives hope those payments will one day be made with Facebook’s very own currency.
And beyond what we use to pay for things, how we pay for things is changing too.
[Sounds from an advertisement for Amazon One]
Announcer: Introducing Amazon One. A free service that lets you use your palm to quickly pay for things, gain access, earn rewards and more.
Strong: This product works by scanning the palm of your hand… and it’s not just for payments. It’s also being marketed as an ID. Something like this could one day be used to unlock the door at the office or to board a plane.
But letting companies use data from our bodies in this way raises all sorts of questions—especially if it mixes with other personal data.
Vosburg: We can see in great detail how people, for example, are interacting with their device. We can see the position in which they’re holding it. We can understand the way in which they’re typing. We can understand the pressure that’s being applied on the screen as people are hitting the keystrokes. All of these things can be useful with the combination of artificial intelligence to process the data to create sort of an interaction fingerprint.
Strong: I’m Jennifer Strong, and in this first of a series on automation and our wallets, we explore a digital revolution in how we pay for things.
[SHOW ID]
Farooq: So, if you think about how we operate today, we primarily operate through central authorities.
Strong: Omar Farooq is the CEO of Onyx… from J.P. Morgan. It focuses on futuristic payment products.
Farooq: Frankly, the biggest central authority in some ways is, in the US, for the money purpose, is the US federal reserve and the U S Treasury. You pull out a dollar bill. It says U S Treasury. It’s issued by the, you know, in some ways, quote-unquote, the top of the house. The top of the house guarantees it. And you carry it around with you. But when you give it to someone, you’re ultimately trusting that central authority in how you are transacting.
Strong: This can be a good thing. The value of that otherwise worthless paper bill is guaranteed because it’s issued and backed by the US government. But it can also slow things down. And though we now take for granted being able to transfer money in real time, the ability to do so hasn’t been around that long.
Farooq: Payments actually do, as a technology, evolve somewhat slowly. Just to give you an example, the U S recently, a couple of years back, launched the real-time payments scheme, which literally was the first new payments, you know, sort of, rails in the U S for decades. As crazy as that sounds.
Strong: A payment rail is the infrastructure that lets money move from one place to another. And those “real time payments” are a big deal because until recently when money left your account it took time, often days, before it reached its destination.
It’s why we can send money through apps like Venmo and hear the ding that it’s been received on the other person’s phone just a few seconds later. Also, Venmo’s chief competitor, called Zelle, only exists because of unprecedented cooperation between otherwise competing banks.
Farooq: I think where the world is going is towards more open platforms where it’s not just one party’s capabilities, but multiple parties’ capabilities that come together. And the value that is generated is by the ability for anyone to connect to anyone else. So I think what we are seeing is a rapid evolution in the digital sphere where more and more payment types, whether they are wholesale or retail are going into new modes, new rails, 24/7, 365, the ability to pay anyone anywhere in any currency. All those things are basically getting accelerated.
Strong: This is where cryptocurrencies could come in. Which isn’t just about digital money.
Farooq: We believe that there’s a path forward where money can be smarter itself. So you can actually program the coin and it can control who it goes to.
Strong: In other words, the trust we usually place in banks or governments would be transferred to an algorithm and a shared ledger.
Farooq: So you’re almost relying on that decentralized nature of the algorithm and say, “I think I can trust your token coming to me” because there’s, you know, X… X thousand or X hundred-thousand copies of a ledger that shows you as the owner of that token. And then when you give it to me, All those copies get updated. And now this shows me as the owner of that token.
Strong: And not only could this make payments faster and more seamless. It could also help people who’ve been largely excluded from the banking system.
Farooq: No matter what we do, we cannot really get around this Know Your Customer issue. And I think, you know, our view is that the tech is almost there, but the regulation and the infrastructure around it is not there yet. But, what we do want to do is we want to create these decentralized systems where these people can, over time, be included.
Strong: But sorting out the tech… is just one side of the coin. There’s also a need for better regulation.
Farooq: But I think it’s unfortunately a little bit more than what a bank could do. I think this.. some of these things rise to the level of like, you know, how does a government, or how does a state really enable identity at a global level? And I think that’s why when you look at China or you look at Nordics or some of those countries, I mean, you have national IDs and you have a very standardized method of knowing who someone is.
Strong: And the shift it allows in banking can be transformative…
Farooq: So if you look at a country like India, India has made dramatic progress in how many people have gone from being unbanked to banked in terms of having a wallet on their mobile phone. So I think these technologies are going to turbocharge people’s ability to come into this ecosystem. What I would hope as someone who grew up in the developing world before migrating here is that you would make those connections so, you know, everyone in those countries has access to markets—to bigger markets. So I mean, whether you’re sitting in Sub-Saharan Africa or you’re sitting in like, you know, a village in India or Pakistan or Bangladesh, wherever, you can actually sell something through Amazon and get paid for it. I mean, you know, those sorts of things. I think there’s tremendous potential human potential that could be unlocked if we could take payments in a digital manner to some of those parts of the world.
Strong: And this vision?… extends not only to connecting anyone, anywhere to a bank… but also anything with an internet connection.
Farooq: doing some initial R and D work in the IOT space, which is, if, you know, I mean, if one day your fridge had to order milk by itself. Like, does it have to go through your bank or could it just send the money to someone who’ll deliver your milk?
McLaughlin: Every device you use has potential to be a commerce device and our network brings that together.
Strong: Ed McLaughlin is president of operations and technology for MasterCard. He’s speaking at our A-I conference, EmTech Digital.
McLaughlin: So, what all of that connectivity results in? Is.. bringing together pretty much every financial institution in the world, tens of millions of merchants, governments, tech CO’s, and all of that, which results in billions of transactions a year we see. MasterCard across all of those devices and cards is serving about two and a half billion accounts. So we get the data and transactions from a Facebook sized population, if you think about that… And as far as the scope goes, we’ve been probably seeing 20 to 25% of all internet transactions outside of China—since there was an internet.
Strong: But this connectivity creates its own set of new problems. Maybe you’ve had the experience of going out of town and suddenly your card stops working because the change of location triggered a fraud alert.
McLaughlin: One of the keys in applying AI is how you frame the question and our teams very early on and said it wasn’t to stop transactions. It was to make sure as many good transactions as possible made it through.
Strong: Another key is to have an abundance of data.
McLaughlin: It’s a massive in-memory grid in our network that holds over 2 billion card profiles with about 200 analytical vectors on it. And we make decisions in every transaction that flows through. We have less than 50 milliseconds to make that decision. So in order to do that, we have 13 different AI technologies that we’ve modeled and experimented over the years that we apply to it.
Strong: Banks are also turning to A-I to look for money laundering. In the physical world, organized crime is often hidden behind the storefronts of real businesses. And in the digital world? Hiding is even easier.
Illegal money can quickly change hands dozens of times and cross borders until there’s no clear trail back to its source. It’s a massive problem. And most of it goes undetected. It’s possible only one percent of the profits earned by criminals gets caught. And the turmoil of the global economy over the last year has only made things worse.
McLaughlin: Our adversary.. They’re using AI too. And if you look online, it’s just bots fighting bots. So you have to pick up things you weren’t looking for before, like low and slow attacks where they stay inside, what looks like acceptable tolerances, but they’re constantly probing or doing a tumbler attack on your systems. Hard to pick up. When COVID hit, you know, the world moved online. Spending patterns shifted dramatically. And what we were able to do because the AI’s are rich enough and look at so many different variables.. We were able to really tell you’re still you and you’re just behaving a little bit differently.
Strong: And the types of attacks change too…
McLaughlin: So we saw one attack factor, which was pretty amazing is they thought, okay, people won’t block transactions for personal protective gear. It’s a specific merchant class we have. And we saw the fraudsters pile on in trying to get transactions through because they figured nobody would be blocking. The good news is we look at enough other elements that we could immediately pick that up and block those transactions.
Strong: They’re building machine learning tools to identify patterns of normal activity. And to flag outliers when they’re detected. Humans can then double check those alerts and approve or reject them.
McLaughlin: We constantly have AIs running also, not just blocking the fraud or looking at it, but I’m just calling it weirdness detection—where we’re constantly predicting what we would expect to see. In fact it’s a great way to step into AI because you have KPIs you’re already tracking. Try to start predicting them. When you see something which is an immediate deviation from it, the first thing we actually do is say, what’s going on here? So we may see something the model hasn’t caught up to, we just throw a rule to block it. And we can do that instantly.
Strong: The payments industry used to be slow moving… but it’s adapting to a world where any device might one day be connected to a payments network… including self driving cars.
McLaughlin: So whether you’re using your browser to order online, if it’s your iPhone, we’re using an Apple Pay to tap, or Mercedes just announced that, uh, they’re going to be connecting their cars to gas pumps. So you can simply drive up and authorize your transaction, right from your car. And in fact, as things move away from the card and to devices, we’re seeing even more data coming in through the network.
Strong: We’ll be back… right after this.
[MIDROLL]
Strong: With more and more of our financial lives being documented, tracked and mediated online, that data turns into fodder for AI—which is being enlisted into a whole host of other roles with payments.
Woodward: People have a really complex relationship with their money. It can be stressful. It’s often boring a lot of the time.
Strong: Josh Woodward leads the Google Pay team for the US. He sees it as an opportunity to change not just payments…but the entire experience of how people think about…and engage with…their money.
Woodward: And so what we’re trying to do as a team is think about how can we simplify that relationship with money where people feel in control and they feel confidence when they’re using our app and seeing how their spending is going in and out.
Strong: Google Pay began as a peer to peer payment solution—where the main goal was digitizing the plastic cards in your wallet. But over the years, it’s evolved into a tool meant to help you more holistically manage your finances, and relationships with businesses.
Strong: And it’s taken some cues from social media. Instead of card numbers or accounts, transactions are organized around pictures of people and businesses you’ve recently paid.
Woodward: We realized that transactions, in some ways, the.. the money, that the digits, the dollars and cents, is secondary. It’s a lot more about the person or the memory around that transaction. So we’ve tried to bring that out. Similarly, we’ve taken that same relationship based design and applied it to businesses. And this is something that’s very different. So when you look today at our home screen, // what you see is actually the icon of the business. And when you tap on that, you are taken to that business page where you can actually. Really see, like your relationship with the business. If you have a loyalty card you can see that there, you can see how your points are progressing. So the next time you go buy, you can get 20% off for example. And so we’ve tried to create this… Really almost like a threaded relationship of all your activity with that business inside the Google Pay app a little bit like Gmail, threaded email messages.
Strong: It also lets users sort transactions in a way that mirrors a web search.
Woodward: So you can do things like search for food. And you’ll get all of the transactions at places where you bought food and Google Pay can understand that this restaurant, for example, is a restaurant. You don’t have to go in and manually categorize that. Or you can get more specific and do things like a search for Mexican restaurants. And it’ll just take that subset of Mexican restaurants. There’s no part of that transaction that has the phrase, Mexican restaurant in it. Google Pay’s able to make that connection for you.
Strong: And using computer vision…it can sort through photos of receipts.
Woodward: What we’ve been able to do in Google Pay, again with someone’s permission, this feature is off by default, is that you can say, I want all the photos I’ve taken of receipts to be searchable in Google Pay. And what that allows you to do is actually search very specifically for individual items that are printed on the receipt. So for example, a couple of months ago, before Christmas, I bought a shirt, uh, it was a Christmas present from Lulu. I can go into Google Pay now and search for “shirt.” And that Lulu receipt comes up.
Strong: It’s designed to give users a greater sense of control over their spending.
Woodward: It creates a place where you get that full picture. And that’s what we’ve seen. Time and time again, in the research and in talking to people is that different apps have provided different slices of that picture, but being able to bring it all together is really what we aspire to.
[music transition]
Strong: It’s one more way our lives might become a little easier and more efficient with the help of technology… But also where the gathering… filtering… and processing… of vast amounts of personal data raises big questions… even before we get to things like paying with our faces or gestures… or how all of that data… might mix with the rest of our massive data trails.
And longer-term, what would it mean for companies like Facebook to establish their own currencies and take over the global payments system?
It’s worth asking whether we as consumers really get more control over our finances… or companies get more control over us…
[MUSIC IN]
Next episode…
Bennett: We couldn’t have imagined something like Siri or Alexa. You know we just thought we were doing just generic phone voice messaging… and so in 2011 when suddenly Siri appeared, it’s like, “I’m WHO??” [laughing]… “WHAT??”…
Strong: We look at what it takes to make a voice… and how that’s rapidly changing.
[CREDITS]
Strong: This episode was produced by Anthony Green, with help from Jennifer Strong, Karen Hao, Will Douglas Heaven and Emma Cillekens. We’re edited by Michael Reilly. Special thanks to our events team for recording part of this episode at our AI conference: Emtech Digital.
You may like
-
What’s changed in the US since the breakthrough climate bill passed a year ago?
-
What’s next for China’s digital currency?
-
The Download: what’s next for the moon, and facial recognition’s stalemate
-
What’s next for the moon
-
Introducing MIT Technology Review Roundtables, real-time conversations about what’s next in tech
-
What “rewilding” means—and what’s missing from this new movement
My senior spring in high school, I decided to defer my MIT enrollment by a year. I had always planned to take a gap year, but after receiving the silver tube in the mail and seeing all my college-bound friends plan out their classes and dorm decor, I got cold feet. Every time I mentioned my plans, I was met with questions like “But what about school?” and “MIT is cool with this?”
Yeah. MIT totally is. Postponing your MIT start date is as simple as clicking a checkbox.
Now, having finished my first year of classes, I’m really grateful that I stuck with my decision to delay MIT, as I realized that having a full year of unstructured time is a gift. I could let my creative juices run. Pick up hobbies for fun. Do cool things like work at an AI startup and teach myself how to create latte art. My favorite part of the year, however, was backpacking across Europe. I traveled through Austria, Slovakia, Russia, Spain, France, the UK, Greece, Italy, Germany, Poland, Romania, and Hungary.
Moreover, despite my fear that I’d be losing a valuable year, traveling turned out to be the most productive thing I could have done with my time. I got to explore different cultures, meet new people from all over the world, and gain unique perspectives that I couldn’t have gotten otherwise. My travels throughout Europe allowed me to leave my comfort zone and expand my understanding of the greater human experience.
“In Iceland there’s less focus on hustle culture, and this relaxed approach to work-life balance ends up fostering creativity. This was a wild revelation to a bunch of MIT students.”
When I became a full-time student last fall, I realized that StartLabs, the premier undergraduate entrepreneurship club on campus, gives MIT undergrads a similar opportunity to expand their horizons and experience new things. I immediately signed up. At StartLabs, we host fireside chats and ideathons throughout the year. But our flagship event is our annual TechTrek over spring break. In previous years, StartLabs has gone on TechTrek trips to Germany, Switzerland, and Israel. On these fully funded trips, StartLabs members have visited and collaborated with industry leaders, incubators, startups, and academic institutions. They take these treks both to connect with the global startup sphere and to build closer relationships within the club itself.
Most important, however, the process of organizing the TechTrek is itself an expedited introduction to entrepreneurship. The trip is entirely planned by StartLabs members; we figure out travel logistics, find sponsors, and then discover ways to optimize our funding.
In organizing this year’s trip to Iceland, we had to learn how to delegate roles to all the planners and how to maintain morale when making this trip a reality seemed to be an impossible task. We woke up extra early to take 6 a.m. calls with Icelandic founders and sponsors. We came up with options for different levels of sponsorship, used pattern recognition to deduce the email addresses of hundreds of potential contacts at organizations we wanted to visit, and all got scrappy with utilizing our LinkedIn connections.
And as any good entrepreneur must, we had to learn how to be lean and maximize our resources. To stretch our food budget, we planned all our incubator and company visits around lunchtime in hopes of getting fed, played human Tetris as we fit 16 people into a six-person Airbnb, and emailed grocery stores to get their nearly expired foods for a discount. We even made a deal with the local bus company to give us free tickets in exchange for a story post on our Instagram account.
Tech
The Download: spying keyboard software, and why boring AI is best
Published
1 year agoon
22 August 2023By
Terry Power
This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.
How ubiquitous keyboard software puts hundreds of millions of Chinese users at risk
For millions of Chinese people, the first software they download onto devices is always the same: a keyboard app. Yet few of them are aware that it may make everything they type vulnerable to spying eyes.
QWERTY keyboards are inefficient as many Chinese characters share the same latinized spelling. As a result, many switch to smart, localized keyboard apps to save time and frustration. Today, over 800 million Chinese people use third-party keyboard apps on their PCs, laptops, and mobile phones.
But a recent report by the Citizen Lab, a University of Toronto–affiliated research group, revealed that Sogou, one of the most popular Chinese keyboard apps, had a massive security loophole. Read the full story.
—Zeyi Yang
Why we should all be rooting for boring AI
Earlier this month, the US Department of Defense announced it is setting up a Generative AI Task Force, aimed at “analyzing and integrating” AI tools such as large language models across the department. It hopes they could improve intelligence and operational planning.
But those might not be the right use cases, writes our senior AI reporter Melissa Heikkila. Generative AI tools, such as language models, are glitchy and unpredictable, and they make things up. They also have massive security vulnerabilities, privacy problems, and deeply ingrained biases.
Applying these technologies in high-stakes settings could lead to deadly accidents where it’s unclear who or what should be held responsible, or even why the problem occurred. The DoD’s best bet is to apply generative AI to more mundane things like Excel, email, or word processing. Read the full story.
This story is from The Algorithm, Melissa’s weekly newsletter giving you the inside track on all things AI. Sign up to receive it in your inbox every Monday.
The ice cores that will let us look 1.5 million years into the past
To better understand the role atmospheric carbon dioxide plays in Earth’s climate cycles, scientists have long turned to ice cores drilled in Antarctica, where snow layers accumulate and compact over hundreds of thousands of years, trapping samples of ancient air in a lattice of bubbles that serve as tiny time capsules.
By analyzing those cores, scientists can connect greenhouse-gas concentrations with temperatures going back 800,000 years. Now, a new European-led initiative hopes to eventually retrieve the oldest core yet, dating back 1.5 million years. But that impressive feat is still only the first step. Once they’ve done that, they’ll have to figure out how they’re going to extract the air from the ice. Read the full story.
—Christian Elliott
This story is from the latest edition of our print magazine, set to go live tomorrow. Subscribe today for as low as $8/month to ensure you receive full access to the new Ethics issue and in-depth stories on experimental drugs, AI assisted warfare, microfinance, and more.
The must-reads
I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.
1 How AI got dragged into the culture wars
Fears about ‘woke’ AI fundamentally misunderstand how it works. Yet they’re gaining traction. (The Guardian)
+ Why it’s impossible to build an unbiased AI language model. (MIT Technology Review)
2 Researchers are racing to understand a new coronavirus variant
It’s unlikely to be cause for concern, but it shows this virus still has plenty of tricks up its sleeve. (Nature)
+ Covid hasn’t entirely gone away—here’s where we stand. (MIT Technology Review)
+ Why we can’t afford to stop monitoring it. (Ars Technica)
3 How Hilary became such a monster storm
Much of it is down to unusually hot sea surface temperatures. (Wired $)
+ The era of simultaneous climate disasters is here to stay. (Axios)
+ People are donning cooling vests so they can work through the heat. (Wired $)
4 Brain privacy is set to become important
Scientists are getting better at decoding our brain data. It’s surely only a matter of time before others want a peek. (The Atlantic $)
+ How your brain data could be used against you. (MIT Technology Review)
5 How Nvidia built such a big competitive advantage in AI chips
Today it accounts for 70% of all AI chip sales—and an even greater share for training generative models. (NYT $)
+ The chips it’s selling to China are less effective due to US export controls. (Ars Technica)
+ These simple design rules could turn the chip industry on its head. (MIT Technology Review)
6 Inside the complex world of dissociative identity disorder on TikTok
Reducing stigma is great, but doctors fear people are self-diagnosing or even imitating the disorder. (The Verge)
7 What TikTok might have to give up to keep operating in the US
This shows just how hollow the authorities’ purported data-collection concerns really are. (Forbes)
8 Soldiers in Ukraine are playing World of Tanks on their phones
It’s eerily similar to the war they are themselves fighting, but they say it helps them to dissociate from the horror. (NYT $)
9 Conspiracy theorists are sharing mad ideas on what causes wildfires
But it’s all just a convoluted way to try to avoid having to tackle climate change. (Slate $)
10 Christie’s accidentally leaked the location of tons of valuable art
Seemingly thanks to the metadata that often automatically attaches to smartphone photos. (WP $)
Quote of the day
“Is it going to take people dying for something to move forward?”
—An anonymous air traffic controller warns that staffing shortages in their industry, plus other factors, are starting to threaten passenger safety, the New York Times reports.
The big story
Inside effective altruism, where the far future counts a lot more than the present
October 2022
Since its birth in the late 2000s, effective altruism has aimed to answer the question “How can those with means have the most impact on the world in a quantifiable way?”—and supplied methods for calculating the answer.
It’s no surprise that effective altruisms’ ideas have long faced criticism for reflecting white Western saviorism, alongside an avoidance of structural problems in favor of abstract math. And as believers pour even greater amounts of money into the movement’s increasingly sci-fi ideals, such charges are only intensifying. Read the full story.
—Rebecca Ackermann
We can still have nice things
A place for comfort, fun and distraction in these weird times. (Got any ideas? Drop me a line or tweet ’em at me.)
+ Watch Andrew Scott’s electrifying reading of the 1965 commencement address ‘Choose One of Five’ by Edith Sampson.
+ Here’s how Metallica makes sure its live performances ROCK. ($)
+ Cannot deal with this utterly ludicrous wooden vehicle.
+ Learn about a weird and wonderful new instrument called a harpejji.
Tech
Why we should all be rooting for boring AI
Published
1 year agoon
22 August 2023By
Terry Power
This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
I’m back from a wholesome week off picking blueberries in a forest. So this story we published last week about the messy ethics of AI in warfare is just the antidote, bringing my blood pressure right back up again.
Arthur Holland Michel does a great job looking at the complicated and nuanced ethical questions around warfare and the military’s increasing use of artificial-intelligence tools. There are myriad ways AI could fail catastrophically or be abused in conflict situations, and there don’t seem to be any real rules constraining it yet. Holland Michel’s story illustrates how little there is to hold people accountable when things go wrong.
Last year I wrote about how the war in Ukraine kick-started a new boom in business for defense AI startups. The latest hype cycle has only added to that, as companies—and now the military too—race to embed generative AI in products and services.
Earlier this month, the US Department of Defense announced it is setting up a Generative AI Task Force, aimed at “analyzing and integrating” AI tools such as large language models across the department.
The department sees tons of potential to “improve intelligence, operational planning, and administrative and business processes.”
But Holland Michel’s story highlights why the first two use cases might be a bad idea. Generative AI tools, such as language models, are glitchy and unpredictable, and they make things up. They also have massive security vulnerabilities, privacy problems, and deeply ingrained biases.
Applying these technologies in high-stakes settings could lead to deadly accidents where it’s unclear who or what should be held responsible, or even why the problem occurred. Everyone agrees that humans should make the final call, but that is made harder by technology that acts unpredictably, especially in fast-moving conflict situations.
Some worry that the people lowest on the hierarchy will pay the highest price when things go wrong: “In the event of an accident—regardless of whether the human was wrong, the computer was wrong, or they were wrong together—the person who made the ‘decision’ will absorb the blame and protect everyone else along the chain of command from the full impact of accountability,” Holland Michel writes.
The only ones who seem likely to face no consequences when AI fails in war are the companies supplying the technology.
It helps companies when the rules the US has set to govern AI in warfare are mere recommendations, not laws. That makes it really hard to hold anyone accountable. Even the AI Act, the EU’s sweeping upcoming regulation for high-risk AI systems, exempts military uses, which arguably are the highest-risk applications of them all.
While everyone is looking for exciting new uses for generative AI, I personally can’t wait for it to become boring.
Amid early signs that people are starting to lose interest in the technology, companies might find that these sorts of tools are better suited for mundane, low-risk applications than solving humanity’s biggest problems.
Applying AI in, for example, productivity software such as Excel, email, or word processing might not be the sexiest idea, but compared to warfare it’s a relatively low-stakes application, and simple enough to have the potential to actually work as advertised. It could help us do the tedious bits of our jobs faster and better.
Boring AI is unlikely to break as easily and, most important, won’t kill anyone. Hopefully, soon we’ll forget we’re interacting with AI at all. (It wasn’t that long ago when machine translation was an exciting new thing in AI. Now most people don’t even think about its role in powering Google Translate.)
That’s why I’m more confident that organizations like the DoD will find success applying generative AI in administrative and business processes.
Boring AI is not morally complex. It’s not magic. But it works.
Deeper Learning
AI isn’t great at decoding human emotions. So why are regulators targeting the tech?
Amid all the chatter about ChatGPT, artificial general intelligence, and the prospect of robots taking people’s jobs, regulators in the EU and the US have been ramping up warnings against AI and emotion recognition. Emotion recognition is the attempt to identify a person’s feelings or state of mind using AI analysis of video, facial images, or audio recordings.
But why is this a top concern? Western regulators are particularly concerned about China’s use of the technology, and its potential to enable social control. And there’s also evidence that it simply does not work properly. Tate Ryan-Mosley dissected the thorny questions around the technology in last week’s edition of The Technocrat, our weekly newsletter on tech policy.
Bits and Bytes
Meta is preparing to launch free code-generating software
A version of its new LLaMA 2 language model that is able to generate programming code will pose a stiff challenge to similar proprietary code-generating programs from rivals such as OpenAI, Microsoft, and Google. The open-source program is called Code Llama, and its launch is imminent, according to The Information. (The Information)
OpenAI is testing GPT-4 for content moderation
Using the language model to moderate online content could really help alleviate the mental toll content moderation takes on humans. OpenAI says it’s seen some promising first results, although the tech does not outperform highly trained humans. A lot of big, open questions remain, such as whether the tool can be attuned to different cultures and pick up context and nuance. (OpenAI)
Google is working on an AI assistant that offers life advice
The generative AI tools could function as a life coach, offering up ideas, planning instructions, and tutoring tips. (The New York Times)
Two tech luminaries have quit their jobs to build AI systems inspired by bees
Sakana, a new AI research lab, draws inspiration from the animal kingdom. Founded by two prominent industry researchers and former Googlers, the company plans to make multiple smaller AI models that work together, the idea being that a “swarm” of programs could be as powerful as a single large AI model. (Bloomberg)
You must be logged in to post a comment Login