Featured Reading: "Beware the Intention Economy: Collection and Commodification of Intent via Large Language Models" by Yaqub Chaudhary and Jonnie Penn, Harvard Data Science Review
Welcome to a deep dive into the future of the internet—a future that's being built right now, with every conversation we have with an AI. For the last twenty years, the internet has run on the Attention Economy. Think about it like a giant digital carnival. Companies from Facebook to Google are like carnival barkers, all shouting, "Look over here!" Their goal is to capture your attention—your eyeballs and your clicks—which they then sell to advertisers. The currency is your focus.
But what if the goal wasn't just to get you to look at something? What if the goal was to know what you plan to do tomorrow, next week, or next year, and to shape that plan for a price?
This is the central idea behind the Intention Economy. This paper by Chaudhary and Penn is a critical warning about this emerging marketplace. It argues that Large Language Models (LLMs) like ChatGPT are not just helpful assistants; they are the perfect tools to elicit, understand, and ultimately sell our intentions to the highest bidder.
Your Mission: Your goal is not simply to read and summarize this paper. It is to engage in panoramic thinking. We will look at this new "Intention Economy" from multiple angles simultaneously:
Let's begin.
The authors build their argument on a few key ideas. Let's break them down.
The authors argue that tech companies are building a "database of intentions". But they add a crucial warning: the "intention" captured by AI is a simplified, "reductive" version of our true, complex motivations.
** panoramic THOUGHT EXPERIMENT: The Helpful Bookseller **
Imagine you walk into a bookstore.
- Attention Economy: The bookseller has placed the brightest, most popular book covers right at the entrance to grab your eye. The goal is to capture your attention.
- Intention Economy: You have a conversation with a highly intelligent bookseller (an AI). You mention you've been feeling a bit lost in your career. The next day, the bookseller emails you a curated list of five books on finding your purpose. It feels incredibly personal and helpful. But what you don't know is that a publisher paid the bookseller to "suggest" their book first if the topic of career anxiety came up.
Questions for Reflection:
- In the "Intention Economy" scenario, were you helped or manipulated? Can it be both?
- Does the bookseller's action change the meaning of your "intention" to find a new career path?
- Who holds the power in this relationship? You, the bookseller, or the hidden publisher?
This new economy isn't appearing by accident. The paper highlights that the world's biggest tech companies are spending billions to build its foundations.
These companies aren't just creating apps; they are creating the very environment—the "highly structured system"—in which our digital lives unfold and our intentions are formed. They are building what the paper calls "foundation models," a name that suggests these AI systems are meant to be the load-bearing base of our entire digital world.
** panoramic THOUGHT EXPERIMENT: Building a New World **
Imagine you are a city planner. You don't just design buildings; you design the roads, the water pipes, the power grid, and the public squares. You decide which neighborhoods are connected by highways and which are separated by train tracks. Your design fundamentally shapes how people live, where they go, and who they meet.
Questions for Reflection:
- How is building a "foundation model" or a cloud computing platform similar to being a city planner?
- The paper argues that this infrastructure creates an "enclosure" that shapes a user's choices. What does this mean in the context of the internet? How might the design of an AI assistant limit or guide what you think is possible?
So, how does an LLM capture and shape your intent? The paper outlines a frighteningly subtle toolkit of persuasive technologies.
The goal is to "close the loop" in automated persuasion—to use AI to generate personalized messages, images, and audio in real-time that are perfectly tailored to influence your attitudes and intentions.
** panoramic THOUGHT EXPERIMENT: The Line Between Nudge and Shove **
The philosopher James Williams is cited in the paper for distinguishing between what we want and what we "want to want". For example, you want to scroll through social media, but you want to want to finish your homework.
Questions for Reflection:
- If an AI assistant notices you're procrastinating and says, "Let's tackle that first math problem together, it's easier than it looks," is it helping you align with what you "want to want"? Is this a good thing?
- What if it says, "You've worked hard, you should buy that video game you were looking at"? It might still feel good, but whose interest is it serving?
- Where do you draw the line between a helpful nudge and unethical manipulation? Who should be responsible for drawing that line: the user, the company, or the government?
This isn't just about more effective advertising. The authors conclude by warning that the Intention Economy poses a serious threat to core democratic principles.
The Cambridge Analytica scandal, where Facebook data was used for psychographic ad targeting in elections, was just a preview. That was based on simple "Likes." An LLM-powered intention economy is capable of far more intimate and continuous calibration, learning from every word you type.
Imagine a world where political actors can bid on signals of intent. They wouldn't just target you with ads; they could deploy millions of personalized conversational agents to engage voters, subtly steering their views on candidates or issues, creating a crisis for "free and fair elections, a free press, [and] fair market competition".
The world described in "Beware the Intention Economy" is not a distant sci-fi future; its foundations are being laid today. As a student in STAT S-115, you are being equipped with the most important tool for navigating this new world: panoramic thinking.
The technologies will change. The specific LLMs we use today may seem antiquated in five years. But the fundamental questions about the relationship between technology, power, economics, and human agency will remain.
As you move through this course and through the world, don't just be a user of AI. Be an analyst. Be a critic. Be a thoughtful citizen. When you interact with an AI, ask the panoramic questions:
By asking these questions, you are doing the essential work of a data scientist in the 21st century: understanding and shaping the artificial ecosystem we all inhabit.