For all of 2025, despite a flurry of proposals, not a single state managed to ban “surveillance pricing.” This spring, that changed. In April, Maryland became the first state to prohibit food retailers and delivery services from using consumers’ personal data to set prices. In June, Connecticut became the second. California and New York are considering similar measures as part of a broader wave of efforts to limit surveillance pricing.
Yet the real issue is not fundamentally about pricing. It is about how companies choose to use data, algorithms, and increasingly AI: when technology lets them understand customers and workers in unprecedented detail, will they use that insight to create value or to extract it?
The question is whether companies are pricing the transaction or the person. Consider two identical Uber requests from Midtown Manhattan to Newark Airport. Most riders understand why the trip costs more on a rainy Friday afternoon than on a quiet Sunday morning; adjusting prices for weather, traffic, or supply is a transparent way to balance a market.
But consumers increasingly question something different: two riders standing on the same corner at the same moment paying different prices based on their data profiles, purchase histories, devices, or inferred willingness to pay. In the first case, the platform is pricing the ride. In the second, it is pricing the rider. The Federal Trade Commission made the stakes concrete in a 2025 study, showing how algorithms drawing on personal data can infer when consumers may have fewer alternatives, greater urgency, or a higher willingness to pay—and adjust prices or offers accordingly.
The same logic operates on the other side of the marketplace. An algorithm may offer a lower payout to a driver it predicts will accept anyway—because she is nearing a daily earnings goal or unlikely to switch apps. When companies shift from pricing a transaction’s conditions to exploiting the vulnerabilities of the people in it, they drift from market-clearing efficiency toward extraction—eroding trust, deepening worker dissatisfaction, and inviting the regulation now spreading across the country.
The debate is urgent because the ability to understand and influence individuals is about to expand dramatically. Until now, the limiting factor has been fragmentation. Each of us generates enormous amounts of data—searches, purchases, locations, streaming habits, and information from wearables—yet no single company sees more than a slice of our digital lives.
AI agents change that.
As people delegate real tasks to AI—booking travel, reordering goods, managing a move—they reveal far more than a search query ever did. Where search captured a momentary question, an agent observes the broader “job to be done.”
Consider what an AI agent managing your household might observe: it knows you’re running low on medication, that you typically shop when stressed, that you are charging the meal delivery to a corporate account, and that you rarely comparison shop. This behavioral profile becomes a roadmap for either serving your interests—finding genuine savings and filtering out manipulative offers—or exploiting your patterns for maximum extraction.
This is no longer projection. The leading AI platforms are rapidly developing agents that can understand users’ preferences, remember context, and increasingly take actions on their behalf. Bain & Company estimates that AI agents could influence $300 billion to $500 billion in U.S. commerce by 2030. This could give AI platforms a continuous and real-time understanding of people’s preferences, needs, and behaviors—including signals they may never explicitly express.
This capability can be used in two very different ways. It can power genuine personalization—an agent that finds a better fare, flags a needed refill, or filters out the noise. Or it can be turned inward—to charge each person closer to the maximum they will pay, to reach them when they are most vulnerable, and to withhold better options when they are likely to accept worse ones. The question is no longer whether companies can personalize at scale, but whether they will establish principles for how far that personalization should go.
That makes alignment the central issue. An agent that knows us this well can draw on behavioral science—the same biases and triggers that have always influenced human decisions—to observe, understand, and either serve or manipulate us. So when an agent acts on your behalf, whose interests does it serve—yours, the platform that built it, or the highest bidder for its recommendations? Given the unprecedented sums now being invested in AI, the economic incentives to monetize that influence will be immense.
This is where our research points to a different path. Drawing on interviews with more than 200 CEOs at Yale’s Program on Stakeholder Innovation and Management, we have found a consistent pattern: the most effective way to build long-term shareholder value is to grow the business as you grow the trust — creating value for stakeholders including customers, workers, suppliers, and the communities a company serves, while earning their confidence over time. As AI sharpens the ability to understand and influence each of them, the temptation will be to optimize every relationship for immediate gain. The most enduring companies will resist it, using these capabilities to deepen trust and strengthen the relationships on which lasting economic value depends.
Surveillance pricing is only the first test. How companies answer it will reveal how they intend to govern the far more powerful tools now arriving—and whether the agents acting in our name end up working for us, or on us.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
