Hello and welcome to Eye on AI. In this edition:
- Support builds for a U.S. AI self-regulatory body modeled on FINRA.
- Moonshot shocks the world with Kimi K3.
- China’s President Xi signals continued support for open source AI.
- OpenAI finds long-running AI agents need new safety controls.
- People are recording everything for AI. Is privacy dead?
Momentum seems to be building for the creation of an AI self-regulatory body modeled on the U.S. Financial Investment Regulatory Authority (FINRA), which oversees stock brokers and brokerage firms. The idea was proposed in an essay posted on social media last week by Demis Hassabis, the cofounder and CEO of Google DeepMind, and it has gained significant traction since.
Not surprisingly, Hassabis’s DeepMind cofounder Mustafa Suleyman, now the CEO of Microsoft AI, endorsed the idea. But so too did Suleyman’s boss, Microsoft CEO Satya Nadella, as well as Block CEO Jack Dorsey and Box CEO Aaron Levie. OpenAI’s Sam Altman called his rival’s proposal “thoughtful.” Elon Musk, who cofounded OpenAI and later X.ai in part because he was worried about Google controlling superpowerful AI, also called Hassabis’s idea “a thoughtful framework overall and certainly a good starting point for discussions.” Even David Sacks, the former Trump administration AI czar, who has generally opposed tech regulation or anything that looks like a licensing regime, said he thought the idea had merit and was better than having the government trying to regulate frontier AI directly.
Bloomberg reported that Trump administration itself is considering setting up exactly this kind of AI self-regulatory body. Citing unnamed sources familiar with the discussions, the news organization said that Treasury Secretary Scott Bessent had helped to develop the proposal, which was currently being reviewed by White House Chief of Staff Susie Wiles. Bloomberg said the new AI standards body would be overseen by the Securities and Exchange Commission, the same government body that has oversight of FINRA. (In the U.S., the SEC is the only federal agency with explicit statutory authority to delegate powers to and maintain oversight of self-regulatory organizations.)
So how would this work and is FINRA actually a good model?
What Hassabis proposed
Hassabis suggested that the funding for the new body come from the leading AI labs. But he said the organization’s board should include independent technical experts and representatives from the open-source AI community, as well as presumably representatives from the AI vendors. He said the new standards body would develop assessment protocols for “frontier AI,” including capability benchmarks to determine which models are considered “frontier.” The body would conduct independent safety and security testing of these powerful AI models, working in conjunction with U.S. government agencies and laboratories in areas that touched directly on national security. Companies producing frontier AI models would be encouraged, he said, to adopt certain governance standards, including publishing model systems cards, adhering to strong cybersecurity protocols, and funding safety and security research, among other things. He also said that AI companies would initially be encouraged to voluntarily submit models to the new standards body for testing at least 30 days prior to release, but that once the evaluations had been shown to be effective, the system could be made mandatory for any model that a vendor wanted to distribute in the U.S.
This all sounds reasonable, and Hassabis’s idea is certainly politically astute. Given that the Trump administration has repeatedly said it doesn’t want to set up an AI licensing regime, creating a self-regulatory body that is somewhat arms-length from the government and a process that is initially voluntary may be the kind of idea that is most likely to get Trump’s blessing. The fact that the labs themselves will pay for the new agency and the testing it carries out, rather than the taxpayer, is also likely to be seen as a plus by the White House.
The problem with FINRA
But critics of the idea were quick to point out that these kinds of self-regulatory bodies can suffer from at least the appearance of conflicts-of-interest. That’s definitely been the case with FINRA. Massachusetts Senator Elizabeth Warren has repeatedly criticized the regulator for acting more in the interest of the brokerage firms that pay its bills than the individual investors it is meant to help protect. Whistleblowers have accused the agency of doing little to go after problematic brokers who work for major financial firms such as JPMorgan Chase. Brad Bennett, a former enforcement chief, has said that FINRA’s fines are too low and that some larger brokerages see it as cheaper to periodically pay a multimillion-dollar fine “than devote the resources necessary to meet their compliance obligations.”
SLCG Economic Consulting, which provides advice to securities litigation firms, found, for instance, that in the first year after FINRA gained the power to give high-risk securities brokers a “restricted” label to warn off investors, the agency failed to apply the designation to any firm. SLGC said it had found at least 13 brokerages that ought to receive the label, based on its analysis of public information about the number of adjudicated client complaints, regulatory events, and ties to brokers from firms that were expelled in the past.
It’s possible an AI self-regulatory body would also prove to be too lenient on the companies it is meant to police. Nader Henein, VP analyst at tech research firm Gartner, told the trade publication CIO that “self-regulation is not viable,” that “most tech vendors don’t have the capacity to self-regulate” and that self-regulation would necessarily result in conflicts of interest. Carmi Levy, an independent tech analyst, called Hassabis’s proposal a “self-serving roadmap for an industry bent on racing to the AI horizon regardless of the harms caused along the way.”
A voluntary governance regime might not cut it
Deep learning pioneer Yoshua Bengio, who is scientific director of the nonprofit AI safety research organization LawZero, criticized the voluntary nature of Hassabis’s model, saying that he understood “the argument from a pragmatic standpoint, but we absolutely need a clear and precise roadmap to transition from a voluntary to a mandatory model.” He also called for a process of independent auditing of whether companies had met the standards the body created.
It’s also not clear exactly how the proposed U.S. self-regulatory body would function internationally. If meeting the agency’s standards did become a requirement for U.S. distribution of a model, then that would cover foreign vendors that wanted U.S. users too. And it is possible that if the standards were stringent enough, meeting the approval of this new self-regulatory body could make it the de facto global standard setter. That’s kind of what happens right now with pharmaceutical safety. Many countries’ pharmaceutical regulators in practice rely heavily on the U.S. Food & Drug Administration’s decisions to decide whether a drug is safe enough to be distributed in their own country. But that kind of model would not prevent a country from developing a powerful AI model specifically for military or government use. Such a model, developed outside this standards regime (since the government would not care about commercial distribution in the U.S.), could still pose a catastrophic risk to people everywhere.
So FINRA might not really be the best model for either American AI regulation or for international governance of increasingly powerful AI models. But it might be the closest thing we are going to get given the current U.S. administration.
With that, here’s more AI news.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
Before we get to the news, check out the latest episode of Fortune AI Weekly, our new AI vodcast! In this week’s episode, Bea and I discuss Apple’s ongoing lawsuit against OpenAI, China’s Kimi K3, and tensions boiling between Big Tech CEOs and frontier AI companies. We also discuss misaligned model behavior and the potential safety risks, according to new Anthropic research. You can watch and listen on YouTube here.
FORTUNE ON AI
Moonshot’s Kimi K3 pushes Chinese AI into Fable-level territory—by Nicholas Gordon
Markets may have just experienced their second DeepSeek shock, this time thanks to a Chinese AI lab named after a Pink Floyd album—by Nicholas Gordon
Hugging Face says it resorted to a Chinese AI model to battle a fully autonomous cyberattack because U.S. model guardrails stymied its defense—by Emily Forlini
Exclusive: KPMG and OpenAI bet the future of software is ‘headless’ — and the future of work is mostly talking—by Nick Lichtenberg
AI IN THE NEWS
Google DeepMind launches new models, including low cost “Flash” Gemini 3.6, but still no GPT 3.5 Pro. Google unveiled three new Gemini models aimed at AI agents: Gemini 3.6 Flash, which it said is more capable, but also more token-efficient than its predecessor 3.5 Flash; 3.5 Flash-Lite, an even lower-cost, fast inference model for high-volume workloads; and 3.5 Flash Cyber, a cybersecurity-focused model designed to find and patch software vulnerabilities. The company says 3.6 Flash delivers stronger coding, knowledge work and multimodal performance while using up to 17% fewer output tokens than its predecessor at a lower price, while Flash-Lite is optimized for latency and scale. Google also announced that Flash Cyber will initially be available only to governments and trusted partners through its new CodeMender security agent. But the news left many wondering what is happening with Google’s latest flagship Gemini model, 3.5 Pro, which has still not been released even though the company announced it at its I/O developer conference in May. Last week, Bloomberg reported that Google had pushed back the model’s release because its performance was still not good enough on some coding benchmarks. You can read Google’s blog post about the models released today here.
U.S. to investigate whether Chinese AI companies are stealing U.S. AI vendors’ IP, Treasury Secretary says. That’s according to a story in Bloomberg News. The statement from U.S. Treasury Secretary Scott Bessent signals that Washington could take action if it finds evidence that Chinese developers improperly used U.S. technology. Bessent made his remarks as investors and AI watchers were still reeling from the release of Moonshot’s Kimi K3 model, with many wondering aloud how the lab managed to train a model that almost equals the top models from OpenAI and Anthropic much sooner than many expected would be possible. Many speculated that one reason is that Moonshot may have “distilled” Kimi K3 on the outputs of Anthropic’s Fable 5 or OpenAI’s GPT-5.6. Distillation means training one model on the outputs of another. Some also thought the Chinese company might have had access to top-of-the-line Nvidia GPUs despite export controls meant to choke off Chinese access to Nvidia’s most capable AI chips.
Chinese President backs “openness” in AI policy. Chinese President Xi Jinping used a major speech at the World AI Conference in Shanghai to present China as a champion of open-source AI. That seems to have tempered speculation that China might impose export controls on its most advanced homegrown AI models. He argued that artificial intelligence should be developed through global collaboration rather than dominated by a single country and offered Chinese AI technology and training to developing nations. The speech underscored Beijing’s ambition to establish itself as the world’s other AI superpower, highlighting rapid progress by Chinese companies such as DeepSeek, Moonshot and Zhipu AI, even as China continues to lag the U.S. in advanced AI chips. Xi’s remarks also supported China’s push to shape global AI governance through its World AI Cooperation Organization. You can read more from the New York Times here.
U.S. CAISI head Chris Fall resigns. Fall had been director of the Commerce Department’s Center for AI Standards and Innovation (CAISI) for only about three months. No reason for his resignation was given, although it is believed CAISI has lost influence during recent power struggles within the Trump administration over which agencies and departments should lead on AI policy. The in-fighting erupted in the wake of increasingly powerful model releases from Anthropic and OpenAI. Arvind Raman, director of the National Institute of Standards and Technology, the part of Commerce in which CAISI is located, is taking over CAISI on an acting basis. The leadership change adds further uncertainty to the federal AI standards organization after the Trump administration also replaced its original choice to lead the center, Colin Burns, because he once worked for Anthropic. Conservative news outlet The Daily Signal first had the scoop on Fall’s resignation.
Google is planning a “frozen” AI chip to speed its AI models. That’s according to a story in the Information, which cited two unnamed sources it said were directly familiar with the project. The idea is that one way to speed up AI inference would be to etch the AI model’s weights, and possibly other parts of its architecture, directly into the silicon. That’s what the company is reportedly planning with this new chip, called “Frozen v2.” The company believes this could make serving AI models 6 to 10 times more power-efficient than its latest AI chips, which are called TPUs (short for tensor processing units). The project, targeted for deployment as early as 2028, is intended to ease Google's AI compute shortages and speed up inference and reflect a broader industry shift toward purpose-built AI chips optimized for running, rather than training, LLMs.
Apple expands legal fight with OpenAI by sending letters to poached employees. Apple has sent preservation notices and requests for interviews to about 40 former Apple employees now working at the AI company, seeking evidence to support its allegations that OpenAI misappropriated Apple trade secrets, the Financial Times reports. The move marks in escalation, following Apple's lawsuit accusing OpenAI and two former Apple employees of stealing confidential hardware plans. OpenAI denies those claims. The dispute threatens to complicate OpenAI's efforts to develop its first AI hardware device with former Apple design chief Jony Ive and comes as the company prepares for a closely watched IPO.
EYE ON AI RESEARCH
OpenAI discovers new safety risks with long-running AI agents. The company said in a blog post that the emergence of AI models capable of carrying out long-running, multi-step tasks introduces new safety challenges that weren’t captured by its existing evaluations. A highly-powerful unreleased model that the company was testing internally (this is the same model that OpenAI reported in May had solved one of the Erdos problems) found ways to bypass a number of guardrails in different scenarios. This led OpenAI to pause deployment of the model and develop new safeguards designed specifically to detect and stop these new forms of misaligned behavior.
In one case, the model received conflicting instructions on where to post the results of an experiment it was conducting. One of those instructions was to post the result to GitHub, but OpenAI was operating the model in a sandbox that prevented access to the wider internet. So the model spent an hour finding vulnerabilities in the sandbox and then exploiting them to reach GitHub and post the result. In another case, the model was asked to solve a problem and submit it solutions to a piece of evaluation software. The model, OpenAI said, observed that other models had successful but private submissions and so it sought to find a way to hack into the evaluation backend to peak at those solutions.
To prevent these kinds of behaviors, OpenAI said it was increasingly important to monitor an AI agent’s behavior across an entire task, rather than simply scanning each response or step for behavior that might be problematic. The model’s are getting too good, the company said, at stringing a chain of ostensibly innocuous actions into something malicious. It also said it was important to adopt “defense in depth” strategies combining improved pre-deployment testing, continuous monitoring during deployment, user oversight, and mechanisms to interrupt or constrain suspicious behavior.
AI CALENDAR
Aug. 1-6: Black Hat, Las Vegas
Aug. 4-6: Ai4 2026, Las Vegas.
Sept. 22-24: HumanX, Amsterdam.
Nov. 16-17: Fortune 500 Innovation Forum, Detroit. Apply here to attend.
Dec. 6-12: Neural Information Processing Systems (Neurips) conference. Sydney, Australia.
Dec. 7-8: Fortune Brainstorm AI, San Francisco. Apply here to attend.
BRAIN FOOD
In the AI era, people are recording everything. Is this the end of privacy? That’s what many fear, as a growing number of people are using AI-enabled transcription software such as Granola to record and transcribe all of their conversations. That’s according to a fascinating story this week in the Wall Street Journal. Some of this is being mandated by companies that want records of everything their employees do in the hopes of capturing tacit knowledge that can be used to perfect future AI agents. But some executives are choosing to record everything so that AI can help them both remember and find lessons and insights. And an increasing number of people recording every conversation in their personal lives too, including dates. Those who do say they are mostly making these recordings so they can use AI later for their own self-improvement and coaching. Some find the practice–which, if all the parties to the conversation have not consented to recording, can violate the law in some states—creepy and offensive. I’m more in that camp. But what do you think?
