The state is partnering with five companies to create generative AI tools using technologies developed by tech giants such as Microsoft-backed OpenAI and Google-backed Anthropic that would ultimately help the state provide better services to the public, administration officials said.

“It is a very good sign that a lot of these companies are putting their focus on using GenAI for governmental service delivery,” said Amy Tong, secretary of government operations for California.

The companies will now start a six-month internal trial where the tools are tested and evaluated by state workers. The companies will be paid $1 for their proposals. The state, which faces a significant budget deficit, can then reassess whether any tools could be fully implemented under new contracts. All the tools are considered low risk, meaning they don’t interact with confidential data or personal information, an administration spokesperson said.

Newsom, a Democrat, touts California as a global hub for AI technology, noting 35 of the world’s top 50 AI companies are located in the state. He signed an executive order last year requiring the state to start exploring responsible ways to incorporate generative AI by this summer, with a goal of positioning California as an AI leader. In January, the state started asking technology companies to come up with generative AI tools for public services. Last month, California was among one of the first states to roll out guidelines on when and how state agencies could buy such tools.

Generative AI, a branch of AI that can create new content such as text, audio and photos, has significant potential to help government agencies become more efficient but there’s also an urgent need for safeguards and oversight to limit risks, state officials and experts said. In New York City, an AI-powered chatbot created by the city to help small businesses was found to dole out false guidance and advise companies to violate the law. The rapidly growing technology has also raised concerns about job loss, misinformation, privacy and automation bias.

While state governments are struggling to regulate AI in the private sectors, many are exploring how public agencies can leverage the powerful technology for public good. California’s approach, which also requires companies to disclose what large language models they use to develop AI tools, is meant to build public trust, officials said.

The state’s efforts of testing the tools extensively and allowing state workers to provide feedback are some of the best practices to limit potential risks, said Meredith Lee, chief technical advisor for UC Berkeley’s College of Computing, Data Science, and Society. The challenge is how the state ensures it continues testing and learning the tools’ potential risks after deployment.

“This is not something where you just work on testing for some small amount of time and that’s it,” Lee said. “Putting in the structures for people to be able to revisit and better understand the deployments further down the line is really crucial.”

The California Department of Transportation is looking for tools that would analyze traffic data and come up with solutions to reduce highway traffic and make roads safer. The state’s Department of Tax and Fee Administration, which administers more than 40 programs, wants an AI tool to help its call center cut wait times and call length. The state is also seeking technologies to provide non-English speakers information about health and social services benefits in their language and to streamline the inspection process for health care facilities.

The tool is designed to assist state workers, not replace them, said Nick Maduros, director of the Department of Tax and Fee Administration.

Call center workers there took more than 660,000 calls last year. The state envisions the AI technology listening along to those calls and pulling up specific tax code information associated with the problem the caller is describing. The worker could decide whether to use the information. Currently, call center workers have to simultaneously listen to the call and manually look up the code, Maduros said.

“If it turns out it doesn’t serve the public better, then we’re out $1,” Maduros said. “And I think that’s a pretty good deal for the citizens of California.”

Tong wouldn’t say when a successfully vetted tool would be deployed, but added that the state is moving as fast as it can.

“The whole essence of using GenAI is it doesn’t take years,” Tong said. “GenAI doesn’t wait for you.”