According to researchers at the University of Florida, Alejandro Lopez-Lira and Yuehua Tang, the answer is yes. In their paper Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models, they found that ChatGPT outperformed today’s analysis methods in predicting stock market sentiment for a specific company based on news headlines. “Our results suggest that incorporating advanced language models into the investment decision-making process can yield more accurate predictions and enhance the performance of quantitative trading strategies,” they wrote in the study. 

Tang and Lopez-Lira explained to Fortune that the study started as a fun experiment to see what the chatbot’s potential could be, and they didn’t necessarily expect the conclusive results they got. To run the study, they prompted ChatGPT to decide how headlines about a company would affect its stock price, and then give its reasoning for why a headline was beneficial, detrimental, or inconclusive for the company’s stock. Over a large-scale quantitive analysis of prompt results, they found that “ChatGPT sentiment scores exhibit a statistically significant predictive power on daily stock market returns.” 

One straightforward implication of their study is that if ChatGPT can produce accurate market analysis, this will significantly alter the finance job market. The researchers explained that using these language learning models could help analysts be more accurate but, on the flip side, likely lead to fewer analysts needed and therefore fewer jobs. 

Their findings are particularly significant to asset managers, many of whom have already begun to design their A.I.-based algorithms for managing funds. Lopez-Lira explained that many of the hedge funds he was talking to about their findings already had “way more sophisticated” algorithms built specifically for stock forecasting. Some ETFs already use tools to manage their funds, and the researchers see this being increasingly adopted. Lopez-Lira noted that fund management companies that build algorithms use more advanced methods than reading headlines, and don’t use existing chatbots to design their algorithm specifically for picking stocks or assets they manage. 

If you’re opening ChatGPT right now on another browser to start stock picking, the researchers caution that their research doesn’t necessarily apply to individual investors. “To implement this, you need the infrastructure to make it and a quant-based strategy, rather than just the individual investor trying to Google one headline,” explained Tang. Lopez-Lira added that on any individual prompt, ChatGPT’s accuracy was only 51%. “It works well because when you’re aggregating across multiple companies on multiple days, you get a result, but for one given headline is basically a little bit better than tossing a coin,” he said. 

And the researchers also pointed out what anyone who has used the chatbot has seen—it’s far from perfect. For example, in the study, the chatbot consistently did not understand that a headline about a CEO or CFO selling stock was bad news for a company—the chatbot read those headlines as having an “unknown” impact on the stock because it didn’t know how much was sold. Lopez-Lira explained that there were moments like these where the algorithm missed connections that human intuition would pick up on.

Yet they did not doubt that the technology would be transformational and change the way that asset managers, investors, and regulators think about stock trading. “This is probably one of the earliest studies [on this topic]; we just had an interesting experiment,” said Lopez-Lira. “This is a disruptive creation,” he added.