Why investors are backing this former Facebook manager’s ‘explainable A.I.’ startup

Fiddler AI, a startup that makes tools to help engineers monitor machine-learning systems, said Thursday that it had raised $32 million in a funding round led by Insight Partners. Other investors include Lightspeed Venture Partners, Lux Capital, Haystack Ventures, Bloomberg Beta, Lockheed Martin, and Amazon’s Alexa Fund. 

The startup’s founder and CEO, Krishna Gade, was previously an engineering manager at Facebook where he helped build tools that enabled Facebook developers to find bugs and troubleshoot machine-learning–related problems in the network’s News Feed. The tools were intended to better understand why News Feed’s algorithms were recommending particular posts and online ads, or why certain posts became viral. 

When he started working on News Feed in 2016, Facebook was reeling from media scrutiny and user distrust related to the social network spreading propaganda and fake news in the run-up to the 2016 U.S. presidential election, Gade said. Among the challenges of A.I.-powered systems like Facebook’s News Feed is that they are black boxes, meaning that it’s “very difficult to understand how it’s making those predictions,” he said. 

But there are ways companies can get some basic explanations.

The tools that Gade developed at Facebook and that he is currently selling to other companies monitor how A.I. systems react and adapt to data through constant testing. In the case of machine learning that determines the likelihood of a person defaulting on a loan, Fiddler’s tool tests the system by changing parameters, like a person’s salary and other financial information, and recording what the system predicts.

By having a running log of how all the different information affects the predictions, companies can more easily decipher how the data affects the system’s overall performance. 

But Gade concedes that it’s still often hard to explain how A.I. systems make their predictions. If the data has been anonymized for privacy reasons, for instance, it’s more difficult to test the impact of certain attributes such as whether a lack of information about a person’s race makes the system perform worse for people of color than for white people.

Other larger companies like IBM, Microsoft, and Google are all selling tools similar to Fiddler’s that help businesses better explain their A.I. systems. They have a leg up considering they have been selling A.I. services for longer and have more customers.

Insight Partners managing director George Mathew sees Fiddler as part of a new wave of companies building A.I. developer tools for non-tech companies that lack the know-how to create their own systems.

Still, even though companies may have more choices of vendors than previously to help them better understand their A.I. systems, that doesn’t mean all their problems are solved. It’s similar to a company thinking its business is secure if it buys cybersecurity tools, but fails to follow up on the alerts the software produces.

After all, Facebook, with all of its cutting-edge technology, continues to be plagued by disinformation.  

“Those problems are product problems, community problems, and policy problems,” Gade said regarding the issues of Facebook and other social media companies. “It’s beyond technology.”