Researchers: A.I. can play a role in eliminating bias in lending, but only with human help

But first, here’s your Latina Equal Pay Day week in review, in Haiku.

Wake up! Feed your kids,
go to work, do your job, (do
his job, maybe)

Clean your house! Check the
money, pay (some) bills, eat
some food, watch TikTok

Still up! Check homework,
pour herb tea, review the deck,
find a mistake and

Fix it! Make the pitch,
text the teacher, represent
at the ERG

And think hard! About
who it benefits that you
work so long for less.

Wishing you a profitable and equitable weekend.

Ellen McGirt
@ellmcgirt
Ellen.McGirt@fortune.com

In brief

There's fresh evidence of racial bias in the financial system.

Researchers from the New York University Stern School of Business recently published a paper examining the racial disparities in small business lending that occurred during the COVID-19 pandemic. Through their examination of loans dispersed through the $806 billion Paycheck Protection Program, the researchers discovered that Black-owned businesses were more likely to receive loans from fintech companies than traditional banks, specifically the smaller firms.

One of the paper’s most noteworthy findings was that the analysis suggests “that automation might help reduce racial discrimination in the loan origination process,” the authors wrote. Furthermore, when small banks incorporated automation into divvying loans, the researchers found that the technology “more than doubled small banks’ propensity to lend to Black-owned businesses.”

Score one for artificial intelligence?

Considering there’s been much research indicating bias as a major issue plaguing A.I.-powered software like facial-recognition technology, the NYU study suggests that automation, when carefully used, doesn’t have to amplify societal biases. Of course, the devil is in the details, or in this case, the data.

Sabrina Howell, the paper’s lead author and assistant professor of finance at the NYU Stern School of Business, tells Fortune that the researchers’ analysis doesn’t attempt to explain why the small banks did not disperse loans to as many Black-owned businesses as their larger counterparts. And it should be noted that The Independent Community Bankers of America, a trade group that represents small banks, criticized the NYU study’s methodology and said that its members “outperformed the rest of the banking industry in serving minority-owned, women-owned and veteran-owned businesses.”

Methodology aside, Howell says she could only speculate as to why the data from the study showed the discrepancy between small banks and fintech companies and larger financial firms. One possible reason for the discrepancy was that the fintech lending systems could have relied on “simple automation” in which computers merely verified if a loan recipient’s driver’s license was fraudulent or not. Small banks, on the other hand, may have relied on humans to verify each recipient’s driver’s license, potentially polluting the ID-verifying process with those people’s personal prejudices against people of color.

“In the fintech setting, you have a computer looking at the picture of the driver's license and gauging whether that driver's license is fraudulent based on millions of driver's licenses that it has seen in the past,” Howell says. “And we can easily imagine how the computer could be less biased than the person who might see a Black face, and then impose a higher bar on the license in terms of ensuring that it's not fraudulent.”

This “simple automation” is not exactly cutting-edge deep learning, the kind of technology that powers today’s facial-recognition systems and excites businesses. In fact, its simplicity might give it an advantage in being less biased than deep learning software that attempts to deduce whether a person’s face belongs to a white male or a black female, among other tasks.

As Howell explains, the PPP loan distribution process is mainly determined through basic background checks like verifying people’s driver’s licenses and corporate payroll accounts. The process doesn’t take in account “credit underwriting” or other calculations that involve credit risk, which could contain hidden proxies for race.

Theoretically, an automation system that was trained on a database of driver’s licenses would excel at the task of verifying each new driver’s license it analyzes. It’s a simple enough task, and it was trained on a relatively simple type of data.

But if financial firms were to make that automated loan dispersion task a bit more complicated, let’s say by incorporating people’s credit risks or other data like zip codes into their calculus, there’s more opportunities for societal bias to creep into the process. The more data sources an automation system relies on to make its decisions, the more hidden societal biases may be lurking in the data.

So I wouldn’t say that this study proves that A.I. can be less biased than humans at making decisions. It depends on the complexity of the decision an A.I. system is trying to make, and the data sources that inform how the A.I. makes its decisions. And after all, it was human academics at NYU who ultimately discovered this potential racial bias issue in the PPP loan dispersion process.

For a more equitable society, humans need to be monitoring and studying every important decision, automated or not, that touches people’s lives.

Jonathan Vanian 
@JonathanVanian
jonathan.vanian@fortune.com

On point

What you need to know about Latina Equal Pay Day On average, Latina workers earn 57 cents for every dollar earned by non-Hispanic white men. This year, October 21 was the day when Latina pay would catch up to that of their white male peers — which meant that it took Latinas 21 months to earn what white men earned in 12. As gender equity pay days go, it historically falls latest in the year. This year, the observance is particularly grim. Latina workers were more likely to become unemployed during Covid-19 than any other demographic, and more Latina adults slipped into poverty this year. According to the American Association of University Women, it would take at least 176 years for Latinas to achieve equal pay at this rate. And the individual wealth gap is profound Latina stands to lose about $1.1 million over a 40-year career.  More stats and resources below.
NBC News

The Dave Chappelle effect: Netflix employees stage walkout This recap does a great job explaining the ongoing uproar around both Chappelle’s final Netflix show—specifically, his remarks mocking the transgender community—and how it relates to ongoing tensions inside the company. It’s a problem that’s not going away. A Netflix employee group, Trans* Employee Resource Group and its supporters, has published a list of demands aimed at Netflix. Among the asks are for a hiring plan to recruit trans, nonbinary, and ethnically diverse leadership; a fund to support transgender creators; and explicit disclaimers on shows that feature transphobic or hateful content.
Fortune

What’s keeping managers up at night? New survey data from online leadership development company ExecOnline finds that leaders are reporting facing a new challenge to their own effectiveness going forward. The Great Resignation is taking a toll; managers say their top challenge is managing a workload with smaller, and increasingly burned-out teams. Some 52% report that employee fatigue and burn-out is their number one concern, followed closely by employee retention, and post-pandemic organizational culture. Bright spot alert: across the board, most self-reported perceived leadership capabilities took a dip this year, with one exception. Diversity leadership is the only capability that’s going up. Good news! But employees want more: Some “51 percent said their leaders will need to be able to demonstrate empathy to support workers’ needs; 44 percent want leaders to have skills that foster the concept of inclusivity; and 32 percent feel leaders will need skills that show an understanding of diversity and equity issues.”
ExecOnline

This edition of raceAhead was edited by Ashley Sylla.

On background

We’re poised to lose a generation of students if we don’t focus up Data shows that if you can read by third grade, you’re on a better path to high school graduation. It’s not a subtle benchmark: Black and Hispanic children are eight times as likely not to graduate if they don’t hit third grade reading proficiency. So, this story about the Achievement Prep public charter elementary school in Washington, DC should raise some alarm bells. In March 2020, 90 percent of the school’s first-graders hit their reading targets. But by the time they returned to school as second graders, every single child was off track. Some were back to kindergarten levels. What does the future look like for them?
Washington Post

Guess who invented hand sanitizer? Turn out, it was a Latina nursing student named Lupe Hernandez. The year was 1966, and Hernandez had come to believe that an alcohol solution delivered via a gel could help medical professionals stay sanitary when they lacked easy access to soap and water. She even registered a patent for the process by calling an inventions hotline. The story was reported by The Guardian in 2012, citing the ubiquity of the product even back then. “In the US alone, the growth of the market is astounding: valued at $28m (£17m) in 2002, it had swollen to $80m (£50m) by 2006, and is predicted to be worth some $402m (£250m) by 2015.” (Here are the global figures by comparison.) But, notes Marcos Hassan in Remezcla, we don’t know anything about Hernandez or her life. “Unsurprisingly, this is yet another case of a Latinx being written out of history, and it’s refreshing to see her get a bit of recognition, albeit late,” he says. 
The Guardian

Becoming more resilient in times of crisis You have to manage yourself, say three leadership and mindfulness author/experts, one of whom is a former Zen monk. Their first point is a hard one — don’t ruminate obsessively on bad things, even though you’ll want to. “When your mind gets stuck in this state, a chain reaction begins,” they write. “Fear begins to narrow your field of vision, and it becomes harder to see the bigger picture and the positive, creative possibilities in front of you.” The problem is that you’ll want to isolate yourself, and distancing aside, that’s the wrong impulse. Click through for advice to calm your mind and how to control “the second arrow.” Peace be with y’all.
HBR

Mood board

Owen Hoffmann—Getty Images for The Latinx House