Thirty years after its founding, Amazon has grown into one of the largest employers in the world and the second-largest corporation in the U.S. by staff size, trailing only Walmart. It’s also one of the most admired.
Along the way, the company has invested in programs for its massive, 1.5-million-employee workforce—spread across warehouses and offices around the world—to provide regular feedback on their work experience, including a daily survey tool known as Connections.
While the results from Connections surveys are meant to identify problem areas (or problem managers) and seek solutions—“Our goal is to help develop leaders who earn trust, remove barriers to excellence, and make Amazon an inspiring place to work”—Amazon has also utilized Connections data when combating public reports of dissatisfied employees.
And when it does, employees have for years been known to reach out to this reporter with a suggestion: Be skeptical of the data.
Most recently, Amazon cited its employee survey results in response to a Fortune article documenting the challenges and fears facing Amazon’s customer service employees. Amazon founder Jeff Bezos also cited Connections data from warehouse staff when he addressed unionization efforts in his final annual shareholder letter as CEO.
To be clear, many Amazon employees note that Connections appears well-intentioned, and some who’ve worked on the program have lauded it as one of the first large-scale experiments involving a company creating a daily employee survey.
Still, many Amazon employees question the accuracy of the feedback—and thus the data Amazon cites—because they say many workers don’t answer Connections questions honestly.
The reasons for this are numerous, employees have said. Chief among employee concerns is that the anonymity of their responses is not as bulletproof as the company would like to suggest. On small teams for example, employees are concerned that their manager can deduce who responded in what way based on the overall manager-employee relationship. Amazon spokesperson Margaret Callahan said employee responses are confidential and that managers see aggregated results only when a team has at least four employees. For smaller teams, survey responses are not visible to managers nor other leaders. But even team sizes of four or more can leave little cover for who answered how, employees argue.
And some managers go so far as to put a thumb on the scale, employees say, by coaching their direct reports on how to answer certain questions or stressing to them that negative feedback can negatively impact the manager’s standing.
“Rather than self-reflection on why her scores are low,” one Amazon employee told Fortune of a manager, “she regularly bullies us to give better scores.”
Amazon’s Callahan said it’s against company policy for managers to attempt to influence employee responses, and that employees can always choose not to answer Connections questions if they prefer.
“We’re constantly working to improve the experience of our employees, and that’s why we collect feedback daily through Connections,” she noted in a statement to Fortune. “Employees can choose whether to answer or not, their responses are always confidential and aggregated, and the feedback that leaders receive help them build stronger organizations. We hear from the vast majority of our team that it’s a useful tool, and the anecdotes that were shared with us for this story don’t reflect most people’s experiences.”
Callahan added that the daily frequency of the surveying allows managers to filter results for various time frames and view trends for several questions that the tool asks regularly over a period of time. In its ideal implementation, the data allows managers to alleviate problems more quickly than if the company only polled workers monthly or annually.
On the other side of the equation, some employees have said they don’t give candid feedback because they fear a manager whom they like or respect will be penalized for employee concerns that are outside of the manager’s control.
Either way, the next time Amazon cites survey data to paint a picture of its work environment, it’s worth at least noting what many employees believe: that the picture is far from complete.