FORTUNE — To the protester, they represent the problem. But you don’t have to search long among Wall Street’s rank and file to find something unexpected: sympathy for the occupiers of Zuccotti Park.

“There are a lot of folks on Wall Street who are just like the 99 percenters,” says a 44-year-old principal at a small investment firm on Wall Street, walking along Nassau Street, about a block from the protests. “Goldman Sachs for example pays out a certain amount in bonuses, and they say that employees get an average of $500,000. That’s not even close to the truth. Most don’t get that. Many on Wall Street are just like them and would like to join them but are afraid.”

In Wall Street terms, 2011 is certainly shaping up to be a bad year. Banks from UBS (UBS) to Goldman Sachs (GS) are planning thousands of layoffs. Bank of America alone is promising to shed 30,000 jobs over the next few years. Yet it’s hard to imagine the blue suits carrying signs along Broadway or sporting New York’s latest fashion must have: a spray-painted t-shirt proclaiming ‘I am a 99 Percenter.’ Sure, bonuses are smaller this year — but they’re still coming.

Still, the protesters have made their mark, and not just in Zuccotti Park. Along the quieter streets east of Broadway, as Wall Street-types make their way along barricaded sidewalks for a sandwich, coffee or smoke, they’re paying attention to the demonstrations. Some may be annoyed at the disruption — closed subway stops, and an overcrowded McDonalds. But others wonder, do the protesters have a point?

“The big banks got the bailouts but it’s much harder for the smaller firms to survive,” says Troy Holland, 39, an officer with HIC Financial, a small investment banking firm on Wall Street, who was heading over to the protest at Zuccotti Park on a recent morning. Donning a pinstriped suit, Holland could hardly be mistaken for a protester. And yet he has concerns about his firm’s ability to compete today. “As entrepreneurs with our own firm we are the 99% just like Occupy Wall Street. [They] got it right.”

What you (still) get with your college degree: A better shot at the 1%

Certainly Wall Street is studded with bankers firmly within the 1%. The average salary for those who work in New York City’s securities industry is, after all, $361,330 — 5.5 times higher than the average in other private sectors, according to the New York State Comptroller. But many on Wall Street are themselves assistants, clerical staff and mailroom clerks — the workers who keep Wall Street humming who are unlikely breaking six figures a year. (I should know. I was once one of them.)

Some of them have mixed feelings about the movement. “I have a little bit of empathy and anger for the protesters,” said an executive assistant for an insurance firm near Wall Street. “I don’t know what they’re expecting. What’s going to change?”

But of course Wall Street itself has changed. Citigroup and Morgan Stanley are headquartered in midtown — far from the dark downtown tunnels of the financial district. And families populate the neighborhood now as much as bankers, with the financial district a popular and expensive residential area. Toddlers from a child care center on Chase Manhattan Plaza outnumbered employees one recent morning, dancing under the Dubuffet sculpture, unaware of the chaos just blocks away.

“I work on Wall Street and Wall Street is just a place,” says the 44-year-old investment bank principal who didn’t want his name printed. “It just signifies what everyone is fighting against.”

Where will Occupy Wall Street take us?

Yet CEOs have realized they need to start sounding the empathy bell. Take Vikram Pandit, Citigroup’s CEO who noted that the complaints of those protesting are “…completely understandable,” he told Fortune magazine’s Andy Serwer last week. “Trust has been broken between financial institutions and the citizens of the U.S., and that is Wall Street’s job, to reach out to Main Street and rebuild that trust.”

But are they really trying? Bank of America (BAC) and Wells Fargo (WFC) both announced new debit card charges, and Citigroup is raising some checking account fees aimed squarely at Main Street.

Solidarity for the protesters can be found among the suits in the financial district, but not easily. For every Wall Street employee thoughtful about what the demonstrators hopes to accomplish, several others walking along Nassau, Pine, William and Liberty Street complained about the inconvenience the protesters have created, as the blocked plazas and streets create an even more maze-like warren in lower Manhattan.

“They have the right to assemble,” said one 29-year-old banking executive in the mortgage industry, chatting with a friend just outside Brown Brothers Harriman & Co.’s building at 140 Broadway. “But this is starting to get bothersome.”