Decontamination is set to resume at the Deutsche Bank building within days and the construction of two out of three World Trade Center skyscrapers began last week. “There has not been this much going on at the site since the cleanup concluded almost six years ago,” said developer Larry Silverstein in a speech.

The rebirth of the two sites—those that once housed the World Trade Center and the Deutsche Bank building – represent just the latest reinvention of one of Manhattan’s longest-settled areas. It’s also a reminder that, in New York, rebirth and renovation are recurring processes. Under the Deutsche Bank building, for example, resides a lot of not-quite-buried history.

Before 9/11, the World Trade Center was condemned by some for destroying a thriving neighborhood of small electronics shops known as Radio Row. What became 1 Bankers Trust Plaza (and later, the Deutsche Bank building after the German corporation acquired Bankers Trust) contributed further destruction: Two blocks largely filled with venerable four- and five-story walk-ups, most of them with mom-and-pop businesses on the ground floor, were demolished to build it. And while Radio Row extended to the site’s northern tip, the rest of the plot was situated in a neighborhood of a different character.

The New York Times surveyed the environs in a 1971 article describing the construction of the World Trade Center and the Bankers Trust Tower. The article noted, for example, a bar on Cedar Street (a block of which would disappear under the bank’s structure) that was serving “45-cent gin specials to take advantage of the big daytime trade from construction workers.” The adjacent street was filled with “saloons with quick-lunch steam tables, a variety of shops, an ale house, [and] the back of the American Stock Exchange.” (The latter still stands.)

One local restaurant owner waxed nostalgic about the “great hole being gouged out” for Bankers Trust because he had been born on the very spot: “I remember it was a coldwater walk-up flat with wood stove and everybody had separate private toilets outside in the yard,” he told the Times. “And the kids in the summer would go down to South Ferry and swim in the buff.”

In the earliest days of Manhattan, swimming would have been easy at the site of the Bankers Trust tower: At least part of the lot was located in the Hudson River. In New York’s earliest years, the city sold parcels of land that were under water at high tide but more or less dry at low tide, according to Ann Buttenweiser’s Manhattan Water-Bound: Manhattan’s Waterfront From the Seventeenth Century to the Present. Despite the plentiful space on the island, it was common to buy such plots and then bolster them with landfill. And so the earliest existing property record for the block that would one day hold the Bankers Trust building is a “indenture” from 1704 which describes the spot as located “without the north gate of the City of New Yorke bounded… in the West by the Hudson’s River at low water mark…”

By the 1820s, the future bank site had been fully converted to dry land and subdivided. But it wasn’t a high-rent district. For decades, owing to its proximity to the end of the island, it was a place where new arrivals and immigrants put down their bags before moving to more permanent lodging. Edgar Alan Poe, for example, briefly rented a room in a boarding house on the site in 1844 before moving uptown.

The arrival of the first elevated train in Manhattan, which ran north along the plot, didn’t improve the atmosphere. From 1870 until it was taken down in 1940, what became the Ninth Avenue El made the area noisy, dark and forbidding. Little surprise, perhaps, that illegal operations seemed to flourish in the area.

In the 1930s and 40s, references to the future Bankers Trust location turned up regularly in the Times, which reported raids on speakeasies, a fraud artist who “out-Ponzied Ponzi,” and an illegal horse-betting operation that went by the name of the “Paragon Institute of Raceology.” One of the more colorful examples appeared on October 2, 1942: “Nine men were arrested … in what the police called a ‘million-dollar-a-year wire service for poolroom bookmakers and other gamblers on horse racing…” The ringleader was Thomas J. Ryan, “a former member of the notorious Purple Gang of prohibition bootleggers and other criminals from Detroit and former muscle man for the late Moses L. Annenberg…” According to the article, Ryan had been warned of the impending raid, but said he wasn’t worried about it. “Big Shot Ryan made remarks over the phone that he wasn’t afraid of anyone,” a deputy police chief exulted, “but when we went there this morning he was hiding in a closet with a secret telephone, with a valise all packed ready to leave the city, and $16,000 in cash.”

It wasn’t all gangsters. A couple of blocks south, for decades, was what the 1939 WPA Guide to New York City referred to as the Syrian quarter. As the Guide put it, “although the fez has given way to the snap-brim, and the narghile has been abandoned for cigarettes, the coffee houses and the tobacco and confectionery shops of the Levantines still remain.” It was a vibrant neighborhood where baklava and shish kebab, then considered exotic foods, could be found – a small cultural foothold just a few blocks from Wall Street, the citadel of American capitalism.

See also: Features on 9/11—Here’s How Much Lower Manhattan Has Changed Since the 9/11 Attacks (2016)

But the World Trade Center would cut off Greenwich Street, the area’s main artery. And the hulking darkness of the Bankers Trust building would help break the area up, leading to the demise of the neighborhood, which has never quite recovered.