What “Brexit” means for London’s financial center

January 31, 2013, 3:00 PM UTC

FORTUNE — The City of London could be in for a rude awakening if the United Kingdom votes to leave the European Union. While bankers and politicians have spent the week downplaying any possible negative externalities associated with “Brexit,” the name given to the UK’s potential withdrawal from the EU, there still exist a number of potentially damaging consequences, which could indeed threaten the City’s position as Europe’s financial hub. Work must be undertaken now by the British government, as well as by firms in the City, to prepare for a possible Brexit. If not, Wall Street, as well as other budding financial centers on the continent and in Asia, could successfully steal some of London’s financial glamour.

Last week, UK Prime Minister David Cameron gave his speech on Britain’s future in Europe. Bending to conservative backbenchers in his party, Cameron announced that the British people would sometime in the future vote on whether or not to remain in the EU. The rules and restrictions of the common economic zone were getting too much for many in his party to bear. Cameron said that he would personally campaign for the UK to remain in the EU at the time of the referendum if the EU were to undertake a number of reforms – much of which involve the UK receiving a pass on a number political changes that would see more national sovereignty transferred to Brussels and Frankfurt.

But throughout his speech and in the comments that followed, Cameron and his administration avoided speaking directly about the consequences that Brexit could have on the nation’s golden goose — the City of London, its massive financial industry. The UK government wasn’t alone here; indeed, one would be hard pressed to find any new studies commenting in any real depth on Brexit’s potential impact on the City of London. Bankers and fund managers who work in the City didn’t give Brexit much thought, either, noting that London has always been a pivotal place for international finance and that any referendum, if it were to happen at all, was to take place in four-year’s time, far too long of a time horizon to speculate on Brexit’s potential impacts.

MORE: Behind the UK’s hemming and hawing over the EU

But it is hard to believe that the City won’t be either directly or at least indirectly impacted by a UK withdrawal from the EU. After all, the financial services industry is massive, accounting for 10% of the nation’s GDP and 11% of its tax receipts. While it might be easy to look the other way and focus on today’s problems, the fact is that Brexit is a real possibility.

Indeed, most Britons aren’t fans of the EU. A Yougov poll of British subjects conducted at the end of 2011 found that in a straight in-or-out referendum, 50% of people surveyed would vote to leave the EU with just 25% saying they would stay in. So this vote shouldn’t be taken for granted. Unless there is a major shift in sentiment then the UK will be indeed withdrawing from the EU in 2017.

If that’s the case then work needs to be done now to figure out how the UK could quickly and easily extricate itself from the EU’s tangled web while retaining as much of the City’s European business as possible. Thousands of laws and dozens of treaties would need to be reworked or renegotiated, a process that could potentially take years and cost tens of billions of euros if not done right.

The potential impacts on the City range from a lack of travel options to a mass exodus of prime City employees and their firms. Indeed, around 1.4 million or so Britons living in the EU may be forced to return home, while 2.2 million EU citizens currently working in the UK, thousands of which work in the City, may get the boot back over the English Channel.

MORE: A 5-point plan for a new globalism

This tit-for-tat exchange of workers would be highly disruptive for both the UK and EU economies, and particularly disruptive to the City, which employs a large number of foreign workers, many of which come from various parts of the EU. The government could work to halt the outflow of these high-earners by immediately granting City workers from the EU long-term working permits, removing any concerns that the City is closed to foreigners. It is unclear if such a scenario could work politically, though, as it could be difficult to avoid public outrage over the government saving bankers while they throw out teachers and medical professionals.

The City will also need to know soon what rules and regulations it would need to adhere to if the UK withdraws. Could UK-based asset managers continue to market their funds to European investors? Approximately three-quarters of the 6.8 trillion euros invested in collective investment schemes throughout the EU are done through so-called Ucits funds. These are funds that meet strict EU guidelines on liquidity and reporting mechanisms, allowing them virtually unrestricted access to the EU market. The EU could require UK firms soliciting European business to up its Ucits standards or it could even ban them from marketing to EU clients all together. Securing this agreement with the EU would be critical for the City.

Euro-skeptics believe that the City would be much better off outside the EU’s web of financial regulations, but that may be overstating the issue a bit, especially given the questions surrounding Ucits funds. Indeed, when the British hate a regulation they often simply don’t follow it. For example, Cameron vetoed legislation in December of 2011 that would have brought eurozone nations closer together in a fiscal compact because the City was against the introduction of a financial transaction tax or a unified banking system. Eurozone members that wanted the tax later banded together to adopt the measure without the UK’s say. They have also moved forward with creating a banking union and harmonizing a number of rules dealing with bank liquidity and the like.

Meanwhile the UK has created its own beefed-up rules for its financial center, introducing new regulations, like the Vickers proposals, forcing UK-based banks to ring-fence their retail operations from their investment banking operations as well as to hold more capital than required under the Basel III agreements. These rules would probably remain in force after Brexit. The UK may want to relax these rules to better compete with banks on the continent or it could risk losing some business to financial centers in Frankfurt or Paris.

MORE: Goldman Sachs braces for bond market blow up

There are a number of potential pitfalls that UK politicians must be ready for if the British people choose to exit the EU completely. The future of the City of London should be their top priority given how important it is to the nation’s economy. While some industries, like the global insurance and reinsurance businesses, will most likely remain in London even with a UK withdrawal from the EU, other investment banking and trading operations could be at threat of jumping the channel to the continent or jumping over the ocean to Wall Street.

The City has already lost a third of its workforce since 2007 and it is slated to lose even more this year, due at least in part to uncertainty regarding the City’s place on the financial world map. The City won’t see all of its banks and funds uproot overnight if the UK withdraws from Europe but it will certainly be affected one way or another. Dismissing that possibility, as remote as it seems, would be negligent on the part of both the politicians of Westminster as well as the money managers in the City.