Being wealthy has its downsides, chief among them the higher risk of an IRS audit.

Now, amid the continuing debate over whether the government is doing enough to combat financial fraud, comes news that gives the uber-wealthy or maybe even the merely wealthy yet another reason to fear Uncle Sam s attention.

In a speech before the American Institute of Certified Public Accountants, or AICPA, IRS Commissioner Douglas Shulman announced the recent formation of a new unit, dubbed Global High Wealth Industry Group, which will focus on the nation s wealthiest individuals and their entities.

The group s work will go way beyond screening tax returns and your garden-variety audit. We will take a unified look at the entire web of business entities controlled by a high-wealth individual, which will enable us to better assess the risk such arrangements pose to tax compliance and the integrity of our tax system, Shulman said in his speech.

That, says Lisa Feathernhill, a CPA and director of estate and financial planning at Wells Fargo s Family Wealth Group, is very broad and very deep. They re trying to understand how very wealthy people have their finances structured, she says. When I read that, it gave me pause.

The IRS has already begun hiring agents and specialists for this wealth squad, including flow-through specialists and international examiners, and plans to add more staff, including economists, appraisal experts and technical advisors in the near future.

How will these specialists decide who to put under the microscope? Several broad clues can be gleaned from Shulman s speech. SmartMoney.com spoke with wealth managers and CPAs who work with high-net-worth individuals for more specifics. Here are some criteria that are likely to be considered a red flag:

1. Income or assets over $10 million

Shulman pointed out that at least initially, [the IRS] will be looking at individuals with tens of millions of dollars of assets or income. Granted, the number of people who earn tens of millions a year may appear infinitely small: In 2007, the IRS received 18,394 tax returns reporting adjusted gross income (AGI) of $10 million or more, less than 0.05% of total tax returns received. It is often limited to celebrities and top-level executives. But certain events, like receiving a particularly juicy bonus or retirement package, could well throw you into that group. Retiring executives who might have distributions from several plans would have very large income tax returns for at least the first year or two after retirement, says Featherngill.

Throwing asset size into the mix expands the target group considerably. It could be anybody: real estate owners, closely-held business owners, [those with significant] investment assets, Featherngill says.

2. Complex financial arrangements

Here s what Shulman had to say of the types of assets that may attract attention: They may include trusts, real estate investments, royalty and licensing agreements, revenue-based or equity-sharing arrangements, private foundations, privately-held companies, and partnerships and other flow-through entities that require looking at the entire, and often huge, spectrum of transactions and entities. A single high-wealth individual may have actual or beneficial ownership of numerous related entities, sometimes alone and sometimes along with other family members or business associates.

Sounds like an all-inclusive resort, right? Harry Abrahamsen, principal of Abrahamsen Financial Group, a wealth management group in Holmdell, N.J., says individuals who have investments in hedge funds that are set up, or raise money, offshore are particularly at risk. This is probably a direct offshoot of the UBS [tax shelter probe] and people setting up offshore accounts, he says.

3. Offshore income or tax residency

Other tax considerations include international sourcing of income and tax residency, and offshore structures and bank accounts, Shulman said. You re more at risk if you have substantial wealth; dual citizenship and assets in another country; are a legal resident alien (i.e., you do not have U.S. citizenship, but are working or living in the U.S. temporarily); or simply have assets outside of the U.S., Abrahamsen says.

4. Movie and rock stars, athletes, expats

What do all these professions have in common? They are much more likely to hear from the new IRS unit, says Abrahamsen. The size of their paychecks aside, entertainers and athletes tend to bring in a lot of their income from performing overseas. So do expatriates including U.S. citizens on temporary assignments in foreign countries.

The IRS won t likely overlook successful business owners, either: The holistic approach of this new type of audit could mean that once you re on the IRS s radar, your company s balance sheet may wind up audited along with your own tax returns, says Featherngill.