Back to Index

2024-05-09_How_to_Not_Get_Nailed_by_a_State_Income_Tax_Auditor


Transcript

Say goodbye to your credit card rewards. Greedy corporate megastores, led by Walmart and Target, are pushing for law in Congress to take away your hard-earned cash back and travel points to line their pockets. The Durbin-Marshall Credit Card Bill would enact harmful credit card routing mandates that would end credit card rewards as we know it.

If you love your credit card rewards, tell your lawmakers, "Hands off my rewards." Tell them to oppose the Durbin-Marshall Credit Card Bill. Welcome to Radical Personal Finance, headline of today's articles from Bloomberg. New York's rich get creative to flee state taxes. Auditors are on to them. At New Jersey's Teterboro and Long Island's Islip airports, dozens of private jets destined for Florida take off at times such as 11.42 p.m.

or 11.54 p.m. Over at JFK, a regular flight from San Juan, Puerto Rico, arrives at a seemingly purposeful time, about 15 minutes after midnight. Meanwhile, tax attorneys tell stories of clients idling in their luxury SUVs near the New Jersey entrance to the George Washington Bridge shortly before 12 a.m., waiting for the clock to turn before crossing the state line to New York.

When it comes to taxes and the wealthy, every minute matters, especially for those who have left New York and declared residency elsewhere. At a time more high earners are departing, or at least are claiming to, state officials are stepping up already intense scrutiny to make sure former residents have actually moved.

It's a complex operation that involves cutting-edge artificial intelligence and tracking everything from travel to the location of people's pets. We'll continue in a moment, but the basic outline of this article is something that you need to be aware of if you are wealthy and you are trying to manage your tax costs properly.

One of the most important concepts in tax planning is to pay the taxes that you owe, but not to inadvertently create a tax obligation through short-sighted behavior. And one such behavior is spending a significant amount of time in a high-tax jurisdiction in an unplanned manner. Now, we'll talk here in just a moment more about the timing, but one of the most important features that causes a state taxing agency to think that they can tax you is whether or not you spend significant amounts of time in the territory of that government taxing agency.

Historically, the number has been something like 183 days. The idea was that if you spent more than 183 days inside of a certain jurisdiction, then you would automatically be considered a tax resident of that jurisdiction, and the government there would believe that they were entitled to take a share of your income.

Now, internationally and globally, that has become much more complicated. It's somewhat uncommon in today's world to find countries that have just a simple 183-day test, although they do still exist. But this does exist in many places in the United States, and so for Americans, being aware of where you spend time is a very important component in order to minimize your tax liability, especially if you are moving from a high-tax jurisdiction to a lower-tax jurisdiction.

Let's continue with the article. For the ultra-rich, even just an extra day in the wrong place could mean millions in income tax liability. Quote, "The minute you file a partial return, you're going to hear from New York State," said Jonathan Mariner, who created TaxDay, an app that tracks users' locations so they don't overstay the threshold of days that would trigger residency status, which is typically 184.

The whereabouts of the wealthy are particularly important to New York, a high-tax area where the loss of even a handful of rich residents can have an outsized effect on budget revenue. The state has been especially hard hit by departures since COVID-19 fueled the rise of remote work. The number of taxpayers earning more than $1 million who moved out more than doubled in 2020 from 2019 and has continued each year to be well above pre-pandemic levels, according to the Department of Taxation and Finance.

That's leading state officials to try to claw back money wherever possible through residency audits, an investigation into whether someone correctly identified themselves as a full-time, part-time or non-resident for income tax purposes. It's also taking place in California, another relatively high-tax, high-departure state facing budget strains, where the number of residency audits closed in the first 11 months of 2023 more than doubled from before the pandemic.

There are two states in the United States that are notoriously aggressive with regard to income tax collection, and these states are New York State and California. Now, there are other states that also have notoriously high income tax rates and that are also aggressive, but these two states are especially aggressive.

And so if you are a resident of one of these states, or if you have been a resident of one of these states, you need to pay careful attention to make certain that when you leave, you leave properly. And that's what I'm going to teach you about in today's podcast.

Continuing with the article. For some of these people, often accustomed to the privacy and ease afforded by wealth, a residency audit can be an uncomfortable dive into their daily life. New York's auditors closely watch travel and apply a standard known as the "teddy bear test," looking to see where individuals keep their most cherished possessions to determine whether a home is their primary residence.

"We always tell people the tax audit from New York is like the tax version of a colonoscopy," said Mark Klein, a tax attorney at Hodgson-Russ. "I've had cases that have hinged on a single dog," Klein said, "and I had a case once that was based on the fact that the person moved their Peloton bicycle to Florida." There was a famous test a number of years ago with a New Yorker who moved to Texas.

And the taxpayer in question alleged that, "Hey, listen, I fully live in the state of Texas. I have all this stuff set up there. I just happen to spend a significant amount of time in New York." And that's what he's alluding to as far as where was the dog.

That was ultimately the final clinching factor that came in to decide whether or not he had actually moved out of the state of New York or not. And in a little bit, I'm going to teach you how to go ahead and make certain that you don't wind up in a situation where it comes down to a specific dog's location, though, because that's not safe for you.

Let's finish off the article, though. Counting days. Residency audits have long been big business in New York. The state collected roughly $1 billion from 15,000 audits between 2013 and 2017, according to data obtained through a Freedom of Information request. While California lags behind New York in the scale and complexity of its residency audit operation, the Golden State collected $85 million in residency audit income last year through November, the largest single-year tally in at least a decade.

Both states are still recording overall growth in the number of millionaires thanks to income and stock market gains, but the key for tax officials is making sure they've genuinely left. Since the pandemic, higher-income people moved out at a faster pace than they moved in. Data from both states show.

While wealthy residents have long pretended to leave to dodge taxes, tax experts say COVID brought more legitimate moves as companies embraced flexible work and set up outposts in places such as Florida and Texas, which have no state individual income tax. New York's tax laws are notoriously tricky. Typically, someone who lives there will be considered a resident for tax purposes, paying levies on their income from all sources, even those outside of the state.

But the state considers someone a resident even if they don't live there, as long as they've spent more than 183 days in New York and maintain a permanent place of abode, which could simply be a vacation home. A few hours spent in New York qualifies as a whole day.

Getting off the highway in New York for lunch while driving from New Jersey to Connecticut can count. So can getting outpatient treatment at a New York hospital. When you can't prove where you were on a given day, New York auditors may assume you were in the state. Quote, even though you have a Florida driver's license, Florida voting record, Florida home, it does not matter, said Mariner, who created his app after facing his own residency audit after moving to the Sunshine State.

You could be on vacation in New York and they'll pull you back in. New York's Department of Taxation and Finance has 300 auditors dedicated to conducting residency audits and they are notorious for their thoroughness. Bank records, phone bills, and family photos are under the microscope. Auditors are backed up by sophisticated artificial intelligence-fueled tax monitoring systems that flag inconsistencies in return.

The point of the article, you can read it if you like, is that auditors are looking for money. Many, many states in the United States, especially high-tax states, are finding that in our current era of significant geographic flexibility, it's harder for them to keep people in a high-tax jurisdiction.

Now, that may not be the case across all income classes. So it's not that common that you might have somebody earning $80,000 a year who says, "You know what? I'm just going to go ahead and move to a different state." But for the very rich, it is more common.

Given the changes that have happened in the last couple of decades with connectivity, meaning the ability to reach out and talk to anybody, anywhere, at any time, anywhere in the globe, given the improvements in transportation infrastructure, the fact that flying around the United States and really around the world these days is pretty fast, pretty simple, pretty inexpensive, it's just relatively easy for people to do business in more places.

It's not uncommon if you're living in Florida, you can fly up to New York, spend half the day there, have meetings, have lunch, whatever you need to do, and you can be back home in your own bed at dinnertime. The flights going back and forth between South Florida and the Northeast are legion.

There are many, many, many of them available. And so people are taking advantage of this and availing themselves of it. And one component of that is lowering their income tax rates. Now, the places, the times in which this really affects you is, especially if you have some kind of property that you would like to sell, something like the sale of a business, something like the sale of stock, because whereas it can be difficult to move to another state just to shelter income taxes, if your income tax, if your business operations are in another state, in the high-tax state, what can be more challenging, excuse me, what can be simpler is when you are moving prior to the sale of a business, prior to the sale of a stock, things like that.

And this is very, very common. Even just down to a simple 401(k), you may work in a high-tax state, a high-income state. You may contribute to your 401(k). Your 401(k) contributions can bypass the state income taxes because they're pre-tax contributions. Then in retirement, you may move to a no-income tax state.

And in that no-income tax state, you have the ability to now put yourself in a situation where you don't ever pay income tax on that money. And that can be helpful. It can be, in many cases, several percent of your income, which can be substantial over time. So let me give you some suggestions and tips as to how to arrange your affairs so that you don't face a significant risk of audit.

First and foremost, I don't think it makes sense to do any move in your life exclusively for or even primarily for tax planning. I really don't. I don't think that you should move from New York to Florida exclusively or primarily for tax planning. I think the people who do anything primarily for tax planning are being short-sighted in their long-term outlook and in their long-term plans.

After all, taxes, what we're talking about here with income taxes, taxes exclusively affect those people who are earning high income. The fact that you're earning high income means that you have more choices. You have more freedom. The entire reason you earn high income or the entire reason you build wealth is so that you have choice, so that you have freedom.

And it doesn't make any long-term sense for you to now abandon your freedom and allow an external government taxing authority to be the one who primarily or exclusively controls your decisions. What's the point of getting rich if you can't live where you want to live and live how you want to live?

What's the point of being wealthy and earning high income if you're looking over your shoulder trying to dodge the taxman all the time? It's a silly way to live. It doesn't lead to freedom and happiness. It's a very high-risk game and it's just flat-out not worth it. Pay your taxes.

It's an extortion fee that the government requires in order for you to stay out of jail. So if you want to stay out of jail, you have two choices. Either option A, pay your taxes or genuinely leave and don't play any games with the taxing authorities. I've already now showed my hand on the second point.

First point is don't make taxes your primary or exclusive choices. They are perfectly reasonable as a secondary criteria. So if you really want to move where it's warmer and sunny and where your friends live in Florida or Texas and you want to move out of New York, then great, go for it.

And if you want to enjoy the fact that you have lower tax bills as one piece of the puzzle, that's a wonderful extra reason to help. And it may be the thing that pushes you over, but it shouldn't be a primary reason or there's a very good chance you'll be back in California within a few years.

You'll be back in New York in a few years. You'll probably be priced out of the market. You can't buy back in. You'll have lost relationships. Just stay in the place that you want to be and keep taxes as secondary consideration. Now, if you do move or if you do want to change your tax situation, the secret to all of this is genuinely change your tax situation.

Don't play games with the taxing authority. If you're going to move from New York to another state, to Florida, then move from New York to Florida. And make sure that when you're moving from New York to Florida, your lifestyle is going to allow you to genuinely live in New York.

Excuse me, I apologize. My brain is at half capacity this morning. It's going to genuinely allow you to live in Florida. So if you think you have to be in New York possibly 180 days this year, that's probably a good sign that you should stay in New York as a tax resident.

But if you can genuinely move to Florida, just genuinely move to Florida. If you read the law and consult with relevant authorities to make certain that you understand how the state that you're dealing with assesses tax residency, and if you can satisfy those conditions, then just go and satisfy the conditions fully.

Don't play games with them. Genuinely move. That will solve basically everything that you need solved. The most important thing is making sure that you're not a tax resident of a place that you don't want to be. Even in an international context, I teach this, especially for Americans. So Americans traveling abroad, Americans are always tax residents in the United States because we have citizenship-based taxation.

So there's not really any question about that. It's one of the benefits of being an American is we have citizenship-based taxation. So either you're a citizen or you're not. And if you're a citizen, you pay tax no matter where in the world you are, according to the relevant exclusions and things like that.

But if you're not, you're not. Now, the key is, however, don't become a tax resident of a place that you don't want to become tax resident. So when I covered the pop singer's issues with the state of Spain when she's just hanging out in Spain and a massive multi-million dollar settlement, the problem is that she allowed herself to become a tax resident of a place he shouldn't have been a tax resident of.

And this is your choice. You don't have to do this. You can control your activities. So make sure that you just control your activities and make sure that you're genuinely not a tax resident of the place that you previously were. Here's where the biggest danger is. There's very little danger of someone like me who's never lived in New York, spent very little time there, of me going there and spending a few months there and then having any kind of trouble.

After all, my footprint in New York is relatively light. And while the state tax auditors can access mobile phone records and bank records and things like that, I would have to do something that would bring me to their attention. But if I had lived in New York for a good number of years and then I was strategically moving out of New York with the goal of changing my tax residency so that two years from now I can sell my business or sell some assets and avoid the issues, now it's a much bigger deal because now I'm on their radar screen and they have a reason to want to keep me there.

And I find this frequently with consulting clients that I work with, California and New York. If you are leaving one of those states and moving elsewhere, you need to genuinely leave and every fact matters. So here is what you want to pay attention to. If you are going to go up against an auditor, the first thing is you want to make sure that all of the facts are on your side.

So if they say don't spend more than 183 days in the state of New York, you better make sure first and foremost that you are not in New York for more than 183 days and be smarter for you if it were significantly less. But the person who's going to win in court is going to come down to who can put the most paperwork down proving that I'm not a tax resident.

And you're going to be going up against the other side that's going to be trying to put paperwork down demonstrating that you're a tax resident. So what you want to do when you leave a place, especially if a place like California or New York, is that you want to genuinely demonstrate full and complete intent to move elsewhere and you want to create a long string of paperwork that you could use to demonstrate your involvement in a new place.

What does that mean? Well, the first thing is obviously you want to make certain that you have a home. So you want to go from California or from New York to a home in another place. Where I often hear this is people who go RVing, somebody who's doing well, wealthy, living in California, says, "Okay, I want to move into an RV and I want to go and be a nomad." All right, well, that's a fairly straightforward thing to do if you're from Florida or Texas.

But if you're moving from a state, by the way, this is inclusive of any other state, any state that has a state income tax, you probably should consider putting in place an in-between step. You should consider genuinely moving to a new place outside of your previous high-tax state. Check the laws of each state.

I'm using California and New York because they're the most aggressive. Each state has different laws that they use to determine residency and you should check the laws of your state. Some states are clear, some states are less clear. But the first most essential factor that determines your domicile is occupancy of a home and ideally ownership of a home.

So where you own or rent a home and as well as the relative size and nature and use of the home is largely going to determine your domicile. If you have a big, beautiful home on the coast of California that you've lived in for many years, but now you go and you rent a tiny little studio apartment in Las Vegas, that is going to be the primary factor that is going to mess you up when it comes to actually being in tax court.

You can't just convince the court that, "Well, I rent this studio in Las Vegas and that's actually where I live while I still own this big, beautiful home in California." So if you're going to keep that big, beautiful home in California, you probably need a bigger and more beautiful home in Nevada or Texas or wherever you're going in order to make certain that it's believable the home in California is your vacation home and the new home is your primary home.

So you need to think very carefully about your housing situation. The simplest thing to do is to amputate all of your connections to the state in question. And in some countries, people who want to become tax residents, become non-tax residents, let's say you're Canadian and you want to become non-tax resident in Canada, then you probably want to sell all of your property because you're dealing in that situation with a very aggressive taxing agency and so you want to go through the steps to position yourself as genuinely ending your connections with the state.

And similar for people from California or from New York. Now, if you're from Michigan or Ohio, some state that has state income taxes, but they're not as aggressive, you may have a little bit more leeway. But the biggest, most important function is going to be the home that you own and the home that you occupy.

And you want to make certain that your big, beautiful primary residence is in the other state. Now, there are other things that have to do. For example, your employment records. If you're serving as an employee of a business that's in California, but now, of course, you move to a small studio in Nevada, that's not going to work.

And so if you're an employee or an officer of a business that's domiciled in the high-tax state, you need to change that relationship. You need to become an independent contractor and you need to move your independent contracting entity, if existent, to another state. Another huge factor is children. Where do your children go to school?

So if somebody's investigating your domicile and they find out that, "Hey, I've got this modest house in Nevada," but in reality, these children are attending a school in Los Angeles, well, that's going to be a key factor to show that, no, you actually still live in California because your center of life is in California.

After all, this is where the children go to school. So you want to make certain that if you're trying to move outside of one of these states, you want to enroll your children in school in the new state or you want to homeschool them or make sure that they're staying with other friends and family until you fully have changed the situation.

Otherwise, wait till after they graduate. Now, if your children are older, their college age are older, then, of course, they're more independent and it's less of a big deal. But K-12 minors, you want to make certain that you arrange their schooling so that it doesn't indicate your tax residency in the high-tax state.

Next is going to be just your days spent in the state. So California is a ratio. I believe California uses a ratio of how many days in California versus other places. New York may have a strict days test, but this is why check your specific state. And what you're trying to come up with is what is the clear line indication.

So if memory is right, I should have checked this. Forgive me for not checking this with California. But the key is that you don't have more connections with the state that's high-tax than you do with another place and don't spend more time there. This is also very common in international planning, that if you spend more days in the high-tax jurisdiction, let's say that you spend lots of time in Spain, and even if you're not passing the 183 days rule, you're just spending, say, 150 days in Spain.

Then you're spending 30 days, 30 days, 30 days, 30 days, 30 days, 30 days in five other countries. Then now there's a good argument that the government can say, all right, well, we appreciate the fact that you were only in Spain for 150 days a year, but the fact that this is your strongest point of connection indicates to us that Spain is actually your tax home, and that can be a factor that is considered depending on the country.

So pay carefully attention to that. You also want to think about where you receive income from, because in many states, income that is generated in the state is taxable regardless of the actual location of the taxpayer. So think carefully and research the laws of your state as to where income is generated.

Those are the most important ones, and you want to genuinely establish yourself as non-resident in the high-tax state. Then after those important ones, because we recognize that there can be things that can change. So here would be an example. Let's say that you genuinely left New York, and you genuinely moved to Florida, and you're in Florida for two years, but then all of a sudden something's going on with your business, and you have a period of time in which you need to spend a significant amount of time in New York.

You can genuinely have moved to Florida, but now you find yourself genuinely spending time in New York, and you're watching your days carefully, but something happens. Again, hospital stay or emergency in a business or opportunities, and now boom, you're sucked back into the tax net all of a sudden because of a genuine reason.

But these other factors can play in if you have them properly aligned. So you want to create a stack of paperwork that you can take into an audit meeting or into a court case that's going to demonstrate that I'm genuinely truly resident in another state, so that if you're ever unanticipated in a questionable scenario, then the paperwork is on your side.

So first thing, driver's license. Anybody in your family who's a driver should go and get a driver's license in another state. Not just one person, not just dad gets a license in Florida. Get everybody to get a license in Florida. Voter registration. Registration is an important component. So if you're going to register to vote, register to move your voter registration.

Move your vehicle registrations to the new state. Make certain that everything is properly tagged and titled and your property is there. Update the billing addresses on all of your banking accounts, all of your investment accounts. Make sure that everything is in the new state. Go through all of your banking and financial accounts.

Make certain that you've changed your residency address with all of those bank accounts. You need to find new family doctors, dentists, consultants, people who you're working with. And go and change your primary care provider with your health insurance. Go and find a new doctor. Go for a checkup. Visit the dentist in the new place.

Make sure you keep those records showing. I don't go to the same doctor that I've gone to for 30 years in New York. I've gone to this new doctor in Florida. Move your professional memberships. Remove your name from the bar association or from the certified financial planners list or anything like that that's in the state.

Move it all to the new place. Any kind of community involvement or volunteer organizations. Move your church membership to a church in a new place. Move your charity records and donations and all of that to the new place. Attend social events in your new location and make certain that those social events are documented.

Documented here I am at all these social events in Florida and that you're not in California. Move your country club membership. Move safety deposit boxes, P.O. boxes. Document the fact that your family members are visiting you in Florida, not in New York. All of your professional work. Move your will.

Make certain that your will is that you get a new will drafted up in the state of Florida and that's located in the new place. Move obviously your utilities that are in your name and make certain that you're visible in the new place. In some audits, the auditor or investigator is going around and literally interviewing your friends and your neighbors and your business associates to figure out how much time you're spending in the new place and so you should be very straightforward of always saying that, you know, I live in Texas or I live in Nevada or I live in Florida and I vacation in California when I want to visit my friends.

Things like that. So make certain that all of your language is properly and fully documented. So you want to genuinely change your affairs. Then you create a big stack of paperwork showing that you've genuinely changed your affairs. And then the final key to it is make certain that you're not generating any kind of paperwork or evidence that would be showing that well, in reality, he's spending lots of time in California or New York because it all comes down to who has the most paperwork.

So you can be super hardcore with this. You know, whenever you're in Florida, that's where you spend money with your credit cards and with all of your digital transactions. When you're in New York, that's when you use physical currency to pay for stuff and you do everything like that.

If you're going to have meetings when you're in Florida, that's where you do all your in-person meetings. When you're in New York, that's where you use Zoom. When you are thinking about your travel records, one of the things that gets people a lot of times is things like travel records.

And a lot of times it comes down to small mistakes like, well, I flew in just at 2 a.m. instead of leaving before midnight. That's why the article that references these travel things is important. So if you can fly into New Jersey instead of New York or you can drive in and out of California instead of flying into California, well, now you're reducing the amount of evidence that an opponent in court can generate to show the specific day and time that you arrived into the state.

If you're going to go on vacation, then you may want to rent a vacation home using an entity of some kind or you want to have a friend of yours rent the vacation home instead of you being the one to do it. Make certain that you have all of the evidence indicating your presence in the other state and make certain that there's a minimum amount of evidence indicating your presence in the high-tax state.

Now, if this sounds like an enormous hassle, it should because it is. And that's why I led with the advice of make certain that if you're going to move, you're moving because you genuinely want to, not just because of tax considerations. If you're going to genuinely move, then genuinely move and don't play around with this.

Don't try to play close to the line. What's the point of being rich if you're worrying about an audit from a tax authority because you're playing around close to the line? And then after those important things that solve most of the problems, make certain that you put in place all this other stuff so that if you ever do wind up in an antagonistic situation, an audit or in tax court, with them alleging you owe us $4.2 million, make certain that you have an enormous stack of paperwork on your side of the table showing your true and bona fide residence in another state and that there's not any significant paperwork that the auditors can come up with from your bank records, from your phone records, from all of the other stuff, showing that, oh, in fact, actually, you were in New York for 184 days because you arrived early and left late.

Pay attention to this stuff and do it right. - Spoiler alert, a two-word warning that we're about to ruin the end, but sometimes spoilers can be worthwhile. And so we designed the Lexus NX to reveal many things, like taking the mystery out of how close you are between parking space lines or the time you'll arrive at your dinner reservation.

Technology designed to remove surprise endings. It's just a better story that way. Click the banner to discover more. Experience amazing at your Lexus dealer.