The IRS is going after high-income earners who skipped out on filing federal income tax returns in more than 125,000 instances since 2017, the agency said Thursday. 

They’re believed to owe, based on a conservative estimate, hundreds of millions of dollars, the IRS said. The number could be much higher – but the IRS said it can’t be sure of an exact amount since the agency doesn’t know what potential credits and deductions these people may have.

“At this time of year when millions of hard-working people are doing the right thing paying their taxes, we cannot tolerate those with higher incomes failing to do a basic civic duty of filing a tax return,” IRS Commissioner Danny Werfel said in a press release. “The IRS is taking this step to address this most basic form of non-compliance, which includes many who are engaged in tax evasion.”

The IRS will start sending out compliance letters this week. More than 25,000 letters will go out to people with more than $1 million in income and more than 100,000 letters will go out to people with incomes between $400,000 and $1 million between tax years 2017 and 2021. It’s not clear how many people will be impacted.

“Some of these non-filers have multiple years included in the case count so the number of taxpayers receiving letters will be smaller than the actual number of notices going out,” the IRS said.

IRS officials said they were able to identify the people who hadn’t filed using third-party information, such as W-2s and 1099s, indicating these individuals were making money, but not filing returns. 

The agency has been able to step up enforcement using funding from the Inflation Reduction Act, which became law in August 2022. Earlier this month, the Treasury Department and the IRS, estimated that tax revenues would rise by as much as $561 billion from 2024 to 2034, thanks to the stepped-up enforcement.

Treasury Secretary Janet Yellen recently told CBS Pittsburgh that the IRS has been “starved” for resources for decades. She said it’s led to bad customer service and very low levels of audits of wealthy individuals, complex partnerships and corporations.

Werfel noted the IRS had only been able to run audits sporadically for non-filers since 2016 because of severe budget and staff limitations. 

“This is one of the clearest examples of the need to have a properly funded IRS,” he said. “With the Inflation Reduction Act resources, the agency finally has the funding to identify non-filers, ensure they meet this core civic responsibility, and ultimately help ensure fairness for everyone who plays by the rules.”

The penalty for failure to file amounts to 5% of the amount owed every month – up to 25% of the tax bill, according to the IRS.

“If someone hasn’t filed a tax return for previous years, this is the time to review their situation and make it right,” Werfel said. “For those who owe, the risk will just grow over time as will the potential for penalties and interest.”