The conservative groups complain their requests were delayed for months or even years through the review process, which is intended to prevent ineligible political groups from getting tax-exempt status.
Miller told the House Ways and Means Committee on Friday that determining the political nature of groups was one of the hardest tasks of IRS officers responsible for assessing requests for tax exempt status.
The investigation by the Treasury inspector general for tax administration was initiated after congressional complaints began to surface in the media in 2012 that the IRS was targeting conservative groups and holding up applications.
In a written response included in the report, the IRS commissioner of the Tax Exempt and Government Entities Division said there was no criminal behavior behind the actions of the agents, but rather inefficient management.
Obama called practices described by the inspector general outrageous and forced Miller's resignation. In addition, the commissioner of the IRS Tax Exempt and Government Entities Division also announced his retirement Thursday. Joseph Grant will leave in June, according to an internal IRS memo provided to CNN. Miller also is scheduled to exit then.
Obama has appointed Danny Werfel, a White House budget office official who has served in both Democratic and Republican administrations, to succeed Miller through the end of the fiscal year on September 30.