The IRS has 3 main types of penalties that they might assess on taxpayers.
Failure to Pay (IRC §6651 or §6654 for estimated taxes) - equal to one-half of 1% for each month up to a maximum of 25%. The one-half of 1% increases to 1% if the tax remains unpaid 10 days after the IRS issues a notice of intent to levy property. Note that the penalty for underpayment of estimated taxes varies from the preceding explanation.
Failure to File (IRC §6651) - equal to 5% of the tax owed for each month up to a maximum of 25%. If the return is over 60 days late, there's also a minimum penalty for late filing which is the lesser of $210 or 100% of the tax owed.
Dishonored Payment (IRC §6657) - equal to 2% for payments of $1,250 or $25 (or the amount of the payment if it is less than $25).
Interest accrues on any unpaid tax from the due date of the return until it is paid in full. The interest rate is calculated by taking the federal short-term rate and adding 3%.
One important thing to note is that while penalties might be waived at the discretion the IRS; interest is statutory and cannot be waived unless the tax liability is also dismissed.
Reference
Penalties
https://www.irs.gov/businesses/small-businesses-self-employed/understanding-penalties-and-interest
Penalties & Interest
https://www.irs.gov/taxtopics/tc653
Disclaimer
Any accounting, business or tax advice contained in this communication, including attachments and enclosures, is not intended as a thorough, in-depth analysis of specific issues, nor a substitute for a formal opinion, nor is it sufficient to avoid tax-related penalties.