I've got a degree in Criminal Justice and there are some important points that should be added.

The first is that 63%(last time I looked) of current inmates are there for non-violent drug offenses. The second is that while 1%(3,000,000) of the population are currently in prison; there are 4-5%(~16,000,000) of the population who are on probation/parole. As it is extremely easy to revoke probation or parole for very minor issues, this much larger group is nothing more than a ready source of additional prisoners if the State ever decides it wants more.