As we grow older times seems to fly by faster and faster. Truth is, as far as physics are concerned, time is a pretty stable. What seems to mess up our perception about the world is the peculiar way our memory works.
Our brains are not that amazing as we like them to be. The information that we use on a daily basis is nothing more than a random aggregate of different stimuli — and most of that essential information is either discarded or manipulated from our imagination.
The simple reason why times seems to run faster and faster we grow older is because our lives start operating on repetitive patterns. For efficiency, our brains repeat much of our actions and functions through elaborate shortcuts in order to retain a shallow set of information. What ends up being retained is determined from emotional stimuli that etch data into long-term memory. This is why traumatic experiences are so hard to forget. Whether they are positive or negative, they carry a certain mnemonic weight due to their emotional impact.
A recent paper published in Neuron revealed that our memory compressed information much like a computer. For those not familiar with the term compression works by mapping specific patterns. For example, if we have the numbers "1,4,2,1,3,4,4" in a data set, then numbers 1 and 4 could be compressed and represented differently (e.g by the letters x and y) because they are repeated data in our set.
Our brains works much the same. As we experience our daily lives, more and more information are the same so the brain does the best it can to group the same data under the same compression. Thing is, our brain, when unable to perform meaningful compression, it starts creating data of its own by mixing it up with the actual information. Dreams are one source where actual information gets mingled with real information, forever corrupting them into different sets of data. Memories can also be implanted and manipulated at will from different sources other than ourselves. The longer we delve into the past the more likely the events did not take place due to sheer entropy of data corruption.
In a sense we do comprehend that "something is off" as we grow older. In an effort to compress the same data as we grow older, our mammalian brain resorted to summing up corrupted data repetitions. Our brain does not care about the meaning we attribute into our lives because all it cares about it survival. All mornings do end up pretty much the same. Sex experiences are summed up into on generic thing after a while. Encounters with friends, walks at the part, traveling. The way our brains summarize information also helped us to communicate with each other more efficiently. We can talk about mornings or math and we all have a general idea about the concept even if our data sets are compressed differently.

One would ask what happens with decompression. It seems that our brains are not as good as the decompression software that we use. Everything gets compressed but our brain is unable to decompress all of the information due to its sheer volume. The process is too slow and humans need to have past information on the go.
Once again the brain resorted to basic evolutionary mechanisms that evolved along with it. The basic blueprints that we get from this compression are extremely generic but also essential to our survival. This is why most people find "love","family", "friends","patriotism" the most rewarding of all experiences. They are data sets that are over-generalized but also wildly shared. They are also emotionally charged information which means they stick around the strongest in our long-term memory. This is how we can create a meaningful narrative about our past in 1 minute by by-passing 99.9% of the actual information. At the end of the day, we end up longing for meaning by chasing information that has been long corrupted from time. Our compressed memories degrade by over copying onto themselves and as a result time travels faster and faster.