Part of this I believe is that time is intangible. It cannot be felt. If a reader says that the time could be fall, due to their visions of seeing leaves on the ground, they don't know the month, or even the year. A big 2013 or 2014 does not flash across their minds eye (although it would be much more helpful if this did happen), but they are just as much in the dark with timing as we are.
I'll give you an example. I taught my dog commands: sit, lie down, shake, backup, come here and 'stay'. Stay took her longer to learn because there was nothing concrete behind it.
I pushed her bottom to the floor to learn sit.
I pushed her body down to learn lie down.
I put her paw in my hand to learn shake.
I pushed her shoulders back to learn backup.
When she was some distance from me on her lease, I would pull her to me and say come here.
I didn't even know what to use to teach her stay other than to say it then leave the room (often times she would follow me). She eventually learned what stay means, but as I mentioned, it took longer because nothing tangible was associated with it.
And time is never constant, although we have routines at certain times of the day. Time is always changing. Always. It's the same thing when you try to take a picture of a moving object. It becomes blurry or distorted and hard to distinguish because it is moving. If readers stop acting like god and understand their limits, IMO they'd be much more accurate.