I hadn't realized just how much your day-to-day experience colors your overall perception. For example, listen to this little (absolutely true) story.
Last night, my wife and I decided to go to the store to get some food. It's been fairly cold recently, so we decided to have a quick look at the weather to see if we should wear a coat or just a light jacket. The thermometer showed that it wasn't too cold, so we both decided to take just a light fleece. We had a fun evening; it was cool out, but not too cold, just like we had envisioned.
Now that you've heard the story, let me fill you in on the details. We live in Colorado up in the "high country," at around 7000 ft elevation. When I said it had been "fairly cold," I mean that temperatures had gotten to -15°F the night before; that's before adding wind chill. Days had stayed in the single digits most of the week.
So when we decided that it was light-fleece weather, it was actually 25°F outside, still well below freezing. Of course, it's not humid here, so it was a nice, cozy 25°; but still, well below freezing. So how exactly did I begin to think that this sort of weather is not that bad? Well, my criteria for this sort of decision has changed. For example, above about 10° it no longer hurts to breathe. That makes me feel a lot warmer. Above 20°, I can make it from my car to the store without losing too much body heat. I never intend to spend more than 2 minutes out in the weather at a time, so extended exposure really doesn't even factor into the decision. Having spent all my childhood in Phoenix, I still find my new perception on this subject quite surprising.