Durango. Here's our math on why we like this town:
1) Drivability - It is a 12ish hour drive from our home in Texas. One day drive is key to us. Also, we get to stop in Santa Fe on the way, which is a big plus! This is why we ruled out Steamboat Springs. Also makes it easier to take the dog.
2) Affordability - It's not cheap, but it isn't INSANE like most of Summit and Eagle Counties.
3) Not a Ski-Town but you can Ski - Durango is NOT a ski town. It's a combination of a college town (Ft. Lewis), mining town and western town. It isn't really seasonal, has a large number of full-time residents (the vast majority), has "real town" stuff - a great downtown, hospital, libraries, businesses, neighborhood restaurants and normal humans. If you want to ski, Purgatory is a solid if not world-class ski resort. It has expanded a lot in the last 8 years, and many folks don't know that. You can also drive to Wolf Creek, or Telluride (for a bit further of a drive).
4) Summer is its Stong Point - There is just so much to do here. The fishing is world-class. The hiking is incredible. The jeep trails are also world class. I love to ride the narrow-gauge railroad (and so do the kids). Restaurants are great. You're within a 3-4 hour drive of many of the big National Parks (Grand Canyon, Canyonlands, Bryce Canyon, Arches, Rocky Mountain National Park, Mesa Verde). An amazing offering, really. Santa Fe is also an easy drive.