If you really want to get away from the city lights and see the night sky, New Mexico has a number of world class stargazing spots, like Chaco Canyon in the northwest, Capulin Volcano in the northeast or the Cosmic campground in the southwestern Gila wilderness to name a few. But you don't have to drive that far to see the unfettered majesty of the nighttime sky if you are in Albuquerque; a quick trip outside of town is all that is required.
One great option you can try if you are on the north end of town is to head up I25 to the Budaghers exit, and head east on 57a towards Highway 14. This is a well maintained dirt road that should be no problem even in a small car. After a few miles you will find yourself at a point where the lights from both Albuquerque and Santa Fe are blocked by the mountains. On clear nights, the stagazing is great out here, and this would be an equally good place to head for if you are in Santa Fe as well. You can loop around the Sandias all the way back to Albuquerque via 14 if you want.
The Ojito wilderness is another great spot to the north, popular for watching meteor showers with those in the know. Just south of the Zia Pueblo, the area is an austere vista of terraced mesa, box canyons and arroyos, dotted with piñon and juniper and the occasional stand of ponderosa pine. To get there from Albuquerque, take US 550 north past Benalillo to county road 906, also known as the Cabezon (Stubborn or Mule-headed) Road.
From points of town south of I40, there are plenty of options as well. You can head west and soon be far away from any civilization, but to really get away from the city lights, head east through the mountains to Tijeras and head south around the far side of the Manzanos. From out here you could never tell that a million people and all of the light pollution that goes with them are so close.