Just south of town is Galisteo Basin, an awesome set of trails that offer challenge to riders. It's definitely singlespeedable. I did it with a fully rigid bike and only had one or two scary moments on some of the steep switchbacks. There is plenty of loose rock and sand, but that won't slow you down too much. Great signage everywhere makes it easy to navigate. Over time I think you could probably loop together some great routes, I was just winging it.
Location: Lamy, NM, just south of Santa Fe
Just take 285 South from Santa Fe to Astral Valley Road in Lamy. It's well marked and you can find the trailhead at the first right on New Moon Overlook Road
Technical Level: Medium. Without the climbing up in the north by the Thumb Trailhead the trails are probably all low technically. There are some fun, tricky off-camber sections to keep you awake.
Trails: Rocky and sandy like much of the Santa Fe area