Ok, maybe this is my first successful project (not counting flashing LEDs in sequence off an arduino). The Ladyada breadboard power supply sorely lacks a voltage display when in adjustable mode.
I found a device called 'LED 1s-6s Lipo Battery Voltage Indicator Checker Tester' on Ebay. Here are links that might work in a few weeks still :)
It is a better choice than the more expensive 1-8S tester with low battery alarm because it will work in 1S mode without flashing voltages. Now the trick it is powered by the voltage it's measuring, but the voltage cannot drop below 3V it won't work.
You can however measure down to 1V (or maybe even less) if you apply 3V or more on one other pin, but you don't want to apply that voltage to the other pins before the unit has initialized. If it 'boots' with only one voltage, it will nicely just display it and then you add voltage to pin 3 and it will measure pin 2 voltage down to low values without shutting off. You just don't want it to boot with voltage on pin 3 or it will alternate between showing voltages on pin 2 and 3, which is what you want for a lipo battery, but not for this application.
Maybe I could have done something complicated with a capacitor or a timer, but I just put a switch that turn off and only turn on to apply power to pin3 when I need to measure small voltages. It's a bit hackish, but it cost $5 for the digital voltage meter, which is hard to beat :)
End result pictures below:
measuring just 1.42V