Yep, or scrap the batteries and the USB supply altogether and use a simple resistor (20k -100k) and a 3v zenner diode.
Or substitute the 3v zenner for a string of 6 conventional diodes.
This would cost pence to build and would fit into the
battery compartment.
View attachment 145393
EDIT.
You of course only need a string of 5 diodes not 6 as drawn to give 3v. Well done at the back!