Yes, its that 0.1V affecting the LEDs, your PSU is fine:
Forward voltage (Vd) is the voltage required for the LED to light up. Looking at the datasheets of an LED, at the graph of current V/S forward voltage, you can see that going from 3.3V to 3.0V, current draw drops from 20mA to 8mA.
To avoid this, you can use 3 LEDs in series instead of 4, and add a current limiting resistor for the remaining voltage:
R = U / I = (12V - 3*3.3V)/20mA = 105Ω
Or you can buy the LED strips, which have internal current limiting resistors for 12V.