Yes if the center tap of the secondary is connected to the neutral side of the of the 240V this will cause all sorts of problems. If center tap of the secondary is connected earth, and earth is connected to the neutral at the fuse board, then essentially you have the same circuit, no?
Leaving it to float, is an option, but isn't the 120V-0V configuration effectively the same as a US outlet now?
You don't want the center tap connected to anything. But yes 120-0V with the 0V connected to the ground lug--and earthed, is essentially the US system. Remember the US distribution:
120-0-120, with the 0 also earthed. The 0 side is the wider of the two prongs on a USA plug--don't get those mixed up. So the wide prong in US outlets is connected to 0, with the 3rd ground prong earthed--usually, the two are connected together in the breaker panel. The narrow prong on the US plug is connected to 120V; it doesn't matter which side of the distribution transformer winding.
US electrical code requires an independent conductor for the grounding prong, even though it may be connected to the neutral side of the line in the breaker (distribution) box. Safety is provided by a GFCI (if the fault circuit interrupter detects that the current between the two flat prongs isn't the same, the circuit is interrupted.) So, tossing a hair dryer into the bathtub will cause the (now code) GFCI to trip, regardless of what the TV murder mysteries say.
240V appliances are a different matter. An electric oven may have 4 conductors; two for the 240 heating elements, one for the 120V to run the clock and controls and a fourth for earthing the body of the oven.
On the other hand, my table saw is 240V with only a protective ground wire--but that requires a special circuit.
I do envy you fellows in the UK. Here in the US, the most powerful home vacuum cleaner available is a paltry 1440 watts. Code requires that a portable device (i.e. something that's not wired in) to draw no more than 80% of the circuit capacity. Since a lot of US homes are still wired with 15A, 120V outlets, the maximum current draw allowed is 12A.