I can duplicate this, and dropping patch 121_only_switch_vt_when_active.diff from xorg-server fixes it here as far as I can see.
I can duplicate this, and dropping patch 121_only_ switch_ vt_when_ active. diff from xorg-server fixes it here as far as I can see.