Agreed, and yes, lots have done it, including, as I recall, Churchill. But just because it's been done, doesn't mean it should.
I'd be in favour of insisting on a by election for anyone wishing to change party allegiance, or even anyone who has 'the whip' permanently removed by their party.