Embattled Housing Minister James Browne came out fighting last night and declared he will not be 'derailed' by 'whispers and ...