Critics have argued for years that the penny was becoming irrelevant and unaffordable and should be put out of its misery.