Britain has lost more than it has gained since Brexit: Prime minister