Byrne was grilled by MPs on the Joint Committee of Human Rights on the controversial changes in November 2006 that toughened the criteria for HSMP visas. Foreign workers already in the UK on the visas were effectively required to reapply under the new rules.
MPs questioned the minister on the fairness of requiring a person to commit to the UK as their main home, and then moving the goalposts when applying for permanent residence.
Campaigners have estimated up to 40,000 foreign workers could be forced out of the country by the retrospective changes.
Byrne admitted there were “emotional arguments” about migrants’ expectations on staying in the UK, but said there was strong evidence of people abusing the system.
“It was quite clear people were coming to the UK and spending extended periods of time without working,” he said. “That defeats the objective of the programme and why it was set up.”
A judicial review into Byrne’s decision is set for next month. Opposition MPs, an immigration judge and the equalities commission have all previously said the changes were unlawful.
But Byrne said: “It has always been clear that there is a further test when applying for leave to remain. That is why the Home Office is contesting the judicial review.”
Amit Kapadia, director of the HSMP Forum campaign group, said he was “deeply concerned” at Byrne’s rigid stance on the move.