Disabled employees have even less chance of getting a job during a recession, the TUC has warned.
Frances O'Grady, the deputy general secretary of the union body, said employers considered it too much time and effort to adjust their workplaces to hire disabled people when budgets were tight.
O'Grady told Personnel Today: "The employment rate among disabled workers is already up to three times worse than others.
"Getting them into work can be tough anyway, because there is prejudice from employers, who don't understand there is practical support and guidance to help [disabled employees] adapt to the workplace," she said.
"It's seen as another hassle - views of what a disabled worker is capable of really vary - and during a recession when times are tough that group [of workers] can be less attractive to employ."
O'Grady added that unions could play a huge role in working with employers to make sure they understood what guidance and financial support was available to them.
She said HR managers needed to work with union representatives and equalities bodies, such as the Equalities and Human Rights Commission, to help business leaders understand the benefits of a diverse workforce.