Nine out of 10 families are affected by the adults working unsocial hours, with the result that children are missing out on crucial time with their parents.
Eight out of 10 working fathers and more than half of all working mothers are forced to work hours outside the range of 8am to 7pm during the week, a report from the National Centre for Social Research found.
The study of 11,000 people claims family life is being irrevocably damaged as a result, because few parents make up the time lost.
Only 17% of families had a parent working a "normal" nine-to-five week.
Mothers working unsocial hours lose about eight hours a week with their children, while one third of working fathers lose more than 15 hours.
"We must take collective responsibility [because] we are depriving children of time with their parents," said Michael Clark, chief executive of the Relationships Foundation, a partner on the study.
Children of all ages ate less regularly with their parents, particularly where the mother worked unsocial hours.
The government said it was providing a series of family-friendly measures to help parents, including extending maternity pay and offering new rights to paternity leave.