The Yorkshire Building Society group will refund approximately £8.4m to about 34,000 of its mortgage borrowers, according to the British market watchdog the Financial Conduct Authority (FCA).
Customers are expected to receive an average refund of £247 each.
According to the FCA, Yorkshire Building Society Group (YBS), which includes Yorkshire, Chelsea and Barnsley building societies, and Accord Mortgages, will repay the wrongly charged fees for failing to keep up with mortgage payments.
The UK financial market regulator said that YBS can charge an arrears administration fee when a mortgage account falls into arrears by two months or more, encompassing the administrative costs in dealing with the arrears.
However, thousands of customers of the Yorkshire, Chelsea and Barnsley Building Societies were charged immediately when they missed a payment and in some cases they were charged arrears administration fees since January 2009.
Financial Conduct Authority supervision director, Clive Adamson, said, "The FCA has worked closely with Yorkshire Building Society to get to the heart of this issue and ensure that customers are refunded what they are owed."
"We have been encouraged that they are now putting customers first by paying redress to everybody in arrears, not just those who were overcharged.
"But this is nonetheless a very important reminder that firms must take responsibility for any wrongdoing and make sure their customers get a fair deal," Adamson added.