The study has cited financial crisis as the reason for the brokerage houses losing the grip on their HNW client assets, adding that the situation is likely to worsen.
The market share of the companies which was at 56% in 2007, reduced to 45% in 2011 and is expected to further fall to 42% by 2014.
Together the companies had US$2.1 trillion in assets from clients with at least US$5 million to invest, the report reveals.
In the current situation, boutique firms, trust companies, family offices and private-client businesses owned by rival investment banks are the ones gaining clients over the bigger brokerage houses, says the study report.
Cerulli analyst Rob Testa said: "Firms that were perceived as safe provided a safe haven for nervous investors and advisers that were ready to make a move."
She cautioned the development as not being positive for the big brokerage houses that are targeting wealthier clientele to boost their revenue and profitability.
Cerulli added that the market shifts were first driven by the 2008 financial crisis, when Merrill was rescued by a Bank of America takeover and taxpayers had bailed out Morgan Stanley, Citigroup and UBS to keep them solvent.
Subsequently, RIAs and family offices grew the fastest, with their AUM being increased by 18% to US$356 billion in 2010, compared with a 2% increase seen among the four brokerages.