Rescuers holding torches were going from apartment to apartment at the charred towers as thick smoke poured out from some windows at the Wang Fuk Court complex in the Tai Po district.
Officials said firefighters were still working on a handful of apartments and trying to enter all units in the seven towers to ensure there were no further casualties.
"Our firefighting operation is almost complete," said Derek Armstrong Chan, deputy director of Fire Services Operations.
Firefighters were working hard to prevent the debris and embers from flaring up.
"What's next is the search and rescue operation," he said.
It was unclear how many people remained missing or trapped. Hong Kong leader John Lee earlier said contact had been lost with 279 people.
Firefighters have been trying to control the flames since Wednesday, when the fire was believed to have started in bamboo scaffolding and construction netting and then spread across seven of the complex's eight buildings.
Chan said the blaze spread "exceptionally fast" across the towers, and emergency workers struggled to gain access inside.
"Debris and scaffolding were falling from upper floors," he told reporters.
"There are also other reasons like high temperature, darkness ... (and) emergency vehicle access was blocked by fallen scaffolding and debris, making our access to the building very difficult."
A distraught woman carrying her daughter's graduation photograph searched for her child outside a shelter, one of eight that authorities said are housing 900 residents.
"She and her father are still not out yet," said the 52-year-old, who gave only her surname, Ng, as she sobbed.
"They didn't have water to save our building."
Police arrested two directors and an engineering consultant of Prestige Construction, a firm contracted to perform maintenance on the buildings.
Police said those arrested were under suspicion of manslaughter for using unsafe materials.
"We have reason to believe that the company's responsible parties were grossly negligent, which led to this accident and caused the fire to spread uncontrollably, resulting in major casualties," police superintendent Eileen Chung said.
Prestige did not answer repeated calls for comment.
Police seized bidding documents, a list of employees, 14 computers and three mobile phones in a raid of the company's office, the government added.
It was Hong Kong's deadliest fire since 1948, when 176 people were killed in a warehouse blaze.
In a telegram to Hong Kong's bishop, Cardinal Stephen Chow Sau-Yan, Pope Leo sent "spiritual solidarity to all those suffering from the effects of this calamity, especially the injured and the families who grieve".
Hong Kong leader Lee said the government would set up a HK$300 million ($A60 million) fund to help residents.
Chinese companies and groups, including car makers Xiaomi, Xpeng and Geely as well as the charity foundation of Alibaba's founder Jack Ma and Tencent announced donations.
The eight blocks of the tightly packed complex of 2000 apartments were home to more than 4600 people in the financial hub, which is struggling to overcome chronic shortages of affordable housing.
The apartments were under renovation and clad in bamboo scaffolding and green mesh.
Police also said they found foam material sealing some windows on one unaffected building, installed in year-long maintenance work.
The city's development bureau has discussed gradually replacing bamboo scaffolding with metal scaffolding as a safety measure.
With AP
