The men, aged 59 and 61, were stopped three kilometres off St Kilda when the cruiser's engine room began giving off smoke about noon on Wednesday.
When the fire suppression systems didn't work, they each grabbed a life jacket and jumped into the water.
Army members who were conducting a drill near Williamstown were first on scene about 15 minutes later.
The police water squad, Australian Border Force and Fire Rescue Victoria also attended.
The men were cold and clearly shaken but otherwise uninjured when crews arrived, Inspector James Dalton said.
They were checked by paramedics when they returned to shore and were able to go home without further treatment.
"They're extremely lucky," Insp Dalton told reporters.
"The boat could have exploded - they could have got trapped in the cabin or not been able to get out because of the smoke."
The boat will stay at the bottom of Port Phillip Bay while insurance is determined, Insp Dalton said.
"It appears to me it was their pride and joy so (the men) are pretty upset," he said.
Several witnesses called emergency services after seeing large plumes of smoke from the boat.
Some pieces of wood and other items were able to be recovered from the water.
12°C