The deal to self regulate was struck during a meeting at the White House on Tuesday aimed at addressing Americans' fears about the technology.
"I think I'm seeing tremendous self-policing. And they understand that they have to self-police," Trump told reporters during an impromptu press conference outside the West Wing.
Trump met with leading executives from top tech firms that are spending hundreds of billions of dollars to build out AI infrastructure, including data centres in communities across the country, which have sparked grassroots, bipartisan opposition.
The accord was signed by Trump, Anthropic CEO Dario Amodei, Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, OpenAI President Greg Brockman, Nvidia CEO Jensen Huang, and Elon Musk, founder of xAI, which is now part of SpaceX.
The accord, which Trump posted on Truth Social, opens the door to future regulation but focused on four voluntary steps for the companies to take.
The accord said the companies would implement "robust internal controls," partner with an "independent external auditor" to assess whether the controls were working, and establish a committee within each company's board of directors to evaluate reports from internal and external auditors.
"Over time, it may make sense to codify these steps into laws and regulations," the accord said.
Trump repeatedly praised the business executives for developing a technology that he said would produce "unbelievable growth".
Investment in data centres and the accompanying computer equipment has accelerated the economy and boosted the stock market, which has swelled investment accounts for higher-income Americans and lifted their spending.
Yet there were some notes of caution. Amodei, who has been one of the most outspoken voices calling for a slowdown in AI development, admitted "the technology has very real risks".
"And, you know, the mechanism, how we address those risks is still under discussion," Amodei said.
Trump suggested about 10 people would be named to a committee to "watch over the whole enterprise" and said the accord would be "morally binding".
He also said he would name someone to oversee the agreement in coming days after consulting with industry.
Some of the actions outlined in the accord are measures the companies are already taking in some form or have previously committed to doing.
Some experts are sceptical of putting "so much faith in self-policing", like Robin Jia, an assistant professor of computer science at the University of Southern California.
He pointed to the recent disclosures of hacking incidents.
"As the models get better, the consequences of such things are only going to get greater".
Some took issue with the accord's vague, broad language, including Alex Pascal, executive director of the Berkman Klein Center for Internet and Society.
"The only way to reduce the risks and harms of AI to Americans is through robust legal liability, regulation and fundamentally changing the race dynamics that are driving unsafe decision-making by frontier AI developers whose costs are borne by the American people," he said.
Earlier on Tuesday, Trump promised that the US government wouldn't limit the development of AI or support calls for "guardrails" or other restrictions.
"I will never stifle the growth of a technology that will be bigger than the industrial revolution," Trump said at the unveiling of an AI chatbot intended to help navigate government services.
"We can't stifle it, and we're leading by a lot."
Trump also said the executives had agreed to assuage the bipartisan opposition to data centres cropping up in local communities across the country by providing more financial support to local schools or taking steps to reduce energy costs.
"You're going to see, data centres are going to be very popular," Trump said.
with AP