The earl has made a series of damaging claims about the King in his memoir chronicling the life of his sister Diana that prompted the response from Buckingham Palace, which said "the pain of fraternal grief can cloud reason".
Now, the earl has accused the King of gaslighting him and called the response a "rather poor imitation of the 'recollections may vary' quote" - the late Queen's response to Prince Harry and his wife Meghan's allegations in their Oprah Winfrey interview, The Mail on Sunday reports.
"It is not every day you wake up to find you've been gaslit by the King," he said.
"It was astonishingly provocative and below the belt because it was essentially trying to say I'd lied without the King being able to deny what I'd said.
"I also thought it was a rather poor imitation of the 'recollections may vary' quote, which was a much more elegant way of putting things, but in these circumstances entirely out of place."
"Diana would be cheering from the rafters at this book," he said.
A Buckingham Palace spokesman on Wednesday responded to the allegations about the King in the memoir.
"While we do not comment on books as a matter of principle, His Majesty is mindful that the pain of fraternal grief can cloud reason, affect judgment and colour memory in ways others do not recognise, even many years after such a loss."
In an extract of a BBC interview released on Friday, the earl insisted he was "telling the truth" and described the King's comments about his sister as "monstrous".
Spencer spoke about an argument he had with the then Prince Charles about Diana's funeral arrangements involving Prince William and Harry.
"(The King) went ballistic, would be the right term, and started really going for my family as a whole, about their sense of duty, not being adequate enough, and ranted for a while, I have no idea how long, but it was a significant period, and I just listened.
"And then he stopped, and I thought, 'Oh now he's going to pull himself together and say something more appropriate'.
"And then he said, 'We'll forget her soon enough'."
Asked whether he had "evidence" of the King's comments, the earl told the BBC: "I told friends and family straight away, close friends and family, about what had been said, because it was so monstrous."
"I'm telling the truth," he said.
Charles is said to have made the comments about his ex-wife in a row with the earl over whether William and Harry should follow Diana's funeral cortege.
In the latest extract of the earl's book serialised in the Daily Mail, Spencer told of his memories of the funeral, and his concern for Harry, after he observed a "lack of obvious care" for the then 12-year-old.
Spencer walked behind Diana's coffin during the funeral procession on September 6, 1997, alongside Harry, William, Charles and Prince Philip.
Recounting the train journey from London to Althorp after the funeral, Spencer said he was "concerned at the lack of obvious care for Harry, as he decompressed from the ghastliness of the morning".
"He needed Diana's hugs, of course, and they would forever be denied him.
"But instead of calm and nurture, he was allowed to spin out of control."
Spencer said he could not get over how "tiny, and slight, and bereft" his nephew looked.
