Bob Marley's Tuff Gong studio in Kingston, say, or a couple of days in the Abbey Road studios in London - Willoughby, 64, just has a feeling songs would flow.
He's been named as the 2024 recipient of the Ted Albert Award for Outstanding Services to Australian Music, a gong that has previously gone to Paul Kelly, Slim Dusty, The Seekers, Cold Chisel and the late Archie Roach.
"It's hitting the bullseye - like winning the lottery," Willoughby told AAP.
"It's amazing to be accepted into one of the most terrific gangs of songwriters."
Willoughby was the lead singer and drummer for the trailblazing outfit No Fixed Address, the first Aboriginal band to score a record deal, tour overseas, and appear on Countdown.
He hopes the spotlight that comes with the award will be a chance for people to revisit the band's early music, and the father of six and grandfather of many more says in the end, the Beatles were right.
"It's just hard work and love, it's amazing what hard work and love does, and I think that's all you need."
First formed in 1978, No Fixed Address was also one of the first Aboriginal bands to play reggae/rock, and to write songs about contemporary politics and the experiences of Aboriginal people.
Its members, Ricky Harrison, Les Graham and John Miller along with Willoughby all had family ties, coming from the Koonibba mission in South Australia.
Willoughby's song We Have Survived, with its hard hitting lyrics about the hurt and torment of the "white man's world" became an anthem for many, with No Fixed Address influencing the likes of Yothu Yindi and the Warumpi Band.
The Pitjantjantjara and Mirning musician was one of the Stolen Generations, taken from his family as a three year old in the South Australian town of Ceduna.
While he puts his storytelling talents down to the bedtime stories he told in the children's home where he grew up, he says it took him years to realise the power of his own songwriting.
"When I put down everything, I could actually make people dance - wow - and also cry too when I do a love song," Willoughby said.
As well as No Fixed Address, which has re-formed several times and toured nationally in 2023, he was also the driving force behind the band Mixed Relations, and played in the Black Arm Band and Yothu Yindi.
In the early days of No Fixed Address, gigs would sometimes be cancelled when promoters discovered the band was Aboriginal.
But in some ways the music industry was less difficult in the 80s and 90s than it is now, said Willoughby - musicians could build audiences through pub gigs, or ring friends to spread the word.
He has learnt a fair bit about songwriting since then: for him, the music speaks first and the lyrics follow, sometimes with political truths that need to be told.
"You tackle it, you make sense out of it, and then you give it to the people, but you got to have confidence in what you're singing about," he said.
A multi-instrumentalist, he can turn to drums, guitar, bass, and says he's also "OK" on piano.
He's recording in a studio based in Ballarat, Victoria, and relishing every moment.
"I'm like a little kid now, and I'm allowed to go into where all the toys are and have fun," he said.
Willoughby will receive the Ted Albert gong at the APRA music awards in Sydney on Wednesday.
4°C