The 34-year-old British pop star admits that it's difficult to raise a family while living in the public eye.
Sheeran - who has daughters Lyra, four, and Jupiter, two, with his wife Cherry Seaborn - said on The Louis Theroux Podcast he was "very, very private about my children's image".
"I can't take them to zoos or parks or anything really without someone trying to film them".
"There are times where the normal parts of life I kind of mourn for and wish I could push my kid on a swing in a public park and it not be weird, you know?"
Sheeran pointed out that his kids "have not signed up" to the pitfalls of fame and success.
"People go, well that's the trade-off. But my kids have not signed up to that."
Despite his best efforts, Sheeran admits that it's impossible to shield his children from fame all the time.
The award-wining star prefers to take commercial flights because of environmental concerns but he conceded that it can lead to problems.
"We're flying back on Ryanair from a gig in Italy last year but all of my fans who'd been at the gig were flying back!
"I like the environment, I like trees. I'd find it hard to justify. No one's perfect. When we're doing intense promo trips, there will be the odd time (taking a private jet)."
Sheeran believes his music career peaked with his Divide album back in 2017. But he's actually feels much happier now.
"I'm kind of coasting now. I don't know if I'll ever get back up there again but (when I had) that level of fame, that was my unhappiest (time).
"I'm much happier with the balance I've got now."
5°C