Harrison suffered a heart attack at his home in Ashburton, Devon in England after performing at the village of Black Dog in Devon.
His wife, Lori Chapman, told the Press Association, "He will be loved and missed by more people than I ever knew."
The son of Rex Harrison and his first wife, Collette Thomas, Harrison was a member of the British ski team and competed in the 1952 and 1956 Winter Olympics. But he soon left the slopes and began playing guitar and singing at nightclubs and bars around Europe before coming to America in 1965.
In 1966, he was cast opposite Stefanie Powers in the lighthearted spy spoof "The Girl From U.N.C.L.E," which lasted one season.
"My darling friend Noel Harrison passed last night," Powers posted on Twitter. "Let us all light a candle to speed him on his way -- he deserves to fly with the angels."
His recording of "The Windmills of Your Mind," which was used as the theme music from the 1968 film "The Thomas Crown Affair," was a top 10 hit in England. He released several albums including "Collage" and "Santa Monica Pier."
On his website, Harrison wrote that he didn't like being a celebrity in Hollywood. "There are lots of perks, pretty women all over you, good tables at fancy restaurants ... but my marriage was crumbling and I felt as if I was exposing my emotional and distressed state in a fishbowl."
He moved to Canada in the early 1970s, toured the U.S. in his father's big stage hits "My Fair Lady" and "Blithe Spirit," and returned in the late 1990s to England, where he continued to perform and record.
"We're living in south Devon now, in the west of England," Harrison said on his site. "There are grown children from my first two marriages, and grandchildren. I'm writing and playing music and trying to figure out who I am, what I really think, if anything, and if it matters. I'm pretty sure it doesn't. Watch this space."