Along the way, he's starred in more than 60 movies and two enormously successful TV shows: It Takes a Thief and Hart to Hart. You can still see him on TV on the Charlie Sheen sitcom, Two and a Half Men.
Wagner's memoir, Pieces of My Heart, is published this week. For the first time, Wagner discusses his four-year relationship with Barbara Stanwyck in the early 1950s, and his two marriages to Natalie Wood, as well as her tragic death in 1981.
Also featured in the book are Wagner's intimate friendships with Spencer Tracy, David Niven, Fred Astaire, Laurence Olivier, Elia Kazan and Peter Sellers, to mention only a few.
An interview with the man his friends and colleagues know as RJ:
Would it be fair to say that the three primary women in your life would be Barbara Stanwyck, Natalie Wood and Jill St. John?
Add one. My mother.
What did each of them give you?
My mother gave me unconditional love. She was constantly attentive to me; there was never any negative thing from my mother, ever. She was a very positive person.
Barbara gave me love and added a different direction to my life. What she gave me had nothing to do with acting or the movie business; it was a broader outlook that went beyond the movie business. She read everything, she had a very fine appreciation of art and music and life; I still have a couple of paintings she gave me, for instance. And to have a woman of her accomplishment and stature be with you was a tremendous validation.
Natalie ... Natalie was just an incredible force in my life. We were young together, discovering our places in show business together and discovering each other, all at the same time. It's rare to be in perfect sync with another human being, and that's what we had. But we were also very young, and it ended, and then years later we discovered each other again at a totally different stage of life. That time, it was even better because it was so unexpected. Because we were older and more experienced it was deeper and truer. I never took it for granted.
Jill came into my life after Natalie died and held me up. I was ripped wide open and she healed me in every way. I don't believe I would be alive today if it wasn't for her.
Why write a book now? Why not 10 years ago, or even 20?
Many people have talked to me about doing a book over the years. God knows, I've met so many fascinating people, and I think I have some insights into their lives. And I always felt I had to write something to acknowledge the people who have been so responsible for my career and my life. And those who helped put me back on my feet after Natalie died.
I felt it was right for me now. I had a hunch that if I didn't do it now, I wouldn't do it at all. It's a hard thing to do, a hard thing to address.
What was the hardest thing you had to do in the course of the book?
To be sure that my memories were correct. And to go back and relive some of those times was very difficult. I'm not speaking specifically of the tragic parts of my life, it's just difficult to go back and think about those times. So many of the people are gone. And to tell the truth, and at the same time, to try to be respectful of the people that were involved, is not always easy.
You have three daughters (Katie, Natasha and Courtney) and two of the three went into show business - was this OK with you?
I never tried to talk them out of it. I wanted Katie and Natasha to be little girls and have a normal life before they started becoming actresses. I wanted them to have the experience of being children. I didn't want them to be something different from their peers. Natalie felt the same way, and she felt it very strongly; she had been a child actress and knew what it cost. We had a lot of requests for them to get into the business, and they were interested in it, they'd been around us and the business seems like it's fun and glamorous. But we resisted that, because we definitely wanted them to have the experience and the joy of being little girls.
What did you learn about acting from making two films with Spencer Tracy?
Everything. Everything. On our first picture, Broken Lance, I played his son, and one day during a take he broke character and said he couldn't hear me. I said, "Boy that's something when I underplay you."
A few hours later it was lunch, and he called me into his dressing room and proceeded to ream me out. "You don't imagine you can underplay me, do you? What are you even thinking about things like that for? Are you thinking at all? You shouldn't be thinking about any of that, you should be thinking about playing the scene. Not about whether you're underplaying, overplaying or anything else. Concentrate on what you're going to do. The scene, that one moment, and nothing else!
"Don't learn the tricks of the trade. Learn the trade."
He really beat me up, but later he came up and put his arm around me and said, "How are you doing now?" and at that moment I realized he liked me. We became very close for the rest of the picture, as well as on The Mountain - where he requested me as his costar - and for the rest of his life. I've been on the board of the John Tracy Clinic for years, which was Spence's great passion, and I'm very proud to be connected with it.
Was there a project you turned down you wish you'd done; conversely, what's the part that got away?
I turned down Westworld. Michael Crichton wanted me to be in the movie, and I read the script and I didn't react to it at all. And it was a tremendous success. It was a mistake on my part. It was a big hit, and I would have made a lot of money.
As for the one that got away, there was some talk at one time of my doing Butch Cassidy and the Sundance Kid with Paul Newman, but that went away. I can't complain about that, because Robert Redford was sensational in the part.
The one that got away was Rosemary's Baby. Bob Evans cast me in it, but Universal wouldn't let me postpone shooting on It Takes a Thief to do the movie. I tried everything to make it work, but couldn't. Without false modesty, I think I would have been more effective than John Cassavetes, because one look at him and you know he's a minion of Satan. And I would have looked more natural with Mia (Farrow) as well.
You still work a fair amount - the Austin Powers movies, 'Two and a Half Men.'
Yeah, but the days when I pull the train are over, and I'm fine with that. When I started out in the business, the young leading men were Chuck Heston at Paramount, Jeff Hunter and Cameron Mitchell at Fox, and John Erickson at MGM, so I'm very happy to be still in the game and part of an ensemble. It's such a pleasure to work on a show like Two and a Half Men. The writing is there, the actors are there, the crew is there, and it's a smoothly running machine. You've sent a fair amount of time in Florida over the years - you shot 'Beneath the 12 Mile Reef' in Tarpon Springs and so forth.
Not just that. Natalie and I came to Stuart for our first honeymoon, where I thought it would be fun to go fishing. Well, Stuart was totally unromantic so we left very quickly, after a couple of nights. We got on a train and went to the Sherry-Netherland in New York. No fish, but a better atmosphere.
Show business has changed so radically since you got into it 60 years ago - what advice do you give young actors?
Simply that, if and when they are rejected, which they will be, it's not them. It's the producer or the director or whoever has a different concept of that character or that project. What they tell you is "'You're too short," or "too tall" or "too young" or "too old." You have to have the foundation to handle the rejections. I have literally seen actors be destroyed by constant rejection; they absorb the negativity and it changes their personalities. It takes away from you, and all any of us have is ourselves.
The many loves of RJ Wagner
From Debbie Reynolds to Anita Ekberg and Joan Collins, Wagner romanced a lot of Hollywood stars. His memories, from Pieces of My Heart:
JOAN CRAWFORD: 'She was a dynamic lover, both domineering - which you might expect - and yielding - which you might not. All in all, a memorable one-night stand.'
YVONNE DE CARLO: 'One thing led to another, and we went back to her house. Three days later, I staggered out, depleted and disheveled.'
BARBARA STANWYCK: Wagner had a four-year relationship with her. She was 45. He was 22. 'To have a woman of her beauty and accomplishment see value in me and give herself totally to me couldn't help but have a powerful impact on my psyche. Barbara was the first savior in my life.'
ELIZABETH TAYLOR: 'Some beautiful women are passive in the bedroom ... Elizabeth was not one of those women.'
When Natalie died:
'Everything went away from me'
In Pieces of My Heart, Wagner recalls the night of Wood's drowning death off Catalina Island, after she apparently slipped off their boat. Wagner says they all drank but were only tipsy, that he got into an argument with her current costar Christopher Walken and that she left to go to bed. He saw her briefly on the boat once more and never saw her again.
He recalls when he got the news from harbor master Doug Bombard:
" 'Where is she?' I asked him.
"Doug looked at me. 'She's dead, RJ.'
"My knees went out; everything went away from me. ... So many of the best times of my life had been spent in and around Catalina Island. It was always one of my favorite places on earth.
"From the day Natalie died to this, I have never gone back."