VIDEO: Bernie Madoff And Wife Tried To Commit Suicide

For the past three years, Ponzi schemer Bernie Madoff, 73, has been one of the most hated men in Amercia, after having conned over $64 billion dollars from individuals, families, charities and hedge funds.

"I feel safter here than outside," Madoff told Barbara Walters in an interview from within the North Carolina federal prison where he is serving his 150-year sentence, the maximum allowed for his crime. "I know that I will die in prison," he said, according to ABC News. "I lived the last 20 years of my life in fear. Now I have no fear — nothing to think about because I'm no longer in control of my own life."

Prior to the relative serenity that prison life has afforded, Madoff and his wife, Ruth, found themselves resorting to suicide. Ruth revealed that the couple made a pact to commit suicide together in December of 2008, after the case broke and Madoff's crimes went public. "I don't know whose idea it was, but we decided to kill ourselves because it was so horrendous what was happening," Ruth told 60 Minutes' Morley Safer, reports DigitalJournal.com. "We had terrible phone calls. Hate mail, just beyond anything, and I said, '… I just can't go on anymore.'"

The Madoffs collected some family heirlooms to send to their son, Andrew, and then took a bunch of Ambien pills. "I took what we had, he took more," Ruth said, but they avoided alcohol to decrease the risk of throwing up the drugs. Despite their efforts, both of them "woke up the next day." Years later, Ruth is glad the attempt didn't work. "It was very impulsive and I am glad we woke up," she said.

Ironically, two years later the Madoff's eldest son, Mark, committed suicide by hanging himself in his New York City apartment on the anniversary of his father's arrest.

All this time later, it is the loss of his family that still affects Madoff. "Not seeing my family and knowing they hate me" is the worst thing about where his life has wound up, he told Walters. "I betrayed them …. I am sorry to have caused [my grandchildren] pain." It's equally hard for Madoff to live with the absence of Ruth, who never visits him any more. After Mark's death she "asked me to let her go, which I understood," Madoff said. But his wife, who is reportedly living in seclusion in Florida, "doesn't hate me," he assures. "She has no one. It's not fair to her. She lost her first son …. She is a devoted wife and didn't care about the money."

As for the people whose lives were ruined by his schemes? "I understand why clients hate me. The gravy train is over. I can live with that," he said.

Watch Walters talk about the interview here: