"He went peacefully at home with his wife of 34 years, sister Joan, and daughters Lydia Hearst and Gillian Hearst-Simonds, along with [her] husband Christian Simonds by his side," a rep told People.