When William Backhouse Astor Sr. died, he disowned his third son, Henry, who had committed the unpardonable sin of marrying a woman who was his social inferior and had refused to give up her dower rights to inherit from her husband. Instead, after dividing the bulk of his estate between William Jr. and John Jacob III, he left about $5 million in property, including the Catskills estate of Rokeby, to his granddaughter Maddie Chanler, the daughter of his first child, Emily. Emily had died while giving birth, and the Astors persuaded her former husband, Samuel Ward, to give up parental rights after he remarried. After a strict childhood, Maddie married John Chanler, a rising and wealthy politician, in 1861. They had the run of Rokeby, named by Emily’s mother for a romantic poem by Sir Walter Scott. Their marriage was happy, except for the premature death of one daughter, but Maddie died in 1875 at the young age of 37, just two weeks after her grandfather’s funeral. She was pregnant with her 12th child at the time.

John Chanler summoned his cousin, Mary Marshall, to help raise the children. He sent the oldest boys, Archie and Wintie (ages 13 and 12), to England for school and the eldest daughter, Bess (age 9), to a ladies’ finishing school.