Benjamin Huger, Jr., did not live a long life. When he died in 1823, Mary found herself a childless widow in charge of her great plantation once more. Doubting now that she would ever raise a family, she threw herself into the continued development of Prospect Hill’s gardens and the welfare of her beloved servants.

Some years later, Mary was diagnosed with a rare, incurable, lingering disease. She did not cloister herself away, but worked even harder to make sure Prospect Hill and all who lived there would prosper after her death. When her illness finally forced her to remain near her bed, she took to spending as much time as possible on the small upstairs portico near her room. From this vantage point, she could look down upon her life’s work and be seen by the slaves she was too weak to join outside.

In addition to grieving over their mistress’s illness, the servants at Prospect Hill dreaded what might become of them after she passed away. Since Mary had no children to inherit Prospect Hill, the cruel facts of slavery dictated that they would have a new master when she was dead and buried. The new owner might treat them just as kindly as "Miss Mary" always had. Or he might treat them indifferently or cruelly. Or he might divide Prospect Hill, selling the slaves to distant plantations and tearing apart their families. They might even have a master who lived elsewhere, leaving their fate in the hands of a hardhearted overseer.

Mary’s death in 1838 cast a pall over Prospect Hill, a sad silence punctuated only by the mournful spirituals sung with great feeling by her hundreds of slaves. During the nights following her death, small groups of them gathered at the boatman’s house, located between the manor house and the Waccamaw River at the head of a canal. Sitting beneath the stars on the wood-plank landing, they sang and prayed, pondering their fate. The comforting sight of the light that had long burned in Miss Mary’s bedroom was gone, leaving the dark façade of the mansion as a poignant reminder of her passing. Those who were old enough to remember back nearly two decades reminisced about the grand visit of President Monroe.

On one of these nights, a slave gazed toward the darkened mansion and, to his surprise and awestruck delight, saw the glowing figure of a lady who could be no other than the beloved Miss Mary standing on the upstairs portico.

Dressed in flowing white and illuminated by the moonlight, the figure was not frightening, for she was a familiar sight to all the slaves. Many nights during the years preceding her death, they had seen Miss Mary standing alone on her portico in her white nightclothes, gazing across the Prospect Hill gardens toward the river. To see their beloved mistress once more was a sign, the slaves knew, that she was still caring for them and that their home was secure.

Shortly thereafter, all of Prospect Hill-slaves included-was purchased by Colonel Joshua John Ward of Brookgreen Plantation. Being familiar with the integrity of the Ward dynasty, the servants knew they had no worries as to the new owner’s treatment of them. They were immensely relieved that their homes and families would remain intact.

Of course, the slaves who saw the loving ghost of Miss Mary and celebrated the Wards’ ownership of their home had no way of knowing that they and their children would be free men and women within three decades. Neither did they know that a Ward heir would come to love Prospect Hill so much that he would rather die than lose it.