George IV began his affair with her in 1782. She would also become romantically involved with members of the English aristocracy, including
Frederick Howard, 5th Earl of Carlisle, among others. It was not until 1794 that she lured the Prince of Wales (George IV) away from his illegal wife,
Maria Fitzherbert, although he would continue to be romantically involved with Maria until 1811.
Having encouraged the Prince of Wales to marry his
first cousin, Princess
Caroline of Brunswick, Lady Jersey nevertheless set out to make Caroline's life difficult. However, the now
Queen consort (Caroline) had very little regard for George IV, nor did he care for her deeply, and after the birth of their child together, Caroline lived abroad for most of her 25 year marriage to him, taking other lovers, and therefore leaving a void Frances could fill.
Since she enjoyed the favour of
Queen Charlotte, even the displeasure of the
king was not enough to threaten Lady Jersey's position, and she continued to run the prince's life and household for some time. Eventually, though, he replaced her and she would slowly decline from any active involvement with the royal court.
She had remained married to George Villiers throughout. In about 1803, her previously undisputed place as senior royal mistress was taken by Lady Hertford. In
1805 her husband died, after a long marriage which had produced ten children.
Lady Jersey died at Cheltenham and was buried at Middleton Stoney.