The gardens were laid out in the 18th century by Sir Thomas Wentworth (1672 – 1739), 1st Earl of Strafford (second creation) and his son William, 2nd Earl (1722 – 1791), initially as a result of a bitter feud with another branch of the Wentworth family.

Thomas Wentworth expected to inherit the landed estate and vast wealth at Wentworth Woodhouse, some 7 miles to the south of Stainborough, when the 2nd Earl of Strafford (first creation) died in 1695. Instead Wentworth Woodhouse to was left to Thomas' cousin. Bitter and resentful, Thomas secretly bought the neighbouring Stainborough estate in 1708 and began a major building programme.

Thomas Wentworth rose to high positions as a soldier and diplomat in the service of King William III and Queen Anne, and in 1708 was made the 1st Earl of Strafford of the second creation. He became determined to create an estate suitable for a man of his importance.

Thomas would have been familiar with the Baroque gardens being laid out for European royalty at that time and copied by members of the aristocracy. Between 1709 – 1715 a new Baroque wing was added to the existing house, then known as Stainborough Hall, and formal gardens were laid out to the south, west and east of the new wing. In 1727 Thomas began to build the mock castle on the highest point of the estate, and in 1731 renamed the house and estate as Wentworth Castle.

Thomas was succeeded by his son William in 1739, by which time the fashion was moving towards laying out gardens in a more natural style. During the 19th century many exotic trees and shrubs, especially hardy hybrid rhododendrons, were planted and a conservatory and rockery built. In the 20th century the Gardens fell into neglect until clearance work in the 1970s and part restoration of Stainborough Castle in the 1980s and the establishment of the first of three National Plant Collections in the Gardens.