Sir William Forbes of Kinaldly, who was the second son of Sir John Forbes of Druminnor, became the founder of the Pitsligo family.

He obtained the lands of Pitsligo when he married Agnes Fraser of Philorth in 1423. The following year Sir William built the massive oblong towerhouse, which was surrounded by a moat. It originally had three large rooms consisting of kitchen, dining room and sleeping quarters. The top section, including battlements, was removed for safety reasons by the last laird in 1703.

The castle underwent major improvements in the 1570s by Sir Alexander the 7th Knight of Pitsligo. He witnessed the home transformed into a quadrangular keep which strengthened his fortification. This included a drum tower housing three gun loops with another four major cannon ports on the North and West wings, one on the stair tower within the forecourt. This knight died in 1587 and it was his grandson, Sir Alexander, who was raised to the peerage to become the first Lord Pitsligo on 24th June, 1633. He was responsible for the creation of Pitsligo Parish in 1630 and the building of the church in 1632. He lived long enough to see the beautiful oak family aisle installed in 1634. This is a masterpiece of Scottish Jacobean wood-carving. Lavishly ornate and thus a work of art, this has become as much a memorial to the local 17th century craftsmen as to the founder of the burgh. Bearing seven heraldic panels adorned with - the family crest; Lord Pitsligo’s initials, and the latter in monogram with that of his wife, Dame Jane Keith. The handsome Dutch carved belfry was erected in 1635 and the laird died in 1636.

Alexander, the second Lord Pitsligo, began developments at Rosehearty. It was created a burgh of barony by Royal Charter in 1681. He died in 1690 with his son, the third Laird, dying only months later. Alexander the 4th and last Lord Pitsligo, lived such an interesting life that the area is steeped in his legendary escapades as a Jacobite and Mystic.