The Hannay family first came to Kirkdale in 1532 from Sorbie in Wigtownshire in the west of Dumfries and Galloway. In the 17th century, members of the family were involved with the covenanters, sat in Parliament and at one time Kirkdale was used as a staging post for Irish mercenaries going to Sweden.

Kirkdale House near Gatehouse of Fleet, Scotland
The present Kirkdale House was built by Sir Samuel Hannay from local granite to plans by Robert Adam between 1780 and 1787. Sir Samuel became wealthy as a manufacturer of chemicals and drugs in London where he was at one time M.P. for Westminister. He made several fortunes and lost them gambling, dying in considerable debt.
Despite a fire which destroyed the center of the house in 1893, the outside remains one of the few unaltered original Adam exteriors in Scotland.
Some Gordon Connections to the Hannays
Although my Gordons don't seem to be related to the Hannays, I have noted three marriages between the Hannays of Kirkdale and members of the Gordons family.
Coincidentally, James Gordon my 4th great-grandfather, leased the Claughread farm on the Kirkdale Estate from about 1806 to 1848 until he retired.
Gordon - Hannay Marriages