The present building incorporates a Scottish style plantation house of ca 1622, built by Patrick Agnew. Between 1803-07 the present Georgian castle was built for Edward Jones Agnew by John Nash in his “romantic castle” style incorporating part of the 17th century castle in the west wing. There is a wide, round tower at one corner; and a polygonal tower at another. During the Second World War Kilwaughter Castle was occupied by soldiers preparing for D-day. After the war the castle remained unoccupied and gradually fell into ruin. The interior was dismantled in 1951, the roof and floors have since collapsed.