Dating back to the early 1200s, Skipness Castle is one of the oldest surviving castles in Scotland, although it was re-modelled several times over subsequent centuries. It is one of three fortifications built by the Clan MacSween who originated from Norway. Skipness was their secondary residence, after Castle Sween. They also had a hall-house castle across the Kilbrannan Sound at Lochranza on the Isle of Arran. 

When Norwegian dominance ended after Hakon IV was defeated by the Scots at the Battle of Largs in 1263, control of Skipness passed to the MacDonalds. They reinforced the castle and also built the nearby chapel. Following their own downfall, the castle came under the control of the Campbell earls of Argyll in 1502.