Ashby Castle forms the backdrop to the famous jousting scenes in Sir Walter Scott's classic novel of 1819, Ivanhoe. Now a ruin, the castle began as a manor house in the twelfth century. It only achieved castle status in the fifteenth century, by which time the hall and buttery had been enlarged, with a solar to the east and a large integral kitchen added to the west.

Between 1474 and his execution by Richard III in 1483, Edward IV's Chamberlain, Lord Hastings, added the chapel and the impressive keep-like Hastings Tower - a castle within a castle. Visitors can now climb the 24 metre (78 feet) tower, which offers fine views. Later, the castle hosted many royal visitors, including Henry VII, Mary, Queen of Scots, James I and Charles I.

A Royalist stronghold during the Civil War, the castle finally fell to Parliament in 1646, and was then made unusable. An underground passage from the kitchen to the tower, probably created during this war, can still be explored today. In 2006 archaeologists investigated the mysterious castle garden, famous for its elaborately shaped sunken features.