The city of Rochester was founded on the east bank of the river Medway by the Romans. It's location at a strategic crossing point was built on by the Normans. The stone castle was erected by Gundulf, Bishop of Rochester, in the 1080s. The French monk was also responsible for building the adjacent cathedral and for the Tower of London.

The castle's Great Keep was added in the 12th century. It remains the tallest surviving fortification of its type in Europe. The cathedral is the second oldest in England, after Canterbury.