Castles have a simple job: to serve as obvious, strong defenses, protecting valuable assets. Giant stone purses, castles also naturally became centers of commerce, meeting places for merchants and decision makers—places of power and wealth.
The problem is that a castle is not subtle; a castle is a giant marker saying, “Here’s where the good stuff is!” The defenders have to be constantly vigilant, and attacks can come from anywhere and at any time. You can’t just move your castle to a new location after it’s been attacked a few times.
The defenders have to be successful every single time. One failure on their part means the castle falls. Attackers, on the other hand, can try as many times as possible to get in; they just need to be successful once.