Start the walk at the beautiful castle, once built by the Portuguese. Walk along the walls alongside sea. You will see a building on a small island, built as a prison. Sea on this part of the coast is windy therefore you get wet by the wildy waves of the sea as it hits the walls. Notice the towers at the end of the seawalls and go down to the streets of Essaouria to discover its medieval architecture. There will be Turkish baths, synagogues shops selling spices and riads on hidden streets. You will notice the European character of this cute town as well. There is a little possibility that you see hustlers here. The atmosphere is more relaxed that the rest of Morocco. There are coffee houses around as well as European style patisseries.
For lunch go back to the harbour and sit a while watching seagulls following the boats and you can have fish at one of the places at the harbour. Once more you will notice that the shops here are much different than in Agadir and you will be treated more as a person than cash-machine and that is a good feeling while traveling.