Shawshank Redemption question (spoilers)

Caught some of this last night, and something stood out to me that I don't think I ever noticed before:

At the end, once Red is released he goes to the spot where Andy told him to go with the black rock. There he finds the letter and cash that Andy left for him. But the letter says something like "You remember the name of the town I told you about, don't you?" and Red says "Zihuatanejo."

I sat there last night and thought: But what if he didn't remember it? Would he have just worked at the grocery store and stayed in that little town the rest of his life, and Andy would've just assumed that Red either never got out or didn't go find the letter he had left for him?

And I get why he maybe wouldn't have said the name in the letter, just in case it was found by the authorities and they could put some pieces together. But I just laughed at the thought that Andy went through all that trouble to bring his friend along, and Red went through all the trouble to carry out Andy's request, only to not be able to meet up because Red didn't remember the name of some random town in Mexico that Andy told him about several years ago. And I don't know exactly how much time has passed between that talk they had and Red getting released, and then from his release to when he finds the letter. But I assume it was at least several years.