Yea it's different for each card. For example, I have a United Mileage Plus Club Card that gets me into the United Club whenever I travel. The fee is $450 so I have been spending a fair amount on it in a preemptive move to try not to pay the fee. If they won't waive the fee I will cancel the card.
Usually if you threaten to cancel they will give you a statement credit or something of that nature.