Oh, for the GBA, the rubber contacts are pushed in to the point that they are cut. You can't notice it on pics but basically, they don't spring pack or make good contact. You have to push down hard.
I tried using rubber glue but that was not good. Only a mild improvement.