ive pulled off the door panels before.

just brace the surrounding area and use a bit of force.
its held in place by a connector joint that sort of resembles the circular lego technic connectors. it can simply be clicked back in again later on.