Do I have to go to the shop (or authorized Orange Money agent) I registered into to perform Orange Money transactions (money deposit or cash withdrawal)?

No. Once your Orange Money account is open and active, you can perform transactions at any Orange shop or authorised Orange Money agent at your convenience (money deposit or cash withdrawal). In addition, you can perform your transactions 24/7 on your own for money transfer, airtime purchase, bill payment...