Thanks to everyone for the help and advice - I managed to get the battery off and will go down the route of tape rather than solder for the replacement!
I've tried to slide the battery out but it will not budge (it won't even turn around in the holder). I haven't tried forcing it too much as I'm nervous of breaking something. I've tried looking underneath the battery and it looks as if it is soldered but it's difficult to tell. Thanks again.
One last question, has anyone got any tips on fitting these batteries? It looks like it's soldered on and I'm not sure of the best way of removing it etc. Thanks