"Charm of courage" which is sold in "Ivory Tower" should prevent you loosing exp during siege, but it doesn't.
I am not sure if "Lucky Charm" obtained via priest/priestess Dawn/Dusk, so if it is not a problem to check it as well while fixing the one above.
Thanks
Peace