Towards the end of the Roman Empire there were a lot of slave uprisings and a lot of religious sects also springing up here and there (Christianity being just one of them).
There could be all sorts of reasons why you could run afoul of the authorities. It didn't take much to get into trouble back then! I think Christianity became the official religion of Rome about 320 A.D. or so. I do know once they decided to embrace it, they clamped down pretty hard on any kind of 'heresy' which was anything they didn't agree with basically. So all that business about 'witchery' and 'special powers' might not be as far fetched as you think. The Roman Empire had people in it from all over the known world, and with them came all kinds of religious practices. So, it's possible you came from some country, like say England, that had certain 'paganistic' practices that would have been considered magical and/or heretical.
Christianity and the Roman Empire
Have you ever tried doing a regression to see if you can get more info?