It is. There's an enchantment cast that prevents me from physically leaving. Not the tower itself, but Avalon.

[Down the line he'll get his cheat code thanks to the combination of observing a sleeping goddess and an old poem that said he was the Antichrist, but he didn't know that yet.]

Although you are the first person to call it 'magic jail.'