The prison was the only part of the palace that went below the surface of the shell, and it's been sealed off, so the rest of the Palace could be taken down without damage to Tu Vishan. That said, if demolishing is the route that's decided on, the NPCs are going to probably insist on something a little more constrained than "giant explosion levelling the place" since a lot of the marble can be reused, and it's pretty rare.