Rotating the stolen key is the fix. It is also what erases the proof anyone used it.
The first flaw supplies the authentication the second one requires. A pair of medium problems is not a medium problem.