Papa John’s recently forced founder and former Chairman John Schnatter to resign over allegations he made a racial slur. Now the pizza chain is battling to keep him from clawing his way back into the company.

To do so, Papa John’s says it’s taking advantage of a corporate strategy often used to fend off hostile takeover attempts:...