In certain cases, the test would fail before creating the (lazily created) compiler object, and then we'd
try to call shutdown() on null in those cases.

Fixes #18610