We were stacking up two dismiss timers. Now we don't start a new timer
if we've already got one. Also, fix an animation leak caught by the
test.

Fixes #4640