This assert was overzealous. You can update a RawGestureDetector any
time before the semantics phase, it doesn't have to only be during your
child's layout.

This is good because Block has additional PointerListeners between its
GestureDetector and its Viewport.