Reverts flutter/flutter#137381
Initiated by: Piinks
This change reverts the following previous change:
Original Description:
This updates the implementation to use the stopwatch from the Clock object and piping it through to the TestWidgetsFlutterBinding so it will be kept in sync with FakeAsync.

Relands #132291
Fixes https://github.com/flutter/flutter/issues/97761

The change was reverted due to flakiness it introduced in tests that use fling gestures.
* https://github.com/flutter/flutter/issues/135728