• Derek Xu's avatar
    Support using lightweight Flutter Engines to run tests (#141726) · dfb5888e
    Derek Xu authored
    This PR implements the functionality described above and hides it behind
    the `--experimental-faster-testing` flag of `flutter test`.
    
    ### The following are some performance measurements from test runs
    conducted on GitHub Actions
    
    run 1 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
    run 2 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
    run 3 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation` step**
    
    run 1: 54s
    run 2: 52s
    run 3: 56s
    
    average: 54s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation` step**
    
    run 1: 27s
    run 2: 27s
    run 3: 29s
    
    average: 27.67s (~48.77% shorter than 54s)
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation test/gestures test/painting test/physics test/rendering
    test/scheduler test/semantics test/services` step**
    
    run 1: 260s
    run 2: 270s
    run 3: 305s
    
    average: 278.33s
    
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation test/gestures
    test/painting test/physics test/rendering test/scheduler test/semantics
    test/services` step**
    
    from a clean build (right after deleting the build folder):
    
    run 1: 215s
    run 2: 227s
    run 3: 245s
    
    average: 229s (~17.72% shorter than 278.33s)
    
    Note that in reality, `test/material` was not passed to `flutter test`
    in the trials below. All of the test files under `test/material` except
    for `test/material/icons_test.dart` were listed out individually
    
    **length of `flutter test --reporter=expanded test/material` step**
    
    run 1: 408s
    run 2: 421s
    run 3: 451s
    
    average: 426.67s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/material` step**
    
    run 1: 382s
    run 2: 373s
    run 3: 400s
    
    average: 385s (~9.77% shorter than 426.67s)
    
    ---------
    Co-authored-by: 's avatarDan Field <dnfield@google.com>
    dfb5888e
runner.dart 28.6 KB