• Derek Xu's avatar
    Support using lightweight Flutter Engines to run tests (#141726) · dfb5888e
    Derek Xu authored
    This PR implements the functionality described above and hides it behind
    the `--experimental-faster-testing` flag of `flutter test`.
    
    ### The following are some performance measurements from test runs
    conducted on GitHub Actions
    
    run 1 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
    run 2 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
    run 3 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation` step**
    
    run 1: 54s
    run 2: 52s
    run 3: 56s
    
    average: 54s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation` step**
    
    run 1: 27s
    run 2: 27s
    run 3: 29s
    
    average: 27.67s (~48.77% shorter than 54s)
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation test/gestures test/painting test/physics test/rendering
    test/scheduler test/semantics test/services` step**
    
    run 1: 260s
    run 2: 270s
    run 3: 305s
    
    average: 278.33s
    
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation test/gestures
    test/painting test/physics test/rendering test/scheduler test/semantics
    test/services` step**
    
    from a clean build (right after deleting the build folder):
    
    run 1: 215s
    run 2: 227s
    run 3: 245s
    
    average: 229s (~17.72% shorter than 278.33s)
    
    Note that in reality, `test/material` was not passed to `flutter test`
    in the trials below. All of the test files under `test/material` except
    for `test/material/icons_test.dart` were listed out individually
    
    **length of `flutter test --reporter=expanded test/material` step**
    
    run 1: 408s
    run 2: 421s
    run 3: 451s
    
    average: 426.67s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/material` step**
    
    run 1: 382s
    run 2: 373s
    run 3: 400s
    
    average: 385s (~9.77% shorter than 426.67s)
    
    ---------
    Co-authored-by: 's avatarDan Field <dnfield@google.com>
    dfb5888e
Name
Last commit
Last update
..
analyze_continuously_test.dart Loading commit data...
analyze_suggestion_test.dart Loading commit data...
analyze_test.dart Loading commit data...
android_analyze_test.dart Loading commit data...
assemble_test.dart Loading commit data...
attach_test.dart Loading commit data...
build_aar_test.dart Loading commit data...
build_darwin_framework_test.dart Loading commit data...
build_ios_test.dart Loading commit data...
build_ipa_test.dart Loading commit data...
build_linux_test.dart Loading commit data...
build_macos_test.dart Loading commit data...
build_test.dart Loading commit data...
build_web_test.dart Loading commit data...
build_windows_test.dart Loading commit data...
clean_test.dart Loading commit data...
config_test.dart Loading commit data...
create_usage_test.dart Loading commit data...
custom_devices_test.dart Loading commit data...
daemon_test.dart Loading commit data...
devices_test.dart Loading commit data...
doctor_test.dart Loading commit data...
downgrade_test.dart Loading commit data...
drive_test.dart Loading commit data...
flutter_web_platform_test.dart Loading commit data...
generate_localizations_test.dart Loading commit data...
http_host_validator_test.dart Loading commit data...
ide_config_test.dart Loading commit data...
install_test.dart Loading commit data...
ios_analyze_test.dart Loading commit data...
logs_test.dart Loading commit data...
precache_test.dart Loading commit data...
proxied_devices_test.dart Loading commit data...
proxy_validator_test.dart Loading commit data...
pub_test.dart Loading commit data...
run_test.dart Loading commit data...
screenshot_command_test.dart Loading commit data...
shell_completion_test.dart Loading commit data...
symbolize_test.dart Loading commit data...
test_test.dart Loading commit data...
update_packages_test.dart Loading commit data...
upgrade_test.dart Loading commit data...