• Derek Xu's avatar
    Support using lightweight Flutter Engines to run tests (#141726) · dfb5888e
    Derek Xu authored
    This PR implements the functionality described above and hides it behind
    the `--experimental-faster-testing` flag of `flutter test`.
    
    ### The following are some performance measurements from test runs
    conducted on GitHub Actions
    
    run 1 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
    run 2 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
    run 3 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation` step**
    
    run 1: 54s
    run 2: 52s
    run 3: 56s
    
    average: 54s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation` step**
    
    run 1: 27s
    run 2: 27s
    run 3: 29s
    
    average: 27.67s (~48.77% shorter than 54s)
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation test/gestures test/painting test/physics test/rendering
    test/scheduler test/semantics test/services` step**
    
    run 1: 260s
    run 2: 270s
    run 3: 305s
    
    average: 278.33s
    
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation test/gestures
    test/painting test/physics test/rendering test/scheduler test/semantics
    test/services` step**
    
    from a clean build (right after deleting the build folder):
    
    run 1: 215s
    run 2: 227s
    run 3: 245s
    
    average: 229s (~17.72% shorter than 278.33s)
    
    Note that in reality, `test/material` was not passed to `flutter test`
    in the trials below. All of the test files under `test/material` except
    for `test/material/icons_test.dart` were listed out individually
    
    **length of `flutter test --reporter=expanded test/material` step**
    
    run 1: 408s
    run 2: 421s
    run 3: 451s
    
    average: 426.67s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/material` step**
    
    run 1: 382s
    run 2: 373s
    run 3: 400s
    
    average: 385s (~9.77% shorter than 426.67s)
    
    ---------
    Co-authored-by: 's avatarDan Field <dnfield@google.com>
    dfb5888e
Name
Last commit
Last update
..
android Loading commit data...
base Loading commit data...
build_system Loading commit data...
commands Loading commit data...
custom_devices Loading commit data...
dart Loading commit data...
debug_adapters Loading commit data...
drive Loading commit data...
fuchsia Loading commit data...
intellij Loading commit data...
ios Loading commit data...
isolated Loading commit data...
linux Loading commit data...
localizations Loading commit data...
macos Loading commit data...
migrations Loading commit data...
proxied_devices Loading commit data...
reporting Loading commit data...
runner Loading commit data...
test Loading commit data...
tester Loading commit data...
vscode Loading commit data...
web Loading commit data...
windows Loading commit data...
application_package.dart Loading commit data...
artifacts.dart Loading commit data...
asset.dart Loading commit data...
build_info.dart Loading commit data...
bundle.dart Loading commit data...
bundle_builder.dart Loading commit data...
cache.dart Loading commit data...
cmake.dart Loading commit data...
cmake_project.dart Loading commit data...
compile.dart Loading commit data...
context_runner.dart Loading commit data...
convert.dart Loading commit data...
daemon.dart Loading commit data...
dart_pub_json_formatter.dart Loading commit data...
desktop_device.dart Loading commit data...
devfs.dart Loading commit data...
device.dart Loading commit data...
device_port_forwarder.dart Loading commit data...
devtools_launcher.dart Loading commit data...
doctor.dart Loading commit data...
doctor_validator.dart Loading commit data...
emulator.dart Loading commit data...
features.dart Loading commit data...
flutter_application_package.dart Loading commit data...
flutter_cache.dart Loading commit data...
flutter_device_manager.dart Loading commit data...
flutter_features.dart Loading commit data...
flutter_manifest.dart Loading commit data...
flutter_plugins.dart Loading commit data...
flutter_project_metadata.dart Loading commit data...
globals.dart Loading commit data...
html_utils.dart Loading commit data...
http_host_validator.dart Loading commit data...
license_collector.dart Loading commit data...
mdns_discovery.dart Loading commit data...
native_assets.dart Loading commit data...
persistent_tool_state.dart Loading commit data...
platform_plugins.dart Loading commit data...
plugins.dart Loading commit data...
pre_run_validator.dart Loading commit data...
preview_device.dart Loading commit data...
project.dart Loading commit data...
project_validator.dart Loading commit data...
project_validator_result.dart Loading commit data...
protocol_discovery.dart Loading commit data...
proxy_validator.dart Loading commit data...
resident_devtools_handler.dart Loading commit data...
resident_runner.dart Loading commit data...
run_cold.dart Loading commit data...
run_hot.dart Loading commit data...
sksl_writer.dart Loading commit data...
template.dart Loading commit data...
tracing.dart Loading commit data...
update_packages_pins.dart Loading commit data...
version.dart Loading commit data...
vmservice.dart Loading commit data...
xcode_project.dart Loading commit data...