• Derek Xu's avatar
    Support using lightweight Flutter Engines to run tests (#141726) · dfb5888e
    Derek Xu authored
    This PR implements the functionality described above and hides it behind
    the `--experimental-faster-testing` flag of `flutter test`.
    
    ### The following are some performance measurements from test runs
    conducted on GitHub Actions
    
    run 1 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
    run 2 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
    run 3 logs:
    https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation` step**
    
    run 1: 54s
    run 2: 52s
    run 3: 56s
    
    average: 54s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation` step**
    
    run 1: 27s
    run 2: 27s
    run 3: 29s
    
    average: 27.67s (~48.77% shorter than 54s)
    
    **length of `flutter test --reporter=expanded test/animation
    test/foundation test/gestures test/painting test/physics test/rendering
    test/scheduler test/semantics test/services` step**
    
    run 1: 260s
    run 2: 270s
    run 3: 305s
    
    average: 278.33s
    
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/animation test/foundation test/gestures
    test/painting test/physics test/rendering test/scheduler test/semantics
    test/services` step**
    
    from a clean build (right after deleting the build folder):
    
    run 1: 215s
    run 2: 227s
    run 3: 245s
    
    average: 229s (~17.72% shorter than 278.33s)
    
    Note that in reality, `test/material` was not passed to `flutter test`
    in the trials below. All of the test files under `test/material` except
    for `test/material/icons_test.dart` were listed out individually
    
    **length of `flutter test --reporter=expanded test/material` step**
    
    run 1: 408s
    run 2: 421s
    run 3: 451s
    
    average: 426.67s
    
    **length of `flutter test --experimental-faster-testing
    --reporter=expanded test/material` step**
    
    run 1: 382s
    run 2: 373s
    run 3: 400s
    
    average: 385s (~9.77% shorter than 426.67s)
    
    ---------
    Co-authored-by: 's avatarDan Field <dnfield@google.com>
    dfb5888e
Name
Last commit
Last update
..
debug_adapter Loading commit data...
isolated Loading commit data...
test_data Loading commit data...
README.md Loading commit data...
analyze_all_templates_test.dart Loading commit data...
analyze_once_test.dart Loading commit data...
analyze_size_test.dart Loading commit data...
analyze_suggestions_integration_test.dart Loading commit data...
android_e2e_api_test.dart Loading commit data...
android_gradle_daemon_cache_test.dart Loading commit data...
android_gradle_deprecated_plugin_apply_test.dart Loading commit data...
android_gradle_flutter_source_path_test.dart Loading commit data...
android_gradle_java_version_test.dart Loading commit data...
android_gradle_outputs_app_link_settings_test.dart Loading commit data...
android_gradle_print_build_variants_test.dart Loading commit data...
android_plugin_compilesdkversion_mismatch_test.dart Loading commit data...
android_plugin_example_app_build_test.dart Loading commit data...
android_plugin_ndkversion_mismatch_test.dart Loading commit data...
android_plugin_new_output_dir_test.dart Loading commit data...
android_plugin_skip_unsupported_test.dart Loading commit data...
background_isolate_test.dart Loading commit data...
bash_entrypoint_test.dart Loading commit data...
batch_entrypoint_test.dart Loading commit data...
break_on_framework_exceptions_test.dart Loading commit data...
build_ios_config_only_test.dart Loading commit data...
build_macos_config_only_test.dart Loading commit data...
build_preview_test.dart Loading commit data...
cache_test.dart Loading commit data...
command_output_test.dart Loading commit data...
coverage_collection_test.dart Loading commit data...
daemon_mode_test.dart Loading commit data...
debugger_stepping_test.dart Loading commit data...
deferred_components_test.dart Loading commit data...
deprecated_gradle_settings_test.dart Loading commit data...
devtools_uri_test.dart Loading commit data...
downgrade_upgrade_integration_test.dart Loading commit data...
exit_code_test.dart Loading commit data...
expression_evaluation_test.dart Loading commit data...
flutter_attach_test.dart Loading commit data...
flutter_build_android_app_project_builddir_test.dart Loading commit data...
flutter_build_apk_verbose_test.dart Loading commit data...
flutter_build_config_only_test.dart Loading commit data...
flutter_build_wasm_test.dart Loading commit data...
flutter_build_windows_test.dart Loading commit data...
flutter_build_with_compilation_error_test.dart Loading commit data...
flutter_gen_test.dart Loading commit data...
flutter_run_test.dart Loading commit data...
flutter_run_with_error_test.dart Loading commit data...
forbidden_imports_test.dart Loading commit data...
gen_l10n_test.dart Loading commit data...
gradle_non_android_plugin_test.dart Loading commit data...
hot_reload_errors_test.dart Loading commit data...
hot_reload_test.dart Loading commit data...
hot_reload_with_asset_test.dart Loading commit data...
lifetime_test.dart Loading commit data...
observatory_port_test.dart Loading commit data...
overall_experience_test.dart Loading commit data...
plist_parser_test.dart Loading commit data...
shader_compiler_test.dart Loading commit data...
stateless_stateful_hot_reload_test.dart Loading commit data...
template_manifest_test.dart Loading commit data...
test_driver.dart Loading commit data...
test_test.dart Loading commit data...
test_utils.dart Loading commit data...
timeline_test.dart Loading commit data...
tool_backend_test.dart Loading commit data...
transition_test_utils.dart Loading commit data...
unit_coverage_test.dart Loading commit data...
variable_expansion_windows.dart Loading commit data...
variable_expansion_windows_test.dart Loading commit data...
vmservice_integration_test.dart Loading commit data...
web_plugin_registrant_test.dart Loading commit data...
xcode_backend_test.dart Loading commit data...