# Macrobenchmarks Performance benchmarks use either flutter drive or the web benchmark harness. ## Mobile benchmarks ### Cull opacity benchmark To run the cull opacity benchmark on a device: ``` flutter drive --profile test_driver/cull_opacity_perf.dart ``` Results should be in the file `build/cull_opacity_perf.timeline_summary.json`. More detailed logs should be in `build/cull_opacity_perf.timeline.json`. ### Cubic bezier benchmark To run the cubic-bezier benchmark on a device: ``` flutter drive --profile test_driver/cubic_bezier_perf.dart ``` Results should be in the file `build/cubic_bezier_perf.timeline_summary.json`. More detailed logs should be in `build/cubic_bezier_perf.timeline.json`. ### Backdrop filter benchmark To run the backdrop filter benchmark on a device: To run a mobile benchmark on a device: ```bash flutter drive --profile -t test_driver/run_app.dart --driver test_driver/[test_name]_test.dart ``` Results should be in the file `build/[test_name].timeline_summary.json`. More detailed logs should be in `build/[test_name].timeline.json`. The key `[test_name]` can be: - `animated_placeholder_perf` - `backdrop_filter_perf` - `color_filter_and_fade_perf` - `cubic_bezier_perf` - `cull_opacity_perf` - `fading_child_animation_perf` - `imagefiltered_transform_animation_perf` - `multi_widget_construction_perf` - `picture_cache_perf` - `post_backdrop_filter_perf` - `simple_animation_perf` - `textfield_perf` - `fullscreen_textfield_perf` ### E2E benchmarks (On-going work) [E2E](https://pub.dev/packages/e2e)-based tests are driven independent of the host machine. The following tests are E2E: - `cull_opacity_perf.dart` - `multi_widget_construction_perf` These tests should be run by: ```bash flutter drive --profile -t test/[test_name]_e2e.dart --driver test_driver/e2e_test.dart ``` ## Web benchmarks Web benchmarks are compiled from the same entry point in `lib/web_benchmarks.dart`. ### How to write a web benchmark Create a new file for your benchmark under `lib/src/web`. See `bench_draw_rect.dart` as an example. Choose one of the two benchmark types: - A "raw benchmark" records performance metrics from direct interactions with `dart:ui` with no framework. This kind of benchmark is good for benchmarking low-level engine primitives, such as layer, picture, and semantics performance. - A "widget benchmark" records performance metrics using a widget. This kind of benchmark is good for measuring the performance of widgets, often together with engine work that widget-under-test incurs. - A "widget build benchmark" records the cost of building a widget from nothing. This is different from the "widget benchmark" because typically the latter only performs incremental UI updates, such as an animation. In contrast, this benchmark pumps an empty frame to clear all previously built widgets and rebuilds them from scratch. For a raw benchmark extend `RawRecorder` (tip: you can start by copying `bench_draw_rect.dart`). For a widget benchmark extend `WidgetRecorder` (tip: you can start by copying `bench_simple_lazy_text_scroll.dart`). For a widget build benchmark extend `WidgetBuildRecorder` (tip: you can start by copying `bench_build_material_checkbox.dart`). Pick a unique benchmark name and class name and add it to the `benchmarks` list in `lib/web_benchmarks.dart`. ### How to run a web benchmark Web benchmarks can be run using `flutter run` in debug, profile, and release modes, using either the HTML or the CanvasKit rendering backend. Note, however, that running in debug mode will result in worse numbers. Profile mode is useful for profiling in Chrome DevTools because the numbers are close to release mode and the profile contains unobfuscated names. Example: ``` cd dev/benchmarks/macrobenchmarks # Runs in profile mode using the HTML renderer flutter run --web-renderer=html --profile -d web-server lib/web_benchmarks.dart # Runs in profile mode using the CanvasKit renderer flutter run --web-renderer=canvaskit --profile -d web-server lib/web_benchmarks.dart ``` You can also run all benchmarks exactly as the devicelab runs them: ``` cd dev/devicelab # Runs using the HTML renderer ../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_html.dart # Runs using the CanvasKit renderer ../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_canvaskit.dart ``` ## Frame policy test File `test/frame_policy.dart` and its driving script `test_driver/frame_policy_test.dart` are used for testing [`fullyLive`](https://api.flutter.dev/flutter/flutter_test/LiveTestWidgetsFlutterBindingFramePolicy-class.html) and [`benchmarkLive`](https://api.flutter.dev/flutter/flutter_test/LiveTestWidgetsFlutterBindingFramePolicy-class.html) policies in terms of its effect on [`WidgetTester.handlePointerEventRecord`](https://master-api.flutter.dev/flutter/flutter_test/WidgetTester/handlePointerEventRecord.html).