README.md 4.84 KB
Newer Older
1 2
# Macrobenchmarks

3
Performance benchmarks use either flutter drive or the web benchmark harness.
4

5 6
## Mobile benchmarks

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
### Cull opacity benchmark

To run the cull opacity benchmark on a device:

```
flutter drive --profile test_driver/cull_opacity_perf.dart
```

Results should be in the file `build/cull_opacity_perf.timeline_summary.json`.

More detailed logs should be in `build/cull_opacity_perf.timeline.json`.

### Cubic bezier benchmark

To run the cubic-bezier benchmark on a device:

```
flutter drive --profile test_driver/cubic_bezier_perf.dart
```

Results should be in the file `build/cubic_bezier_perf.timeline_summary.json`.

More detailed logs should be in `build/cubic_bezier_perf.timeline.json`.

### Backdrop filter benchmark

To run the backdrop filter benchmark on a device:
34
To run a mobile benchmark on a device:
35

36
```bash
37
flutter drive --profile -t test_driver/run_app.dart --driver test_driver/[test_name]_test.dart
38 39
```

40
Results should be in the file `build/[test_name].timeline_summary.json`.
41

42
More detailed logs should be in `build/[test_name].timeline.json`.
43

44
The key `[test_name]` can be:
45

46 47
- `animated_placeholder_perf`
- `backdrop_filter_perf`
48 49
- `color_filter_and_fade_perf`
- `cubic_bezier_perf`
50 51 52 53 54 55 56 57
- `cull_opacity_perf`
- `fading_child_animation_perf`
- `imagefiltered_transform_animation_perf`
- `multi_widget_construction_perf`
- `picture_cache_perf`
- `post_backdrop_filter_perf`
- `simple_animation_perf`
- `textfield_perf`
58
- `fullscreen_textfield_perf`
59

60 61 62 63 64 65 66 67
### E2E benchmarks

(On-going work)

[E2E](https://pub.dev/packages/e2e)-based tests are driven independent of the
host machine. The following tests are E2E:

- `cull_opacity_perf.dart`
68
- `multi_widget_construction_perf`
69 70 71 72 73 74 75

These tests should be run by:

```bash
flutter drive --profile -t test/[test_name]_e2e.dart --driver test_driver/e2e_test.dart
```

76 77
## Web benchmarks

78
Web benchmarks are compiled from the same entry point in `lib/web_benchmarks.dart`.
79 80 81 82 83 84 85 86

### How to write a web benchmark

Create a new file for your benchmark under `lib/src/web`. See `bench_draw_rect.dart`
as an example.

Choose one of the two benchmark types:

87
- A "raw benchmark" records performance metrics from direct interactions with
88 89
  `dart:ui` with no framework. This kind of benchmark is good for benchmarking
  low-level engine primitives, such as layer, picture, and semantics performance.
90
- A "widget benchmark" records performance metrics using a widget. This kind of
91 92
  benchmark is good for measuring the performance of widgets, often together with
  engine work that widget-under-test incurs.
93
- A "widget build benchmark" records the cost of building a widget from nothing.
94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124
  This is different from the "widget benchmark" because typically the latter
  only performs incremental UI updates, such as an animation. In contrast, this
  benchmark pumps an empty frame to clear all previously built widgets and
  rebuilds them from scratch.

For a raw benchmark extend `RawRecorder` (tip: you can start by copying
`bench_draw_rect.dart`).

For a widget benchmark extend `WidgetRecorder` (tip: you can start by copying
`bench_simple_lazy_text_scroll.dart`).

For a widget build benchmark extend `WidgetBuildRecorder` (tip: you can start by copying
`bench_build_material_checkbox.dart`).

Pick a unique benchmark name and class name and add it to the `benchmarks` list
in `lib/web_benchmarks.dart`.

### How to run a web benchmark

Web benchmarks can be run using `flutter run` in debug, profile, and release
modes, using either the HTML or the CanvasKit rendering backend. Note, however,
that running in debug mode will result in worse numbers. Profile mode is useful
for profiling in Chrome DevTools because the numbers are close to release mode
and the profile contains unobfuscated names.

Example:

```
cd dev/benchmarks/macrobenchmarks

# Runs in profile mode using the HTML renderer
125
flutter run --web-renderer=html --profile -d web-server lib/web_benchmarks.dart
126 127

# Runs in profile mode using the CanvasKit renderer
128
flutter run --web-renderer=canvaskit --profile -d web-server lib/web_benchmarks.dart
129 130
```

131
You can also run all benchmarks exactly as the devicelab runs them:
132 133 134 135 136 137 138 139 140 141

```
cd dev/devicelab

# Runs using the HTML renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_html.dart

# Runs using the CanvasKit renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_canvaskit.dart
```
142 143 144 145 146 147 148

## Frame policy test

File `test/frame_policy.dart` and its driving script `test_driver/frame_policy_test.dart`
are used for testing [`fullyLive`](https://api.flutter.dev/flutter/flutter_test/LiveTestWidgetsFlutterBindingFramePolicy-class.html)
and [`benchmarkLive`](https://api.flutter.dev/flutter/flutter_test/LiveTestWidgetsFlutterBindingFramePolicy-class.html)
policies in terms of its effect on [`WidgetTester.handlePointerEventRecord`](https://master-api.flutter.dev/flutter/flutter_test/WidgetTester/handlePointerEventRecord.html).