Unverified Commit 07772a3d authored by Yegor's avatar Yegor Committed by GitHub

[framework,web] add FlutterTimeline and semantics benchmarks that use it (#128366)

## FlutterTimeline

Add a new class `FlutterTimeline` that's a drop-in replacement for `Timeline` from `dart:developer`. In addition to forwarding invocations of `startSync`, `finishSync`, `timeSync`, and `instantSync` to `dart:developer`, provides the following extra methods that make is easy to collect timings for code blocks on a frame-by-frame basis:

* `debugCollect()` - aggregates timings since the last reset, or since the app launched.
* `debugReset()` - forgets all data collected since the previous reset, or since the app launched. This allows clearing data from previous frames so timings can be attributed to the current frame.
* `now` - this was enhanced so that it works on the web by calling `window.performance.now` (in `Timeline` this is a noop in Dart web compilers).
* `collectionEnabled` - a field that controls whether `FlutterTimeline` stores timings in memory. By default this is disabled to avoid unexpected overhead (although the class is designed for minimal and predictable overhead). Specific benchmarks can enable collection to report to Skia Perf.

## Semantics benchmarks

Add `BenchMaterial3Semantics` that benchmarks the cost of semantics when constructing a screen full of Material 3 widgets from nothing. It is expected that semantics will have non-trivial cost in this case, but we should strive to keep it much lower than the rendering cost. This is the case already. This benchmark shows that the cost of semantics is <10%.

Add `BenchMaterial3ScrollSemantics` that benchmarks the cost of scrolling a previously constructed screen full of Material 3 widgets. The expectation should be that semantics will have trivial cost, since we're just shifting some widgets around. As of today, the numbers are not great, with semantics taking >50% of frame time, which is what prompted this PR in the first place. As we optimize this, we want to see this number improve.
parent 22005779
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:async';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter/scheduler.dart';
import 'package:flutter/semantics.dart';
import 'material3.dart';
import 'recorder.dart';
/// Measures the cost of semantics when constructing screens containing
/// Material 3 widgets.
class BenchMaterial3Semantics extends WidgetBuildRecorder {
BenchMaterial3Semantics() : super(name: benchmarkName);
static const String benchmarkName = 'bench_material3_semantics';
@override
Future<void> setUpAll() async {
FlutterTimeline.debugCollectionEnabled = true;
super.setUpAll();
SemanticsBinding.instance.ensureSemantics();
}
@override
Future<void> tearDownAll() async {
FlutterTimeline.debugReset();
}
@override
void frameDidDraw() {
// Only record frames that show the widget. Frames that remove the widget
// are not interesting.
if (showWidget) {
final AggregatedTimings timings = FlutterTimeline.debugCollect();
final AggregatedTimedBlock semanticsBlock = timings.getAggregated('SEMANTICS');
final AggregatedTimedBlock getFragmentBlock = timings.getAggregated('Semantics.GetFragment');
final AggregatedTimedBlock compileChildrenBlock = timings.getAggregated('Semantics.compileChildren');
profile!.addTimedBlock(semanticsBlock, reported: true);
profile!.addTimedBlock(getFragmentBlock, reported: true);
profile!.addTimedBlock(compileChildrenBlock, reported: true);
}
super.frameDidDraw();
FlutterTimeline.debugReset();
}
@override
Widget createWidget() {
return const SingleColumnMaterial3Components();
}
}
/// Measures the cost of semantics when scrolling screens containing Material 3
/// widgets.
///
/// The implementation uses a ListView that jumps the scroll position between
/// 0 and 1 every frame. Such a small delta is not enough for lazy rendering to
/// add/remove widgets, but its enough to trigger the framework to recompute
/// some of the semantics.
///
/// The expected output numbers of this benchmarks should be very small as
/// scrolling a list view should be a matter of shifting some widgets and
/// updating the projected clip imposed by the viewport. As of June 2023, the
/// numbers are not great. Semantics consumes >50% of frame time.
class BenchMaterial3ScrollSemantics extends WidgetRecorder {
BenchMaterial3ScrollSemantics() : super(name: benchmarkName);
static const String benchmarkName = 'bench_material3_scroll_semantics';
@override
Future<void> setUpAll() async {
FlutterTimeline.debugCollectionEnabled = true;
super.setUpAll();
SemanticsBinding.instance.ensureSemantics();
}
@override
Future<void> tearDownAll() async {
FlutterTimeline.debugReset();
}
@override
void frameDidDraw() {
final AggregatedTimings timings = FlutterTimeline.debugCollect();
final AggregatedTimedBlock semanticsBlock = timings.getAggregated('SEMANTICS');
final AggregatedTimedBlock getFragmentBlock = timings.getAggregated('Semantics.GetFragment');
final AggregatedTimedBlock compileChildrenBlock = timings.getAggregated('Semantics.compileChildren');
profile!.addTimedBlock(semanticsBlock, reported: true);
profile!.addTimedBlock(getFragmentBlock, reported: true);
profile!.addTimedBlock(compileChildrenBlock, reported: true);
super.frameDidDraw();
FlutterTimeline.debugReset();
}
@override
Widget createWidget() => _ScrollTest();
}
class _ScrollTest extends StatefulWidget {
@override
State<_ScrollTest> createState() => _ScrollTestState();
}
class _ScrollTestState extends State<_ScrollTest> with SingleTickerProviderStateMixin {
late final Ticker ticker;
late final ScrollController scrollController;
@override
void initState() {
super.initState();
scrollController = ScrollController();
bool forward = true;
// A one-off timer is necessary to allow the framework to measure the
// available scroll extents before the scroll controller can be exercised
// to change the scroll position.
Timer.run(() {
ticker = createTicker((_) {
scrollController.jumpTo(forward ? 1 : 0);
forward = !forward;
});
ticker.start();
});
}
@override
void dispose() {
ticker.dispose();
scrollController.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return SingleColumnMaterial3Components(
scrollController: scrollController,
);
}
}
......@@ -203,6 +203,7 @@ class BenchBuildColorsGrid extends WidgetBuildRecorder {
@override
Future<void> setUpAll() async {
super.setUpAll();
registerEngineBenchmarkValueListener('text_layout', (num value) {
_textLayoutMicros += value;
});
......
This diff is collapsed.
......@@ -426,12 +426,18 @@ abstract class WidgetRecorder extends Recorder implements FrameRecorder {
_runCompleter!.completeError(error, stackTrace);
}
late final _RecordingWidgetsBinding _binding;
@override
@mustCallSuper
Future<void> setUpAll() async {
_binding = _RecordingWidgetsBinding.ensureInitialized();
}
@override
Future<Profile> run() async {
_runCompleter = Completer<void>();
final Profile localProfile = profile = Profile(name: name, useCustomWarmUp: useCustomWarmUp);
final _RecordingWidgetsBinding binding =
_RecordingWidgetsBinding.ensureInitialized();
final Widget widget = createWidget();
registerEngineBenchmarkValueListener(kProfilePrerollFrame, (num value) {
......@@ -449,7 +455,7 @@ abstract class WidgetRecorder extends Recorder implements FrameRecorder {
);
});
binding._beginRecording(this, widget);
_binding._beginRecording(this, widget);
try {
await _runCompleter!.future;
......@@ -508,6 +514,14 @@ abstract class WidgetBuildRecorder extends Recorder implements FrameRecorder {
}
}
late final _RecordingWidgetsBinding _binding;
@override
@mustCallSuper
Future<void> setUpAll() async {
_binding = _RecordingWidgetsBinding.ensureInitialized();
}
@override
@mustCallSuper
void frameWillDraw() {
......@@ -546,9 +560,7 @@ abstract class WidgetBuildRecorder extends Recorder implements FrameRecorder {
Future<Profile> run() async {
_runCompleter = Completer<void>();
final Profile localProfile = profile = Profile(name: name);
final _RecordingWidgetsBinding binding =
_RecordingWidgetsBinding.ensureInitialized();
binding._beginRecording(this, _WidgetBuildRecorderHost(this));
_binding._beginRecording(this, _WidgetBuildRecorderHost(this));
try {
await _runCompleter!.future;
......@@ -948,6 +960,15 @@ class Profile {
}
}
/// A convenience wrapper over [addDataPoint] for adding [AggregatedTimedBlock]
/// to the profile.
///
/// Uses [AggregatedTimedBlock.name] as the name of the data point, and
/// [AggregatedTimedBlock.duration] as the duration.
void addTimedBlock(AggregatedTimedBlock timedBlock, { required bool reported }) {
addDataPoint(timedBlock.name, Duration(microseconds: timedBlock.duration.toInt()), reported: reported);
}
/// Checks the samples collected so far and sets the appropriate benchmark phase.
///
/// If enough warm-up samples have been collected, stops the warm-up phase and
......
......@@ -19,6 +19,7 @@ import 'src/web/bench_draw_rect.dart';
import 'src/web/bench_dynamic_clip_on_static_picture.dart';
import 'src/web/bench_image_decoding.dart';
import 'src/web/bench_material_3.dart';
import 'src/web/bench_material_3_semantics.dart';
import 'src/web/bench_mouse_region_grid_hover.dart';
import 'src/web/bench_mouse_region_grid_scroll.dart';
import 'src/web/bench_mouse_region_mixed_grid_hover.dart';
......@@ -64,6 +65,8 @@ final Map<String, RecorderFactory> benchmarks = <String, RecorderFactory>{
BenchPlatformViewInfiniteScroll.benchmarkName: () => BenchPlatformViewInfiniteScroll.forward(),
BenchPlatformViewInfiniteScroll.benchmarkNameBackward: () => BenchPlatformViewInfiniteScroll.backward(),
BenchMaterial3Components.benchmarkName: () => BenchMaterial3Components(),
BenchMaterial3Semantics.benchmarkName: () => BenchMaterial3Semantics(),
BenchMaterial3ScrollSemantics.benchmarkName: () => BenchMaterial3ScrollSemantics(),
// CanvasKit-only benchmarks
if (isCanvasKit) ...<String, RecorderFactory>{
......
......@@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:developer';
import 'package:flutter/foundation.dart';
import '../common.dart';
......@@ -16,8 +16,8 @@ void main() {
final Stopwatch watch = Stopwatch();
watch.start();
for (int i = 0; i < _kNumIterations; i += 1) {
Timeline.startSync('foo');
Timeline.finishSync();
FlutterTimeline.startSync('foo');
FlutterTimeline.finishSync();
}
watch.stop();
......@@ -31,14 +31,14 @@ void main() {
watch.reset();
watch.start();
for (int i = 0; i < _kNumIterations; i += 1) {
Timeline.startSync('foo', arguments: <String, dynamic>{
FlutterTimeline.startSync('foo', arguments: <String, dynamic>{
'int': 1234,
'double': 0.3,
'list': <int>[1, 2, 3, 4],
'map': <String, dynamic>{'map': true},
'bool': false,
});
Timeline.finishSync();
FlutterTimeline.finishSync();
}
watch.stop();
......
......@@ -1375,6 +1375,14 @@ Future<void> _runWebTreeshakeTest() async {
pos = javaScript.indexOf(word, pos);
}
// The following are classes from `timeline.dart` that should be treeshaken
// off unless the app (typically a benchmark) uses methods that need them.
expect(javaScript.contains('AggregatedTimedBlock'), false);
expect(javaScript.contains('AggregatedTimings'), false);
expect(javaScript.contains('_BlockBuffer'), false);
expect(javaScript.contains('_StringListChain'), false);
expect(javaScript.contains('_Float64ListChain'), false);
const int kMaxExpectedDebugFillProperties = 11;
if (count > kMaxExpectedDebugFillProperties) {
throw Exception(
......
......@@ -46,4 +46,5 @@ export 'src/foundation/serialization.dart';
export 'src/foundation/service_extensions.dart';
export 'src/foundation/stack_frame.dart';
export 'src/foundation/synchronous_future.dart';
export 'src/foundation/timeline.dart';
export 'src/foundation/unicode.dart';
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment