• Michael Goderbauer's avatar
    Stand-alone widget tree with multiple render trees to enable multi-view rendering (#125003) · 6f09064e
    Michael Goderbauer authored
    This change enables Flutter to generate multiple Scenes to be rendered into separate FlutterViews from a single widget tree. Each Scene is described by a separate render tree, which are all associated with the single widget tree.
    
    This PR implements the framework-side mechanisms to describe the content to be rendered into multiple views. Separate engine-side changes are necessary to provide these views to the framework and to draw the framework-generated Scene into them.
    
    ## Summary of changes
    
    The details of this change are described in [flutter.dev/go/multiple-views](https://flutter.dev/go/multiple-views). Below is a high-level summary organized by layers.
    
    ### Rendering layer changes
    
    * The `RendererBinding` no longer owns a single `renderView`. In fact, it doesn't OWN any `RenderView`s at all anymore. Instead, it offers an API (`addRenderView`/`removeRenderView`) to add and remove `RenderView`s that then will be MANAGED by the binding. The `RenderView` itself is now owned by a higher-level abstraction (e.g. the `RawView` Element of the widgets layer, see below), who is also in charge of adding it to the binding. When added, the binding will interact with the `RenderView` to produce a frame (e.g. by calling `compositeFrame` on it) and to perform hit tests for incoming pointer events. Multiple `RenderView`s can be added to the binding (typically one per `FlutterView`) to produce multiple Scenes.
    * Instead of owning a single `pipelineOwner`, the `RendererBinding` now owns the root of the `PipelineOwner` tree (exposed as `rootPipelineOwner` on the binding). Each `PipelineOwner` in that tree (except for the root) typically manages its own render tree typically rooted in one of the `RenderView`s mentioned in the previous bullet. During frame production, the binding will instruct each `PipelineOwner` of that tree to flush layout, paint, semantics etc. A higher-level abstraction (e.g. the widgets layer, see below) is in charge of adding `PipelineOwner`s to this tree.
    * Backwards compatibility: The old `renderView` and `pipelineOwner` properties of the `RendererBinding` are retained, but marked as deprecated. Care has been taken to keep their original behavior for the deprecation period, i.e. if you just call `runApp`, the render tree bootstrapped by this call is rooted in the deprecated `RendererBinding.renderView` and managed by the deprecated `RendererBinding.pipelineOwner`.
    
    ### Widgets layer changes
    
    * The `WidgetsBinding` no longer attaches the widget tree to an existing render tree. Instead, it bootstraps a stand-alone widget tree that is not backed by a render tree. For this, `RenderObjectToWidgetAdapter` has been replaced by `RootWidget`.
    * Multiple render trees can be bootstrapped and attached to the widget tree with the help of the `View` widget, which internally is backed by a `RawView` widget. Configured with a `FlutterView` to render into, the `RawView` creates a new `PipelineOwner` and a new `RenderView` for the new render tree. It adds the new `RenderView` to the `RendererBinding` and its `PipelineOwner` to the pipeline owner tree.
    * The `View` widget can only appear in certain well-defined locations in the widget tree since it bootstraps a new render tree and does not insert a `RenderObject` into an ancestor. However, almost all Elements expect that their children insert `RenderObject`s, otherwise they will not function properly. To produce a good error message when the `View` widget is used in an illegal location, the `debugMustInsertRenderObjectIntoSlot` method has been added to Element, where a child can ask whether a given slot must insert a RenderObject into its ancestor or not. In practice, the `View` widget can be used as a child of the `RootWidget`, inside the `view` slot of the `ViewAnchor` (see below) and inside a `ViewCollection` (see below). In those locations, the `View` widget may be wrapped in other non-RenderObjectWidgets (e.g. InheritedWidgets).
    * The new `ViewAnchor` can be used to create a side-view inside a parent `View`. The `child` of the `ViewAnchor` widget renders into the parent `View` as usual, but the `view` slot can take on another `View` widget, which has access to all inherited widgets above the `ViewAnchor`. Metaphorically speaking, the view is anchored to the location of the `ViewAnchor` in the widget tree.
    * The new `ViewCollection` widget allows for multiple sibling views as it takes a list of `View`s as children. It can be used in all the places that accept a `View` widget.
    
    ## Google3
    
    As of July 5, 2023 this change passed a TAP global presubmit (TGP) in google3: tap/OCL:544707016:BASE:545809771:1688597935864:e43dd651
    
    ## Note to reviewers
    
    This change is big (sorry). I suggest focusing the initial review on the changes inside of `packages/flutter` first. The majority of the changes describe above are implemented in (listed in suggested review order):
    
    * `rendering/binding.dart`
    * `widgets/binding.dart`
    * `widgets/view.dart`
    * `widgets/framework.dart`
    
    All other changes included in the PR are basically the fallout of what's implemented in those files. Also note that a lot of the lines added in this PR are documentation and tests.
    
    I am also very happy to walk reviewers through the code in person or via video call, if that is helpful.
    
    I appreciate any feedback.
    
    ## Feedback to address before submitting ("TODO")
    6f09064e
platform_view_test.dart 12.6 KB
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:async';
import 'dart:ui' as ui;

import 'package:fake_async/fake_async.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/gestures.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';
import '../services/fake_platform_views.dart';
import 'rendering_tester.dart';

void main() {
  final TestRenderingFlutterBinding binding = TestRenderingFlutterBinding.ensureInitialized();

  group('PlatformViewRenderBox', () {
    late FakePlatformViewController fakePlatformViewController;
    late PlatformViewRenderBox platformViewRenderBox;
    setUp(() {
      fakePlatformViewController = FakePlatformViewController(0);
      platformViewRenderBox = PlatformViewRenderBox(
        controller: fakePlatformViewController,
        hitTestBehavior: PlatformViewHitTestBehavior.opaque,
        gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{
          Factory<VerticalDragGestureRecognizer>(
            () {
              return VerticalDragGestureRecognizer();
            },
          ),
        },
      );
    });

    test('layout should size to max constraint', () {
      layout(platformViewRenderBox);
      platformViewRenderBox.layout(const BoxConstraints(minWidth: 50, minHeight: 50, maxWidth: 100, maxHeight: 100));
      expect(platformViewRenderBox.size, const Size(100, 100));
    });

    test('send semantics update if id is changed', () {
      final RenderConstrainedBox tree = RenderConstrainedBox(
        additionalConstraints: const BoxConstraints.tightFor(height: 20.0, width: 20.0),
        child: platformViewRenderBox,
      );
      int semanticsUpdateCount = 0;
      final SemanticsHandle semanticsHandle = TestRenderingFlutterBinding.instance.rootPipelineOwner.ensureSemantics(
        listener: () {
          ++semanticsUpdateCount;
        },
      );
      layout(tree, phase: EnginePhase.flushSemantics);
      // Initial semantics update
      expect(semanticsUpdateCount, 1);

      semanticsUpdateCount = 0;

      // Request semantics update even though nothing changed.
      platformViewRenderBox.markNeedsSemanticsUpdate();
      pumpFrame(phase: EnginePhase.flushSemantics);
      expect(semanticsUpdateCount, 0);

      semanticsUpdateCount = 0;

      final FakePlatformViewController updatedFakePlatformViewController = FakePlatformViewController(10);
      platformViewRenderBox.controller = updatedFakePlatformViewController;
      pumpFrame(phase: EnginePhase.flushSemantics);
      // Update id should update the semantics.
      expect(semanticsUpdateCount, 1);

      semanticsHandle.dispose();
    });

    test('mouse hover events are dispatched via PlatformViewController.dispatchPointerEvent', () {
      layout(platformViewRenderBox);
      pumpFrame(phase: EnginePhase.flushSemantics);

      RendererBinding.instance.platformDispatcher.onPointerDataPacket!(ui.PointerDataPacket(data: <ui.PointerData>[
        _pointerData(ui.PointerChange.add, Offset.zero),
        _pointerData(ui.PointerChange.hover, const Offset(10, 10)),
        _pointerData(ui.PointerChange.remove, const Offset(10, 10)),
      ]));

      expect(fakePlatformViewController.dispatchedPointerEvents, isNotEmpty);
    });

    test('touch hover events are dispatched via PlatformViewController.dispatchPointerEvent', () {
      layout(platformViewRenderBox);
      pumpFrame(phase: EnginePhase.flushSemantics);

      RendererBinding.instance.platformDispatcher.onPointerDataPacket!(ui.PointerDataPacket(data: <ui.PointerData>[
        _pointerData(ui.PointerChange.add, Offset.zero),
        _pointerData(ui.PointerChange.hover, const Offset(10, 10)),
        _pointerData(ui.PointerChange.remove, const Offset(10, 10)),
      ]));

      expect(fakePlatformViewController.dispatchedPointerEvents, isNotEmpty);
    });

  });

  // Regression test for https://github.com/flutter/flutter/issues/69431
  test('multi-finger touch test', () {
    final FakeAndroidPlatformViewsController viewsController = FakeAndroidPlatformViewsController();
    viewsController.registerViewType('webview');
    final AndroidViewController viewController =
      PlatformViewsService.initAndroidView(id: 0, viewType: 'webview', layoutDirection: TextDirection.rtl);
    final PlatformViewRenderBox platformViewRenderBox = PlatformViewRenderBox(
      controller: viewController,
      hitTestBehavior: PlatformViewHitTestBehavior.opaque,
      gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{
        Factory<VerticalDragGestureRecognizer>(
          () => VerticalDragGestureRecognizer(),
        ),
      },
    );
    layout(platformViewRenderBox);
    pumpFrame(phase: EnginePhase.flushSemantics);

    viewController.pointTransformer = (Offset offset) => platformViewRenderBox.globalToLocal(offset);

    FakeAsync().run((FakeAsync async) {
      // Put one pointer down.
      RendererBinding.instance.platformDispatcher.onPointerDataPacket!(ui.PointerDataPacket(data: <ui.PointerData>[
        _pointerData(ui.PointerChange.add, Offset.zero, pointer: 1, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.down, const Offset(10, 10), pointer: 1, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.remove, const Offset(10, 10), pointer: 1, kind: PointerDeviceKind.touch),
      ]));
      async.flushMicrotasks();

      // Put another pointer down and then cancel it.
      RendererBinding.instance.platformDispatcher.onPointerDataPacket!(ui.PointerDataPacket(data: <ui.PointerData>[
        _pointerData(ui.PointerChange.add, Offset.zero, pointer: 2, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.down, const Offset(20, 10), pointer: 2, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.cancel, const Offset(20, 10), pointer: 2, kind: PointerDeviceKind.touch),
      ]));
      async.flushMicrotasks();

      // The first pointer can still moving without crashing.
      RendererBinding.instance.platformDispatcher.onPointerDataPacket!(ui.PointerDataPacket(data: <ui.PointerData>[
        _pointerData(ui.PointerChange.add, Offset.zero, pointer: 1, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.move, const Offset(10, 10), pointer: 1, kind: PointerDeviceKind.touch),
        _pointerData(ui.PointerChange.remove, const Offset(10, 10), pointer: 1, kind: PointerDeviceKind.touch),
      ]));
      async.flushMicrotasks();
    });

    // Passes if no crashes.
  });

  test('created callback is reset when controller is changed', () {
    final FakeAndroidPlatformViewsController viewsController = FakeAndroidPlatformViewsController();
    viewsController.registerViewType('webview');
    final AndroidViewController firstController = PlatformViewsService.initAndroidView(
      id: 0,
      viewType: 'webview',
      layoutDirection: TextDirection.rtl,
    );
    final RenderAndroidView renderBox = RenderAndroidView(
      viewController: firstController,
      hitTestBehavior: PlatformViewHitTestBehavior.opaque,
      gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{},
    );
    layout(renderBox);
    pumpFrame(phase: EnginePhase.flushSemantics);

    expect(firstController.createdCallbacks, isNotEmpty);
    expect(firstController.createdCallbacks.length, 1);

    final AndroidViewController secondController = PlatformViewsService.initAndroidView(
      id: 0,
      viewType: 'webview',
      layoutDirection: TextDirection.rtl,
    );
    // Reset controller.
    renderBox.controller = secondController;

    expect(firstController.createdCallbacks, isEmpty);
    expect(secondController.createdCallbacks, isNotEmpty);
    expect(secondController.createdCallbacks.length, 1);
  });

  test('render object changed its visual appearance after texture is created', () {
    FakeAsync().run((FakeAsync async) {
      final AndroidViewController viewController =
        PlatformViewsService.initAndroidView(id: 0, viewType: 'webview', layoutDirection: TextDirection.rtl);
      final RenderAndroidView renderBox = RenderAndroidView(
        viewController: viewController,
        hitTestBehavior: PlatformViewHitTestBehavior.opaque,
        gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{},
      );

      final Completer<void> viewCreation = Completer<void>();
      const MethodChannel channel = MethodChannel('flutter/platform_views');
      binding.defaultBinaryMessenger.setMockMethodCallHandler(channel, (MethodCall methodCall) async {
        assert(methodCall.method == 'create', 'Unexpected method call');
        await viewCreation.future;
        return /*textureId=*/ 0;
      });

      layout(renderBox);
      pumpFrame(phase: EnginePhase.paint);

      expect(renderBox.debugLayer, isNotNull);
      expect(renderBox.debugLayer!.hasChildren, isFalse);
      expect(viewController.isCreated, isFalse);
      expect(renderBox.debugNeedsPaint, isFalse);

      viewCreation.complete();
      async.flushMicrotasks();

      expect(viewController.isCreated, isTrue);
      expect(renderBox.debugNeedsPaint, isTrue);
      expect(renderBox.debugLayer!.hasChildren, isFalse);

      pumpFrame(phase: EnginePhase.paint);
      expect(renderBox.debugLayer!.hasChildren, isTrue);
      expect(renderBox.debugLayer!.firstChild, isA<TextureLayer>());
    });
  });

  test('markNeedsPaint does not get called on a disposed RO', () async {
    FakeAsync().run((FakeAsync async) {
      final AndroidViewController viewController =
        PlatformViewsService.initAndroidView(id: 0, viewType: 'webview', layoutDirection: TextDirection.rtl);
      final RenderAndroidView renderBox = RenderAndroidView(
        viewController: viewController,
        hitTestBehavior: PlatformViewHitTestBehavior.opaque,
        gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{},
      );

      final Completer<void> viewCreation = Completer<void>();
      const MethodChannel channel = MethodChannel('flutter/platform_views');
      binding.defaultBinaryMessenger.setMockMethodCallHandler(channel, (MethodCall methodCall) async {
        assert(methodCall.method == 'create', 'Unexpected method call');
        await viewCreation.future;
        return /*textureId=*/ 0;
      });

      layout(renderBox);
      pumpFrame(phase: EnginePhase.paint);

      expect(renderBox.debugLayer, isNotNull);
      expect(renderBox.debugLayer!.hasChildren, isFalse);
      expect(viewController.isCreated, isFalse);
      expect(renderBox.debugNeedsPaint, isFalse);

      renderBox.dispose();
      viewCreation.complete();
      async.flushMicrotasks();

      expect(viewController.isCreated, isTrue);
      expect(renderBox.debugNeedsPaint, isFalse);
      expect(renderBox.debugLayer, isNull);

      pumpFrame(phase: EnginePhase.paint);
      expect(renderBox.debugLayer, isNull);
    });
  });

  test('markNeedsPaint does not get called when setting the same viewController', () {
    FakeAsync().run((FakeAsync async) {
      final Completer<void> viewCreation = Completer<void>();
      const MethodChannel channel = MethodChannel('flutter/platform_views');
      binding.defaultBinaryMessenger.setMockMethodCallHandler(channel, (MethodCall methodCall) async {
        assert(methodCall.method == 'create', 'Unexpected method call');
        await viewCreation.future;
        return /*textureId=*/ 0;
      });

      bool futureCallbackRan = false;

      PlatformViewsService.initUiKitView(id: 0, viewType: 'webview', layoutDirection: TextDirection.ltr).then((UiKitViewController viewController) {
        final RenderUiKitView renderBox = RenderUiKitView(
          viewController: viewController,
          hitTestBehavior: PlatformViewHitTestBehavior.opaque,
          gestureRecognizers: <Factory<OneSequenceGestureRecognizer>>{},
        );

        layout(renderBox);
        pumpFrame(phase: EnginePhase.paint);
        expect(renderBox.debugNeedsPaint, isFalse);

        renderBox.viewController = viewController;

        expect(renderBox.debugNeedsPaint, isFalse);

        futureCallbackRan = true;
      });

      viewCreation.complete();
      async.flushMicrotasks();
      expect(futureCallbackRan, true);
    });
  });
}

ui.PointerData _pointerData(
  ui.PointerChange change,
  Offset logicalPosition, {
  int device = 0,
  PointerDeviceKind kind = PointerDeviceKind.mouse,
  int pointer = 0,
}) {
  final double devicePixelRatio = RendererBinding.instance.platformDispatcher.implicitView!.devicePixelRatio;
  return ui.PointerData(
    pointerIdentifier: pointer,
    embedderId: pointer,
    change: change,
    physicalX: logicalPosition.dx * devicePixelRatio,
    physicalY: logicalPosition.dy * devicePixelRatio,
    kind: kind,
    device: device,
  );
}