Unverified Commit cfafb910 authored by keyonghan's avatar keyonghan Committed by GitHub

Add DeviceLab build+test separation doc (#131365)

This PR add instructions to separate DeviceLab tasks to build and test model.
parent 48f08e3d
...@@ -19,6 +19,7 @@ for information on using the dashboards. ...@@ -19,6 +19,7 @@ for information on using the dashboards.
* [Adding tests to continuous * [Adding tests to continuous
integration](#adding-tests-to-continuous-integration) integration](#adding-tests-to-continuous-integration)
* [Adding tests to presubmit](#adding-tests-to-presubmit) * [Adding tests to presubmit](#adding-tests-to-presubmit)
* [Migrating to build and test model](#migrating-to-build-and-test-model)
## How the DeviceLab runs tests ## How the DeviceLab runs tests
...@@ -226,3 +227,33 @@ target for each operating system. ...@@ -226,3 +227,33 @@ target for each operating system.
Flutter's DeviceLab has a limited capacity in presubmit. File an infra ticket Flutter's DeviceLab has a limited capacity in presubmit. File an infra ticket
to investigate feasibility of adding a test to presubmit. to investigate feasibility of adding a test to presubmit.
## Migrating to build and test model
To better utilize limited DeviceLab testbed resources and speed up commit validation
time, it is now supported to separate building artifacts (.apk/.app) from testing them.
The artifact will be built on a host only bot, a VM or physical bot without a device,
and the test will run based on the artifact against a testbed with a device.
Steps:
1. Update the task class to extend [`BuildTestTask`](https://github.com/flutter/flutter/blob/master/dev/devicelab/lib/tasks/build_test_task.dart)
- Override function `getBuildArgs`
- Override function `getTestArgs`
- Override function `parseTaskResult`
- Override function `getApplicationBinaryPath`
2. Update the `bin/tasks/{TEST}.dart` to point to the new task class
3. Validate the task locally
- build only: `dart bin/test_runner.dart test -t {NAME_OR_PATH_OF_TEST} --task-args build --task-args application-binary-path={PATH_TO_ARTIFACT}`
- test only: `dart bin/test_runner.dart test -t {NAME_OR_PATH_OF_TEST} --task-args test --task-args application-binary-path={PATH_TO_ARTIFACT}`
4. Add tasks to continuous integration
- Mirror a target with platform `Linux_build_test` or `Mac_build_test`
- The only difference from regular targets is the artifact property: if omitted, it will use the `task_name`.
5. Once validated in CI, enable the target in `PROD` by removing `bringup: true` and deleting the old target entry without build+test model.
Take gallery tasks for example:
1. Linux android
- Separating PR: https://github.com/flutter/flutter/pull/103550
- Switching PR: https://github.com/flutter/flutter/pull/110533
2. Mac iOS: https://github.com/flutter/flutter/pull/111164
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment