Skip to content

F
Front-End
Unverified Commit 0d046b34 authored by Dan Field's avatar Dan Field Committed by GitHub
Browse files
Options

Drop metrics_center (#74535)

parent 120966f9
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 1917 deletions
dev/benchmarks/metrics_center/LICENSE deleted 100644 → 0
View file @ 120966f9
Copyright 2014 The Flutter Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
\ No newline at end of file
dev/benchmarks/metrics_center/README.md deleted 100644 → 0
View file @ 120966f9
Metrics center is a minimal set of code and services to support multiple perf
metrics generators (e.g., Cocoon device lab, Cirrus bots, LUCI bots, Firebase
Test Lab) and destinations (e.g., old Cocoon perf dashboard, Skia perf
dashboard). The work and maintenance it requires is very close to that of just
supporting a single generator and destination (e.g., engine bots to Skia perf),
and the small amount of extra work is designed to make it easy to support more
generators and destinations in the future.
This is currently under migration. More documentations will be added once the
migration is done.
dev/benchmarks/metrics_center/lib/metrics_center.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
export 'src/common.dart';
export 'src/flutter.dart';
export 'src/google_benchmark.dart';
export 'src/skiaperf.dart';
dev/benchmarks/metrics_center/lib/src/common.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:collection';
import 'dart:convert';
import 'package:crypto/crypto.dart';
import 'package:equatable/equatable.dart';
import 'package:googleapis_auth/auth.dart';
import 'package:googleapis_auth/auth_io.dart';
import 'package:http/http.dart';
/// Common format of a metric data point.
class MetricPoint extends Equatable {
MetricPoint(
this.value,
Map<String, String> tags,
) : _tags = SplayTreeMap<String, String>.from(tags);
/// Can store integer values.
final double value;
/// Test name, unit, timestamp, configs, git revision, ..., in sorted order.
UnmodifiableMapView<String, String> get tags =>
UnmodifiableMapView<String, String>(_tags);
/// Unique identifier for updating existing data point.
///
/// We shouldn't have to worry about hash collisions until we have about
/// 2^128 points.
///
/// This id should stay constant even if the [tags.keys] are reordered.
/// (Because we are using an ordered SplayTreeMap to generate the id.)
String get id => sha256.convert(utf8.encode('$_tags')).toString();
@override
String toString() {
return 'MetricPoint(value=$value, tags=$_tags)';
}
final SplayTreeMap<String, String> _tags;
@override
List<Object> get props => <Object>[value, tags];
}
/// Interface to write [MetricPoint].
abstract class MetricDestination {
/// Insert new data points or modify old ones with matching id.
Future<void> update(List<MetricPoint> points);
}
/// Create `AuthClient` in case we only have an access token without the full
/// credentials json. It's currently the case for Chrmoium LUCI bots.
AuthClient authClientFromAccessToken(String token, List<String> scopes) {
final DateTime anHourLater = DateTime.now().add(const Duration(hours: 1));
final AccessToken accessToken =
AccessToken('Bearer', token, anHourLater.toUtc());
final AccessCredentials accessCredentials =
AccessCredentials(accessToken, null, scopes);
return authenticatedClient(Client(), accessCredentials);
}
/// Some common tag keys
const String kGithubRepoKey = 'gitRepo';
const String kGitRevisionKey = 'gitRevision';
const String kUnitKey = 'unit';
const String kNameKey = 'name';
const String kSubResultKey = 'subResult';
/// Known github repo
const String kFlutterFrameworkRepo = 'flutter/flutter';
const String kFlutterEngineRepo = 'flutter/engine';
/// The key for the GCP project id in the credentials json.
const String kProjectId = 'project_id';
dev/benchmarks/metrics_center/lib/src/flutter.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/legacy_datastore.dart';
import 'package:metrics_center/src/legacy_flutter.dart';
/// Convenient class to capture the benchmarks in the Flutter engine repo.
class FlutterEngineMetricPoint extends MetricPoint {
FlutterEngineMetricPoint(
String name,
double value,
String gitRevision, {
Map<String, String> moreTags = const <String, String>{},
}) : super(
value,
<String, String>{
kNameKey: name,
kGithubRepoKey: kFlutterEngineRepo,
kGitRevisionKey: gitRevision,
}..addAll(moreTags),
);
}
/// All Flutter performance metrics (framework, engine, ...) should be written
/// to this destination.
class FlutterDestination extends MetricDestination {
// TODO(liyuqian): change the implementation of this class (without changing
// its public APIs) to remove `LegacyFlutterDestination` and directly use
// `SkiaPerfDestination` once the migration is fully done.
FlutterDestination._(this._legacyDestination);
static Future<FlutterDestination> makeFromCredentialsJson(
Map<String, dynamic> json) async {
final LegacyFlutterDestination legacyDestination =
LegacyFlutterDestination(await datastoreFromCredentialsJson(json));
return FlutterDestination._(legacyDestination);
}
static FlutterDestination makeFromAccessToken(
String accessToken, String projectId) {
final LegacyFlutterDestination legacyDestination = LegacyFlutterDestination(
datastoreFromAccessToken(accessToken, projectId));
return FlutterDestination._(legacyDestination);
}
@override
Future<void> update(List<MetricPoint> points) async {
await _legacyDestination.update(points);
}
final LegacyFlutterDestination _legacyDestination;
}
dev/benchmarks/metrics_center/lib/src/gcs_lock.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:googleapis/storage/v1.dart';
import 'package:googleapis_auth/auth.dart';
/// Global (in terms of earth) mutex using Google Cloud Storage.
class GcsLock {
/// Create a lock with an authenticated client and a GCS bucket name.
///
/// The client is used to communicate with Google Cloud Storage APIs.
GcsLock(this._client, this._bucketName)
: assert(_client != null),
assert(_bucketName != null) {
_api = StorageApi(_client);
}
/// Create a temporary lock file in GCS, and use it as a mutex mechanism to
/// run a piece of code exclusively.
///
/// There must be no existing lock file with the same name in order to
/// proceed. If multiple [GcsLock]s with the same `bucketName` and
/// `lockFileName` try [protectedRun] simultaneously, only one will proceed
/// and create the lock file. All others will be blocked.
///
/// When [protectedRun] finishes, the lock file is deleted, and other blocked
/// [protectedRun] may proceed.
///
/// If the lock file is stuck (e.g., `_unlock` is interrupted unexpectedly),
/// one may need to manually delete the lock file from GCS to unblock any
/// [protectedRun] that may depend on it.
Future<void> protectedRun(String lockFileName, Future<void> f()) async {
await _lock(lockFileName);
try {
await f();
} catch (e, stacktrace) {
print(stacktrace);
rethrow;
} finally {
await _unlock(lockFileName);
}
}
Future<void> _lock(String lockFileName) async {
final Object object = Object();
object.bucket = _bucketName;
object.name = lockFileName;
final Media content = Media(const Stream<List<int>>.empty(), 0);
Duration waitPeriod = const Duration(milliseconds: 10);
bool locked = false;
while (!locked) {
try {
await _api.objects.insert(object, _bucketName,
ifGenerationMatch: '0', uploadMedia: content);
locked = true;
} on DetailedApiRequestError catch (e) {
if (e.status == 412) {
// Status 412 means that the lock file already exists. Wait until
// that lock file is deleted.
await Future<void>.delayed(waitPeriod);
waitPeriod *= 2;
if (waitPeriod >= _kWarningThreshold) {
print(
'The lock is waiting for a long time: $waitPeriod. '
'If the lock file $lockFileName in bucket $_bucketName '
'seems to be stuck (i.e., it was created a long time ago and '
'no one seems to be owning it currently), delete it manually '
'to unblock this.',
);
}
} else {
rethrow;
}
}
}
}
Future<void> _unlock(String lockFileName) async {
await _api.objects.delete(_bucketName, lockFileName);
}
StorageApi _api;
final String _bucketName;
final AuthClient _client;
static const Duration _kWarningThreshold = Duration(seconds: 10);
}
dev/benchmarks/metrics_center/lib/src/github_helper.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:github/github.dart';
/// Singleton class to query some Github info with an in-memory cache.
class GithubHelper {
/// Return the singleton helper.
factory GithubHelper() {
return _singleton;
}
GithubHelper._internal();
/// The result is cached in memory so querying the same thing again in the
/// same process is fast.
///
/// Our unit test requires that calling this method 1000 times for the same
/// `githubRepo` and `sha` should be done in 1 second.
Future<DateTime> getCommitDateTime(String githubRepo, String sha) async {
final String key = '$githubRepo/commit/$sha';
if (_commitDateTimeCache[key] == null) {
final RepositoryCommit commit = await _github.repositories
.getCommit(RepositorySlug.full(githubRepo), sha);
_commitDateTimeCache[key] = commit.commit.committer.date;
}
return _commitDateTimeCache[key];
}
static final GithubHelper _singleton = GithubHelper._internal();
final GitHub _github = GitHub(auth: findAuthenticationFromEnvironment());
final Map<String, DateTime> _commitDateTimeCache = <String, DateTime>{};
}
dev/benchmarks/metrics_center/lib/src/google_benchmark.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:convert';
import 'dart:io';
import 'package:metrics_center/src/common.dart';
const String _kTimeUnitKey = 'time_unit';
const List<String> _kNonNumericalValueSubResults = <String>[
kNameKey,
_kTimeUnitKey,
'iterations',
'big_o',
];
/// Parse the json result of https://github.com/google/benchmark.
class GoogleBenchmarkParser {
/// Given a Google benchmark json output, parse its content into a list of [MetricPoint].
static Future<List<MetricPoint>> parse(String jsonFileName) async {
final Map<String, dynamic> jsonResult =
jsonDecode(File(jsonFileName).readAsStringSync())
as Map<String, dynamic>;
final Map<String, dynamic> rawContext = jsonResult['context'] as Map<String, dynamic>;
final Map<String, String> context = rawContext.map<String, String>(
(String k, dynamic v) => MapEntry<String, String>(k, v.toString()),
);
final List<MetricPoint> points = <MetricPoint>[];
for (final dynamic item in jsonResult['benchmarks']) {
_parseAnItem(item as Map<String, dynamic>, points, context);
}
return points;
}
}
void _parseAnItem(
Map<String, dynamic> item,
List<MetricPoint> points,
Map<String, String> context,
) {
final String name = item[kNameKey] as String;
final Map<String, String> timeUnitMap = <String, String>{
kUnitKey: item[_kTimeUnitKey] as String
};
for (final String subResult in item.keys) {
if (!_kNonNumericalValueSubResults.contains(subResult)) {
num rawValue;
try {
rawValue = item[subResult] as num;
} catch (e) {
print('$subResult: ${item[subResult]} (${item[subResult].runtimeType}) is not a number');
rethrow;
}
final double value = rawValue is int ? rawValue.toDouble() : rawValue as double;
points.add(
MetricPoint(
value,
<String, String>{kNameKey: name, kSubResultKey: subResult}
..addAll(context)
..addAll(subResult.endsWith('time') ? timeUnitMap : <String, String>{}),
),
);
}
}
}
dev/benchmarks/metrics_center/lib/src/legacy_datastore.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// TODO(liyuqian): Remove this file once the migration is fully done and we no
// longer need to fall back to the datastore.
import 'package:gcloud/db.dart';
import 'package:googleapis_auth/auth.dart';
import 'package:googleapis_auth/auth_io.dart';
// The official pub.dev/packages/gcloud documentation uses datastore_impl
// so we have to ignore implementation_imports here.
// ignore: implementation_imports
import 'package:gcloud/src/datastore_impl.dart';
import 'common.dart';
Future<DatastoreDB> datastoreFromCredentialsJson(
Map<String, dynamic> json) async {
final AutoRefreshingAuthClient client = await clientViaServiceAccount(
ServiceAccountCredentials.fromJson(json), DatastoreImpl.SCOPES);
return DatastoreDB(DatastoreImpl(client, json[kProjectId] as String));
}
DatastoreDB datastoreFromAccessToken(String token, String projectId) {
final AuthClient client =
authClientFromAccessToken(token, DatastoreImpl.SCOPES);
return DatastoreDB(DatastoreImpl(client, projectId));
}
dev/benchmarks/metrics_center/lib/src/legacy_flutter.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// TODO(liyuqian): Remove this legacy file once the migration is fully done.
// See go/flutter-metrics-center-migration for detailed plans.
import 'dart:convert';
import 'dart:math';
import 'package:gcloud/db.dart';
import 'common.dart';
import 'legacy_datastore.dart';
const String kSourceTimeMicrosName = 'sourceTimeMicros';
// The size of 500 is currently limited by Google datastore. It cannot write
// more than 500 entities in a single call.
const int kMaxBatchSize = 500;
/// This model corresponds to the existing data model 'MetricPoint' used in the
/// flutter-cirrus GCP project.
///
/// The originId and sourceTimeMicros fields are no longer used but we are still
/// providing valid values to them so it's compatible with old code and services
/// during the migration.
@Kind(name: 'MetricPoint', idType: IdType.String)
class LegacyMetricPointModel extends Model<String> {
LegacyMetricPointModel({MetricPoint fromMetricPoint}) {
if (fromMetricPoint != null) {
id = fromMetricPoint.id;
value = fromMetricPoint.value;
originId = 'legacy-flutter';
sourceTimeMicros = null;
tags = fromMetricPoint.tags.keys
.map((String key) =>
jsonEncode(<String, dynamic>{key: fromMetricPoint.tags[key]}))
.toList();
}
}
@DoubleProperty(required: true, indexed: false)
double value;
@StringListProperty()
List<String> tags;
@StringProperty(required: true)
String originId;
@IntProperty(propertyName: kSourceTimeMicrosName)
int sourceTimeMicros;
}
class LegacyFlutterDestination extends MetricDestination {
LegacyFlutterDestination(this._db);
static Future<LegacyFlutterDestination> makeFromCredentialsJson(
Map<String, dynamic> json) async {
return LegacyFlutterDestination(await datastoreFromCredentialsJson(json));
}
static LegacyFlutterDestination makeFromAccessToken(
String accessToken, String projectId) {
return LegacyFlutterDestination(
datastoreFromAccessToken(accessToken, projectId));
}
@override
Future<void> update(List<MetricPoint> points) async {
final List<LegacyMetricPointModel> flutterCenterPoints =
points.map((MetricPoint p) => LegacyMetricPointModel(fromMetricPoint: p)).toList();
for (int start = 0; start < points.length; start += kMaxBatchSize) {
final int end = min(start + kMaxBatchSize, points.length);
await _db.withTransaction((Transaction tx) async {
tx.queueMutations(inserts: flutterCenterPoints.sublist(start, end));
await tx.commit();
});
}
}
final DatastoreDB _db;
}
dev/benchmarks/metrics_center/lib/src/skiaperf.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:convert';
import 'package:gcloud/storage.dart';
import 'package:googleapis/storage/v1.dart' show DetailedApiRequestError;
import 'package:googleapis_auth/auth.dart';
import 'package:googleapis_auth/auth_io.dart';
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/gcs_lock.dart';
import 'package:metrics_center/src/github_helper.dart';
// Skia Perf Format is a JSON file that looks like:
// {
// "gitHash": "fe4a4029a080bc955e9588d05a6cd9eb490845d4",
// "key": {
// "arch": "x86",
// "gpu": "GTX660",
// "model": "ShuttleA",
// "os": "Ubuntu12"
// },
// "results": {
// "ChunkAlloc_PushPop_640_480": {
// "nonrendering": {
// "min_ms": 0.01485466666666667,
// "options": {
// "source_type": "bench"
// }
// }
// },
// "DeferredSurfaceCopy_discardable_640_480": {
// "565": {
// "min_ms": 2.215988,
// "options": {
// "source_type": "bench"
// }
// },
// ...
class SkiaPerfPoint extends MetricPoint {
SkiaPerfPoint._(this.githubRepo, this.gitHash, this.testName, this.subResult,
double value, this._options, this.jsonUrl)
: assert(_options[kGithubRepoKey] == null),
assert(_options[kGitRevisionKey] == null),
assert(_options[kNameKey] == null),
super(
value,
<String, String>{}
..addAll(_options)
..addAll(<String, String>{
kGithubRepoKey: githubRepo,
kGitRevisionKey: gitHash,
kNameKey: testName,
kSubResultKey: subResult,
}),
) {
assert(tags[kGithubRepoKey] != null);
assert(tags[kGitRevisionKey] != null);
assert(tags[kNameKey] != null);
}
/// Construct [SkiaPerfPoint] from a well-formed [MetricPoint].
///
/// The [MetricPoint] must have [kGithubRepoKey], [kGitRevisionKey],
/// [kNameKey] in its tags for this to be successful.
///
/// If the [MetricPoint] has a tag 'date', that tag will be removed so Skia
/// perf can plot multiple metrics with different date as a single trace.
/// Skia perf will use the git revision's date instead of this date tag in
/// the time axis.
factory SkiaPerfPoint.fromPoint(MetricPoint p) {
final String githubRepo = p.tags[kGithubRepoKey];
final String gitHash = p.tags[kGitRevisionKey];
final String name = p.tags[kNameKey];
if (githubRepo == null || gitHash == null || name == null) {
throw '$kGithubRepoKey, $kGitRevisionKey, $kNameKey must be set in'
' the tags of $p.';
}
final String subResult = p.tags[kSubResultKey] ?? kSkiaPerfValueKey;
final Map<String, String> options = <String, String>{}..addEntries(
p.tags.entries.where(
(MapEntry<String, dynamic> entry) =>
entry.key != kGithubRepoKey &&
entry.key != kGitRevisionKey &&
entry.key != kNameKey &&
entry.key != kSubResultKey &&
// https://github.com/google/benchmark automatically generates a
// 'date' field. If it's included in options, the Skia perf won't
// be able to connect different points in a single trace because
// the date is always different.
entry.key != 'date',
),
);
return SkiaPerfPoint._(
githubRepo, gitHash, name, subResult, p.value, options, null);
}
/// In the format of '<owner>/<name>' such as 'flutter/flutter' or
/// 'flutter/engine'.
final String githubRepo;
/// SHA such as 'ad20d368ffa09559754e4b2b5c12951341ca3b2d'
final String gitHash;
/// For Flutter devicelab, this is the task name (e.g.,
/// 'flutter_gallery__transition_perf'); for Google benchmark, this is the
/// benchmark name (e.g., 'BM_ShellShutdown').
///
/// In Skia perf web dashboard, this value can be queried and filtered by
/// "test".
final String testName;
/// The name of "subResult" comes from the special treatment of "sub_result"
/// in SkiaPerf. If not provided, its value will be set to kSkiaPerfValueKey.
///
/// When Google benchmarks are converted to SkiaPerfPoint, this subResult
/// could be "cpu_time" or "real_time".
///
/// When devicelab benchmarks are converted to SkiaPerfPoint, this subResult
/// is often the metric name such as "average_frame_build_time_millis" whereas
/// the [testName] is the benchmark or task name such as
/// "flutter_gallery__transition_perf".
final String subResult;
/// The url to the Skia perf json file in the Google Cloud Storage bucket.
///
/// This can be null if the point has been stored in the bucket yet.
final String jsonUrl;
Map<String, dynamic> _toSubResultJson() {
return <String, dynamic>{
subResult: value,
kSkiaPerfOptionsKey: _options,
};
}
/// Convert a list of SkiaPoints with the same git repo and git revision into
/// a single json file in the Skia perf format.
///
/// The list must be non-empty.
static Map<String, dynamic> toSkiaPerfJson(List<SkiaPerfPoint> points) {
assert(points.isNotEmpty);
assert(() {
for (final SkiaPerfPoint p in points) {
if (p.githubRepo != points[0].githubRepo ||
p.gitHash != points[0].gitHash) {
return false;
}
}
return true;
}(), 'All points must have same githubRepo and gitHash');
final Map<String, dynamic> results = <String, dynamic>{};
for (final SkiaPerfPoint p in points) {
final Map<String, dynamic> subResultJson = p._toSubResultJson();
if (results[p.testName] == null) {
results[p.testName] = <String, dynamic>{
kSkiaPerfDefaultConfig: subResultJson,
};
} else {
// Flutter currently doesn't support having the same name but different
// options/configurations. If this actually happens in the future, we
// probably can use different values of config (currently there's only
// one kSkiaPerfDefaultConfig) to resolve the conflict.
assert(results[p.testName][kSkiaPerfDefaultConfig][kSkiaPerfOptionsKey]
.toString() ==
subResultJson[kSkiaPerfOptionsKey].toString());
assert(
results[p.testName][kSkiaPerfDefaultConfig][p.subResult] == null);
results[p.testName][kSkiaPerfDefaultConfig][p.subResult] = p.value;
}
}
return <String, dynamic>{
kSkiaPerfGitHashKey: points[0].gitHash,
kSkiaPerfResultsKey: results,
};
}
// Equivalent to tags without git repo, git hash, and name because those two
// are already stored somewhere else.
final Map<String, String> _options;
}
/// Handle writing and updates of Skia perf GCS buckets.
class SkiaPerfGcsAdaptor {
/// Construct the adaptor given the associated GCS bucket where the data is
/// read from and written to.
SkiaPerfGcsAdaptor(this._gcsBucket) : assert(_gcsBucket != null);
/// Used by Skia to differentiate json file format versions.
static const int version = 1;
/// Write a list of SkiaPerfPoint into a GCS file with name `objectName` in
/// the proper json format that's understandable by Skia perf services.
///
/// The `objectName` must be a properly formatted string returned by
/// [computeObjectName].
///
/// The read may retry multiple times if transient network errors with code
/// 504 happens.
Future<void> writePoints(
String objectName, List<SkiaPerfPoint> points) async {
final String jsonString = jsonEncode(SkiaPerfPoint.toSkiaPerfJson(points));
final List<int> content = utf8.encode(jsonString);
// Retry multiple times as GCS may return 504 timeout.
for (int retry = 0; retry < 5; retry += 1) {
try {
await _gcsBucket.writeBytes(objectName, content);
return;
} catch (e) {
if (e is DetailedApiRequestError && e.status == 504) {
continue;
}
rethrow;
}
}
// Retry one last time and let the exception go through.
await _gcsBucket.writeBytes(objectName, content);
}
/// Read a list of `SkiaPerfPoint` that have been previously written to the
/// GCS file with name `objectName`.
///
/// The Github repo and revision of those points will be inferred from the
/// `objectName`.
///
/// Return an empty list if the object does not exist in the GCS bucket.
///
/// The read may retry multiple times if transient network errors with code
/// 504 happens.
Future<List<SkiaPerfPoint>> readPoints(String objectName) async {
// Retry multiple times as GCS may return 504 timeout.
for (int retry = 0; retry < 5; retry += 1) {
try {
return await _readPointsWithoutRetry(objectName);
} catch (e) {
if (e is DetailedApiRequestError && e.status == 504) {
continue;
}
rethrow;
}
}
// Retry one last time and let the exception go through.
return await _readPointsWithoutRetry(objectName);
}
Future<List<SkiaPerfPoint>> _readPointsWithoutRetry(String objectName) async {
ObjectInfo info;
try {
info = await _gcsBucket.info(objectName);
} catch (e) {
if (e.toString().contains('No such object')) {
return <SkiaPerfPoint>[];
} else {
rethrow;
}
}
final Stream<List<int>> stream = _gcsBucket.read(objectName);
final Stream<int> byteStream = stream.expand((List<int> x) => x);
final Map<String, dynamic> decodedJson =
jsonDecode(utf8.decode(await byteStream.toList()))
as Map<String, dynamic>;
final List<SkiaPerfPoint> points = <SkiaPerfPoint>[];
final String firstGcsNameComponent = objectName.split('/')[0];
_populateGcsNameToGithubRepoMapIfNeeded();
final String githubRepo = _gcsNameToGithubRepo[firstGcsNameComponent];
assert(githubRepo != null);
final String gitHash = decodedJson[kSkiaPerfGitHashKey] as String;
final Map<String, dynamic> results =
decodedJson[kSkiaPerfResultsKey] as Map<String, dynamic>;
for (final String name in results.keys) {
final Map<String, dynamic> subResultMap =
results[name][kSkiaPerfDefaultConfig] as Map<String, dynamic>;
for (final String subResult
in subResultMap.keys.where((String s) => s != kSkiaPerfOptionsKey)) {
points.add(SkiaPerfPoint._(
githubRepo,
gitHash,
name,
subResult,
subResultMap[subResult] as double,
(subResultMap[kSkiaPerfOptionsKey] as Map<String, dynamic>)
.cast<String, String>(),
info.downloadLink.toString(),
));
}
}
return points;
}
/// Compute the GCS file name that's used to store metrics for a given commit
/// (git revision).
///
/// Skia perf needs all directory names to be well formatted. The final name
/// of the json file (currently `values.json`) can be arbitrary, and multiple
/// json files can be put in that leaf directory. We intend to use multiple
/// json files in the future to scale up the system if too many writes are
/// competing for the same json file.
static Future<String> computeObjectName(String githubRepo, String revision,
{GithubHelper githubHelper}) async {
assert(_githubRepoToGcsName[githubRepo] != null);
final String topComponent = _githubRepoToGcsName[githubRepo];
final DateTime t = await (githubHelper ?? GithubHelper())
.getCommitDateTime(githubRepo, revision);
final String month = t.month.toString().padLeft(2, '0');
final String day = t.day.toString().padLeft(2, '0');
final String hour = t.hour.toString().padLeft(2, '0');
final String dateComponents = '${t.year}/$month/$day/$hour';
return '$topComponent/$dateComponents/$revision/values.json';
}
static final Map<String, String> _githubRepoToGcsName = <String, String>{
kFlutterFrameworkRepo: 'flutter-flutter',
kFlutterEngineRepo: 'flutter-engine',
};
static final Map<String, String> _gcsNameToGithubRepo = <String, String>{};
static void _populateGcsNameToGithubRepoMapIfNeeded() {
if (_gcsNameToGithubRepo.isEmpty) {
for (final String repo in _githubRepoToGcsName.keys) {
final String gcsName = _githubRepoToGcsName[repo];
assert(_gcsNameToGithubRepo[gcsName] == null);
_gcsNameToGithubRepo[gcsName] = repo;
}
}
}
final Bucket _gcsBucket;
}
class SkiaPerfDestination extends MetricDestination {
SkiaPerfDestination(this._gcs, this._lock);
static const String kBucketName = 'flutter-skia-perf-prod';
static const String kTestBucketName = 'flutter-skia-perf-test';
/// Create from a full credentials json (of a service account).
static Future<SkiaPerfDestination> makeFromGcpCredentials(
Map<String, dynamic> credentialsJson,
{bool isTesting = false}) async {
final AutoRefreshingAuthClient client = await clientViaServiceAccount(
ServiceAccountCredentials.fromJson(credentialsJson), Storage.SCOPES);
return make(
client,
credentialsJson[kProjectId] as String,
isTesting: isTesting,
);
}
/// Create from an access token and its project id.
static Future<SkiaPerfDestination> makeFromAccessToken(
String token, String projectId,
{bool isTesting = false}) async {
final AuthClient client = authClientFromAccessToken(token, Storage.SCOPES);
return make(client, projectId, isTesting: isTesting);
}
/// Create from an [AuthClient] and a GCP project id.
///
/// [AuthClient] can be obtained from functions like `clientViaUserConsent`.
static Future<SkiaPerfDestination> make(AuthClient client, String projectId,
{bool isTesting = false}) async {
final Storage storage = Storage(client, projectId);
final String bucketName = isTesting ? kTestBucketName : kBucketName;
if (!await storage.bucketExists(bucketName)) {
throw 'Bucket $kBucketName does not exist.';
}
final SkiaPerfGcsAdaptor adaptor =
SkiaPerfGcsAdaptor(storage.bucket(bucketName));
final GcsLock lock = GcsLock(client, bucketName);
return SkiaPerfDestination(adaptor, lock);
}
@override
Future<void> update(List<MetricPoint> points) async {
// 1st, create a map based on git repo, git revision, and point id. Git repo
// and git revision are the top level components of the Skia perf GCS object
// name.
final Map<String, Map<String, Map<String, SkiaPerfPoint>>> pointMap =
<String, Map<String, Map<String, SkiaPerfPoint>>>{};
for (final SkiaPerfPoint p
in points.map((MetricPoint x) => SkiaPerfPoint.fromPoint(x))) {
if (p != null) {
pointMap[p.githubRepo] ??= <String, Map<String, SkiaPerfPoint>>{};
pointMap[p.githubRepo][p.gitHash] ??= <String, SkiaPerfPoint>{};
pointMap[p.githubRepo][p.gitHash][p.id] = p;
}
}
// 2nd, read existing points from the gcs object and update with new ones.
for (final String repo in pointMap.keys) {
for (final String revision in pointMap[repo].keys) {
final String objectName =
await SkiaPerfGcsAdaptor.computeObjectName(repo, revision);
final Map<String, SkiaPerfPoint> newPoints = pointMap[repo][revision];
// If too many bots are writing the metrics of a git revision into this
// single json file (with name `objectName`), the contention on the lock
// might be too high. In that case, break the json file into multiple
// json files according to bot names or task names. Skia perf read all
// json files in the directory so one can use arbitrary names for those
// sharded json file names.
_lock.protectedRun('$objectName.lock', () async {
final List<SkiaPerfPoint> oldPoints =
await _gcs.readPoints(objectName);
for (final SkiaPerfPoint p in oldPoints) {
if (newPoints[p.id] == null) {
newPoints[p.id] = p;
}
}
await _gcs.writePoints(objectName, newPoints.values.toList());
});
}
}
}
final SkiaPerfGcsAdaptor _gcs;
final GcsLock _lock;
}
const String kSkiaPerfGitHashKey = 'gitHash';
const String kSkiaPerfResultsKey = 'results';
const String kSkiaPerfValueKey = 'value';
const String kSkiaPerfOptionsKey = 'options';
const String kSkiaPerfDefaultConfig = 'default';
dev/benchmarks/metrics_center/pubspec.yaml deleted 100644 → 0
View file @ 120966f9
name: metrics_center
version: 0.0.4
description:
Support multiple performance metrics sources/formats and destinations.
homepage:
https://github.com/flutter/flutter/tree/master/dev/benchmarks/metrics_center
environment:
sdk: '>=2.10.0 <3.0.0'
dependencies:
args: 1.6.0
crypto: 2.1.5
gcloud: 0.7.3
googleapis_auth: 0.2.12
github: 7.0.4
equatable: 1.2.5
_discoveryapis_commons: 0.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
charcode: 1.2.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
collection: 1.15.0-nullsafety.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
convert: 2.1.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
googleapis: 0.56.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http: 0.12.2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_parser: 3.1.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
json_annotation: 3.1.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
meta: 1.3.0-nullsafety.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
path: 1.8.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
source_span: 1.8.0-nullsafety.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
string_scanner: 1.1.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
term_glyph: 1.2.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
typed_data: 1.3.0-nullsafety.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
dev_dependencies:
test: 1.16.0-nullsafety.9
pedantic: 1.10.0-nullsafety.3
mockito: 4.1.1
fake_async: 1.2.0-nullsafety.3
_fe_analyzer_shared: 12.0.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.40.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
async: 2.5.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 2.1.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
cli_util: 0.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
clock: 1.1.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
coverage: 0.14.2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_multi_server: 2.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
io: 0.3.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.3-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.10-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
mime: 0.9.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
node_interop: 1.2.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
node_io: 1.1.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
node_preamble: 1.4.12 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
package_config: 1.9.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
pool: 1.5.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
pub_semver: 1.4.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
shelf: 0.7.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
shelf_packages_handler: 2.0.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
shelf_static: 0.2.9+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
shelf_web_socket: 0.2.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
source_map_stack_trace: 2.1.0-nullsafety.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
source_maps: 0.10.10-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
stack_trace: 1.10.0-nullsafety.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
stream_channel: 2.1.0-nullsafety.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
test_api: 0.2.19-nullsafety.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
test_core: 0.3.12-nullsafety.9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
vm_service: 5.5.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
watcher: 0.9.7+15 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
web_socket_channel: 1.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
webkit_inspection_protocol: 0.7.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
yaml: 2.2.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
# PUBSPEC CHECKSUM: 0219
dev/benchmarks/metrics_center/test/common.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:io';
import 'package:test/test.dart' hide TypeMatcher, isInstanceOf;
import 'package:test/test.dart' as test_package show TypeMatcher;
export 'package:test/test.dart' hide TypeMatcher, isInstanceOf;
// Defines a 'package:test' shim.
// TODO(ianh): Remove this file once https://github.com/dart-lang/matcher/issues/98 is fixed
/// A matcher that compares the type of the actual value to the type argument T.
test_package.TypeMatcher<T> isInstanceOf<T>() => isA<T>();
void tryToDelete(Directory directory) {
// This should not be necessary, but it turns out that
// on Windows it's common for deletions to fail due to
// bogus (we think) "access denied" errors.
try {
directory.deleteSync(recursive: true);
} on FileSystemException catch (error) {
print('Failed to delete ${directory.path}: $error');
}
}
dev/benchmarks/metrics_center/test/example_google_benchmark.json deleted 100644 → 0
View file @ 120966f9
{
"context": {
"date": "2019-12-17 15:14:14",
"num_cpus": 56,
"mhz_per_cpu": 2594,
"cpu_scaling_enabled": true,
"library_build_type": "release"
},
"benchmarks": [
{
"name": "BM_PaintRecordInit",
"iterations": 6749079,
"real_time": 101,
"cpu_time": 101,
"time_unit": "ns"
},
{
"name": "BM_ParagraphShortLayout",
"iterations": 151761,
"real_time": 4460,
"cpu_time": 4460,
"time_unit": "ns"
},
{
"name": "BM_ParagraphStylesBigO_BigO",
"cpu_coefficient": 6548,
"real_coefficient": 6548,
"big_o": "N",
"time_unit": "ns"
}
]
}
dev/benchmarks/metrics_center/test/flutter_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/flutter.dart';
import 'common.dart';
void main() {
test('FlutterEngineMetricPoint works.', () {
const String gitRevision = 'ca799fa8b2254d09664b78ee80c43b434788d112';
final FlutterEngineMetricPoint simplePoint = FlutterEngineMetricPoint(
'BM_ParagraphLongLayout',
287235,
gitRevision,
);
expect(simplePoint.value, equals(287235));
expect(simplePoint.tags[kGithubRepoKey], kFlutterEngineRepo);
expect(simplePoint.tags[kGitRevisionKey], gitRevision);
expect(simplePoint.tags[kNameKey], 'BM_ParagraphLongLayout');
final FlutterEngineMetricPoint detailedPoint = FlutterEngineMetricPoint(
'BM_ParagraphLongLayout',
287224,
'ca799fa8b2254d09664b78ee80c43b434788d112',
moreTags: const <String, String>{
'executable': 'txt_benchmarks',
'sub_result': 'CPU',
kUnitKey: 'ns',
},
);
expect(detailedPoint.value, equals(287224));
expect(detailedPoint.tags['executable'], equals('txt_benchmarks'));
expect(detailedPoint.tags['sub_result'], equals('CPU'));
expect(detailedPoint.tags[kUnitKey], equals('ns'));
});
}
\ No newline at end of file
dev/benchmarks/metrics_center/test/gcs_lock_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:async';
import 'package:googleapis/storage/v1.dart';
import 'package:fake_async/fake_async.dart';
import 'package:gcloud/storage.dart';
import 'package:googleapis_auth/auth_io.dart';
import 'package:metrics_center/src/gcs_lock.dart';
import 'package:metrics_center/src/skiaperf.dart';
import 'package:mockito/mockito.dart';
import 'common.dart';
import 'utility.dart';
enum TestPhase {
run1,
run2,
}
class MockClient extends Mock implements AuthClient {}
void main() {
const Duration kDelayStep = Duration(milliseconds: 10);
final Map<String, dynamic> credentialsJson = getTestGcpCredentialsJson();
test('GcsLock prints warnings for long waits', () {
// Capture print to verify error messages.
final List<String> prints = <String>[];
final ZoneSpecification spec =
ZoneSpecification(print: (_, __, ___, String msg) => prints.add(msg));
Zone.current.fork(specification: spec).run<void>(() {
fakeAsync((FakeAsync fakeAsync) {
final MockClient mockClient = MockClient();
final GcsLock lock = GcsLock(mockClient, 'mockBucket');
when(mockClient.send(any)).thenThrow(DetailedApiRequestError(412, ''));
final Future<void> runFinished =
lock.protectedRun('mock.lock', () async {});
fakeAsync.elapse(const Duration(seconds: 10));
when(mockClient.send(any)).thenThrow(AssertionError('Stop!'));
runFinished.catchError((dynamic e) {
final AssertionError error = e as AssertionError;
expect(error.message, 'Stop!');
print('${error.message}');
});
fakeAsync.elapse(const Duration(seconds: 20));
});
});
const String kExpectedErrorMessage = 'The lock is waiting for a long time: '
'0:00:10.240000. If the lock file mock.lock in bucket mockBucket '
'seems to be stuck (i.e., it was created a long time ago and no one '
'seems to be owning it currently), delete it manually to unblock this.';
expect(prints, equals(<String>[kExpectedErrorMessage, 'Stop!']));
});
test('GcsLock integration test: single protectedRun is successful', () async {
final AutoRefreshingAuthClient client = await clientViaServiceAccount(
ServiceAccountCredentials.fromJson(credentialsJson), Storage.SCOPES);
final GcsLock lock = GcsLock(client, SkiaPerfDestination.kTestBucketName);
int testValue = 0;
await lock.protectedRun('test.lock', () async {
testValue = 1;
});
expect(testValue, 1);
}, skip: credentialsJson == null);
test('GcsLock integration test: protectedRun is exclusive', () async {
final AutoRefreshingAuthClient client = await clientViaServiceAccount(
ServiceAccountCredentials.fromJson(credentialsJson), Storage.SCOPES);
final GcsLock lock1 = GcsLock(client, SkiaPerfDestination.kTestBucketName);
final GcsLock lock2 = GcsLock(client, SkiaPerfDestination.kTestBucketName);
TestPhase phase = TestPhase.run1;
final Completer<void> started1 = Completer<void>();
final Future<void> finished1 = lock1.protectedRun('test.lock', () async {
started1.complete();
while (phase == TestPhase.run1) {
await Future<void>.delayed(kDelayStep);
}
});
await started1.future;
final Completer<void> started2 = Completer<void>();
final Future<void> finished2 = lock2.protectedRun('test.lock', () async {
started2.complete();
});
// started2 should not be set even after a long wait because lock1 is
// holding the GCS lock file.
await Future<void>.delayed(kDelayStep * 10);
expect(started2.isCompleted, false);
// When phase is switched to run2, lock1 should be released soon and
// lock2 should soon be able to proceed its protectedRun.
phase = TestPhase.run2;
await started2.future;
await finished1;
await finished2;
}, skip: credentialsJson == null);
}
dev/benchmarks/metrics_center/test/github_helper_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
@Timeout(Duration(seconds: 3600))
import 'package:metrics_center/src/github_helper.dart';
import 'common.dart';
void main() {
test('GithubHelper gets correct commit date time', () async {
final GithubHelper helper = GithubHelper();
expect(
await helper.getCommitDateTime(
'flutter/flutter',
'ad20d368ffa09559754e4b2b5c12951341ca3b2d',
),
equals(DateTime.parse('2019-12-06 03:33:01.000Z')),
);
});
test('GithubHelper is a singleton', () {
final GithubHelper helper1 = GithubHelper();
final GithubHelper helper2 = GithubHelper();
expect(helper1, equals(helper2));
});
test('GithubHelper can query the same commit 1000 times within 1 second',
() async {
final DateTime start = DateTime.now();
for (int i = 0; i < 1000; i += 1) {
await GithubHelper().getCommitDateTime(
'flutter/flutter',
'ad20d368ffa09559754e4b2b5c12951341ca3b2d',
);
}
final Duration duration = DateTime.now().difference(start);
expect(duration, lessThan(const Duration(seconds: 1)));
});
}
dev/benchmarks/metrics_center/test/google_benchmark_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/google_benchmark.dart';
import 'common.dart';
import 'utility.dart';
void main() {
test('GoogleBenchmarkParser parses example json.', () async {
final List<MetricPoint> points =
await GoogleBenchmarkParser.parse('test/example_google_benchmark.json');
expect(points.length, 6);
expectSetMatch(
points.map((MetricPoint p) => p.value),
<int>[101, 101, 4460, 4460, 6548, 6548],
);
expectSetMatch(
points.map((MetricPoint p) => p.tags[kSubResultKey]),
<String>[
'cpu_time',
'real_time',
'cpu_coefficient',
'real_coefficient',
],
);
expectSetMatch(
points.map((MetricPoint p) => p.tags[kNameKey]),
<String>[
'BM_PaintRecordInit',
'BM_ParagraphShortLayout',
'BM_ParagraphStylesBigO_BigO',
],
);
});
}
dev/benchmarks/metrics_center/test/legacy_flutter_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:gcloud/src/datastore_impl.dart';
import 'package:googleapis_auth/auth_io.dart';
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/legacy_flutter.dart';
import 'common.dart';
import 'utility.dart';
const String kTestSourceId = 'test';
void main() {
final Map<String, dynamic> credentialsJson = getTestGcpCredentialsJson();
test(
'LegacyFlutterDestination integration test: '
'update does not crash.', () async {
final LegacyFlutterDestination dst =
await LegacyFlutterDestination.makeFromCredentialsJson(credentialsJson);
await dst.update(<MetricPoint>[MetricPoint(1.0, const <String, String>{})]);
}, skip: credentialsJson == null);
test(
'LegacyFlutterDestination integration test: '
'can update with an access token.', () async {
final AutoRefreshingAuthClient client = await clientViaServiceAccount(
ServiceAccountCredentials.fromJson(credentialsJson),
DatastoreImpl.SCOPES,
);
final String token = client.credentials.accessToken.data;
final LegacyFlutterDestination dst =
LegacyFlutterDestination.makeFromAccessToken(
token,
credentialsJson[kProjectId] as String,
);
await dst.update(<MetricPoint>[MetricPoint(1.0, const <String, String>{})]);
}, skip: credentialsJson == null);
}
dev/benchmarks/metrics_center/test/skiaperf_test.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
@Timeout(Duration(seconds: 3600))
import 'dart:convert';
import 'package:gcloud/storage.dart';
import 'package:googleapis/storage/v1.dart' show DetailedApiRequestError;
import 'package:googleapis_auth/auth_io.dart';
import 'package:metrics_center/src/gcs_lock.dart';
import 'package:metrics_center/src/github_helper.dart';
import 'package:mockito/mockito.dart';
import 'package:metrics_center/src/common.dart';
import 'package:metrics_center/src/flutter.dart';
import 'package:metrics_center/src/skiaperf.dart';
import 'common.dart';
import 'utility.dart';
class MockBucket extends Mock implements Bucket {}
class MockObjectInfo extends Mock implements ObjectInfo {}
class MockGithubHelper extends Mock implements GithubHelper {}
class MockGcsLock implements GcsLock {
@override
Future<void> protectedRun(
String exclusiveObjectName, Future<void> Function() f) async {
await f();
}
}
class MockSkiaPerfGcsAdaptor implements SkiaPerfGcsAdaptor {
@override
Future<List<SkiaPerfPoint>> readPoints(String objectName) async {
return _storage[objectName] ?? <SkiaPerfPoint>[];
}
@override
Future<void> writePoints(
String objectName, List<SkiaPerfPoint> points) async {
_storage[objectName] = points.toList();
}
// Map from the object name to the list of SkiaPoint that mocks the GCS.
final Map<String, List<SkiaPerfPoint>> _storage =
<String, List<SkiaPerfPoint>>{};
}
Future<void> main() async {
const double kValue1 = 1.0;
const double kValue2 = 2.0;
const double kValue3 = 3.0;
const String kFrameworkRevision1 = '9011cece2595447eea5dd91adaa241c1c9ef9a33';
const String kFrameworkRevision2 = '372fe290e4d4f3f97cbf02a57d235771a9412f10';
const String kEngineRevision1 = '617938024315e205f26ed72ff0f0647775fa6a71';
const String kEngineRevision2 = '5858519139c22484aaff1cf5b26bdf7951259344';
const String kTaskName = 'analyzer_benchmark';
const String kMetric1 = 'flutter_repo_batch_maximum';
const String kMetric2 = 'flutter_repo_watch_maximum';
final MetricPoint cocoonPointRev1Metric1 = MetricPoint(
kValue1,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision1,
kNameKey: kTaskName,
kSubResultKey: kMetric1,
kUnitKey: 's',
},
);
final MetricPoint cocoonPointRev1Metric2 = MetricPoint(
kValue2,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision1,
kNameKey: kTaskName,
kSubResultKey: kMetric2,
kUnitKey: 's',
},
);
final MetricPoint cocoonPointRev2Metric1 = MetricPoint(
kValue3,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision2,
kNameKey: kTaskName,
kSubResultKey: kMetric1,
kUnitKey: 's',
},
);
final MetricPoint cocoonPointBetaRev1Metric1 = MetricPoint(
kValue1,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision1,
kNameKey: 'beta/$kTaskName',
kSubResultKey: kMetric1,
kUnitKey: 's',
'branch': 'beta',
},
);
final MetricPoint cocoonPointBetaRev1Metric1BadBranch = MetricPoint(
kValue1,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision1,
kNameKey: kTaskName,
kSubResultKey: kMetric1,
kUnitKey: 's',
// If we only add this 'branch' tag without changing the test or sub-result name, an exception
// would be thrown as Skia Perf currently only supports the same set of tags for a pair of
// kNameKey and kSubResultKey values. So to support branches, one also has to add the branch
// name to the test name.
'branch': 'beta',
},
);
const String engineMetricName = 'BM_PaintRecordInit';
const String engineRevision = 'ca799fa8b2254d09664b78ee80c43b434788d112';
const double engineValue1 = 101;
const double engineValue2 = 102;
final FlutterEngineMetricPoint enginePoint1 = FlutterEngineMetricPoint(
engineMetricName,
engineValue1,
engineRevision,
moreTags: const <String, String>{
kSubResultKey: 'cpu_time',
kUnitKey: 'ns',
'date': '2019-12-17 15:14:14',
'num_cpus': '56',
'mhz_per_cpu': '2594',
'cpu_scaling_enabled': 'true',
'library_build_type': 'release',
},
);
final FlutterEngineMetricPoint enginePoint2 = FlutterEngineMetricPoint(
engineMetricName,
engineValue2,
engineRevision,
moreTags: const <String, String>{
kSubResultKey: 'real_time',
kUnitKey: 'ns',
'date': '2019-12-17 15:14:14',
'num_cpus': '56',
'mhz_per_cpu': '2594',
'cpu_scaling_enabled': 'true',
'library_build_type': 'release',
},
);
test('Throw if invalid points are converted to SkiaPoint', () {
final MetricPoint noGithubRepoPoint = MetricPoint(
kValue1,
const <String, String>{
kGitRevisionKey: kFrameworkRevision1,
kNameKey: kTaskName,
},
);
final MetricPoint noGitRevisionPoint = MetricPoint(
kValue1,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kNameKey: kTaskName,
},
);
final MetricPoint noTestNamePoint = MetricPoint(
kValue1,
const <String, String>{
kGithubRepoKey: kFlutterFrameworkRepo,
kGitRevisionKey: kFrameworkRevision1,
},
);
expect(() => SkiaPerfPoint.fromPoint(noGithubRepoPoint), throwsA(anything));
expect(
() => SkiaPerfPoint.fromPoint(noGitRevisionPoint), throwsA(anything));
expect(() => SkiaPerfPoint.fromPoint(noTestNamePoint), throwsA(anything));
});
test('Correctly convert a metric point from cocoon to SkiaPoint', () {
final SkiaPerfPoint skiaPoint1 =
SkiaPerfPoint.fromPoint(cocoonPointRev1Metric1);
expect(skiaPoint1, isNotNull);
expect(skiaPoint1.testName, equals(kTaskName));
expect(skiaPoint1.subResult, equals(kMetric1));
expect(skiaPoint1.value, equals(cocoonPointRev1Metric1.value));
expect(skiaPoint1.jsonUrl, isNull); // Not inserted yet
});
test('Cocoon points correctly encode into Skia perf json format', () {
final SkiaPerfPoint p1 = SkiaPerfPoint.fromPoint(cocoonPointRev1Metric1);
final SkiaPerfPoint p2 = SkiaPerfPoint.fromPoint(cocoonPointRev1Metric2);
final SkiaPerfPoint p3 =
SkiaPerfPoint.fromPoint(cocoonPointBetaRev1Metric1);
const JsonEncoder encoder = JsonEncoder.withIndent(' ');
expect(
encoder
.convert(SkiaPerfPoint.toSkiaPerfJson(<SkiaPerfPoint>[p1, p2, p3])),
equals('''
{
"gitHash": "9011cece2595447eea5dd91adaa241c1c9ef9a33",
"results": {
"analyzer_benchmark": {
"default": {
"flutter_repo_batch_maximum": 1.0,
"options": {
"unit": "s"
},
"flutter_repo_watch_maximum": 2.0
}
},
"beta/analyzer_benchmark": {
"default": {
"flutter_repo_batch_maximum": 1.0,
"options": {
"branch": "beta",
"unit": "s"
}
}
}
}
}'''));
});
test('Engine metric points correctly encode into Skia perf json format', () {
const JsonEncoder encoder = JsonEncoder.withIndent(' ');
expect(
encoder.convert(SkiaPerfPoint.toSkiaPerfJson(<SkiaPerfPoint>[
SkiaPerfPoint.fromPoint(enginePoint1),
SkiaPerfPoint.fromPoint(enginePoint2),
])),
equals(
'''
{
"gitHash": "ca799fa8b2254d09664b78ee80c43b434788d112",
"results": {
"BM_PaintRecordInit": {
"default": {
"cpu_time": 101.0,
"options": {
"cpu_scaling_enabled": "true",
"library_build_type": "release",
"mhz_per_cpu": "2594",
"num_cpus": "56",
"unit": "ns"
},
"real_time": 102.0
}
}
}
}''',
),
);
});
test(
'Throw if engine points with the same test name but different options are converted to '
'Skia perf points', () {
final FlutterEngineMetricPoint enginePoint1 = FlutterEngineMetricPoint(
'BM_PaintRecordInit',
101,
'ca799fa8b2254d09664b78ee80c43b434788d112',
moreTags: const <String, String>{
kSubResultKey: 'cpu_time',
kUnitKey: 'ns',
'cpu_scaling_enabled': 'true',
},
);
final FlutterEngineMetricPoint enginePoint2 = FlutterEngineMetricPoint(
'BM_PaintRecordInit',
102,
'ca799fa8b2254d09664b78ee80c43b434788d112',
moreTags: const <String, String>{
kSubResultKey: 'real_time',
kUnitKey: 'ns',
'cpu_scaling_enabled': 'false',
},
);
const JsonEncoder encoder = JsonEncoder.withIndent(' ');
expect(
() => encoder.convert(SkiaPerfPoint.toSkiaPerfJson(<SkiaPerfPoint>[
SkiaPerfPoint.fromPoint(enginePoint1),
SkiaPerfPoint.fromPoint(enginePoint2),
])),
throwsA(anything),
);
});
test(
'Throw if two Cocoon metric points with the same name and subResult keys '
'but different options are converted to Skia perf points', () {
final SkiaPerfPoint p1 = SkiaPerfPoint.fromPoint(cocoonPointRev1Metric1);
final SkiaPerfPoint p2 =
SkiaPerfPoint.fromPoint(cocoonPointBetaRev1Metric1BadBranch);
expect(
() => SkiaPerfPoint.toSkiaPerfJson(<SkiaPerfPoint>[p1, p2]),
throwsA(anything),
);
});
test('SkiaPerfGcsAdaptor computes name correctly', () async {
final MockGithubHelper mockHelper = MockGithubHelper();
when(mockHelper.getCommitDateTime(
kFlutterFrameworkRepo, kFrameworkRevision1))
.thenAnswer((_) => Future<DateTime>.value(DateTime(2019, 12, 4, 23)));
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo,
kFrameworkRevision1,
githubHelper: mockHelper,
),
equals('flutter-flutter/2019/12/04/23/$kFrameworkRevision1/values.json'),
);
when(mockHelper.getCommitDateTime(kFlutterEngineRepo, kEngineRevision1))
.thenAnswer((_) => Future<DateTime>.value(DateTime(2019, 12, 3, 20)));
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterEngineRepo,
kEngineRevision1,
githubHelper: mockHelper,
),
equals('flutter-engine/2019/12/03/20/$kEngineRevision1/values.json'),
);
when(mockHelper.getCommitDateTime(kFlutterEngineRepo, kEngineRevision2))
.thenAnswer((_) => Future<DateTime>.value(DateTime(2020, 1, 3, 15)));
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterEngineRepo,
kEngineRevision2,
githubHelper: mockHelper,
),
equals('flutter-engine/2020/01/03/15/$kEngineRevision2/values.json'),
);
});
test('Successfully read mock GCS that fails 1st time with 504', () async {
final MockBucket testBucket = MockBucket();
final SkiaPerfGcsAdaptor skiaPerfGcs = SkiaPerfGcsAdaptor(testBucket);
final String testObjectName = await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1);
final List<SkiaPerfPoint> writePoints = <SkiaPerfPoint>[
SkiaPerfPoint.fromPoint(cocoonPointRev1Metric1),
];
final String skiaPerfJson =
jsonEncode(SkiaPerfPoint.toSkiaPerfJson(writePoints));
await skiaPerfGcs.writePoints(testObjectName, writePoints);
verify(testBucket.writeBytes(testObjectName, utf8.encode(skiaPerfJson)));
// Emulate the first network request to fail with 504.
when(testBucket.info(testObjectName))
.thenThrow(DetailedApiRequestError(504, 'Test Failure'));
final MockObjectInfo mockObjectInfo = MockObjectInfo();
when(mockObjectInfo.downloadLink)
.thenReturn(Uri.https('test.com', 'mock.json'));
when(testBucket.info(testObjectName))
.thenAnswer((_) => Future<ObjectInfo>.value(mockObjectInfo));
when(testBucket.read(testObjectName))
.thenAnswer((_) => Stream<List<int>>.value(utf8.encode(skiaPerfJson)));
final List<SkiaPerfPoint> readPoints =
await skiaPerfGcs.readPoints(testObjectName);
expect(readPoints.length, equals(1));
expect(readPoints[0].testName, kTaskName);
expect(readPoints[0].subResult, kMetric1);
expect(readPoints[0].value, kValue1);
expect(readPoints[0].githubRepo, kFlutterFrameworkRepo);
expect(readPoints[0].gitHash, kFrameworkRevision1);
expect(readPoints[0].jsonUrl, 'https://test.com/mock.json');
});
test('Return empty list if the GCS file does not exist', () async {
final MockBucket testBucket = MockBucket();
final SkiaPerfGcsAdaptor skiaPerfGcs = SkiaPerfGcsAdaptor(testBucket);
final String testObjectName = await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1);
when(testBucket.info(testObjectName))
.thenThrow(Exception('No such object'));
expect((await skiaPerfGcs.readPoints(testObjectName)).length, 0);
});
// The following is for integration tests.
Bucket testBucket;
GcsLock testLock;
final Map<String, dynamic> credentialsJson = getTestGcpCredentialsJson();
if (credentialsJson != null) {
final ServiceAccountCredentials credentials =
ServiceAccountCredentials.fromJson(credentialsJson);
final AutoRefreshingAuthClient client =
await clientViaServiceAccount(credentials, Storage.SCOPES);
final Storage storage =
Storage(client, credentialsJson['project_id'] as String);
const String kTestBucketName = 'flutter-skia-perf-test';
assert(await storage.bucketExists(kTestBucketName));
testBucket = storage.bucket(kTestBucketName);
testLock = GcsLock(client, kTestBucketName);
}
Future<void> skiaPerfGcsAdapterIntegrationTest() async {
final SkiaPerfGcsAdaptor skiaPerfGcs = SkiaPerfGcsAdaptor(testBucket);
final String testObjectName = await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1);
await skiaPerfGcs.writePoints(testObjectName, <SkiaPerfPoint>[
SkiaPerfPoint.fromPoint(cocoonPointRev1Metric1),
SkiaPerfPoint.fromPoint(cocoonPointRev1Metric2),
]);
final List<SkiaPerfPoint> points =
await skiaPerfGcs.readPoints(testObjectName);
expect(points.length, equals(2));
expectSetMatch(
points.map((SkiaPerfPoint p) => p.testName), <String>[kTaskName]);
expectSetMatch(points.map((SkiaPerfPoint p) => p.subResult),
<String>[kMetric1, kMetric2]);
expectSetMatch(
points.map((SkiaPerfPoint p) => p.value), <double>[kValue1, kValue2]);
expectSetMatch(points.map((SkiaPerfPoint p) => p.githubRepo),
<String>[kFlutterFrameworkRepo]);
expectSetMatch(points.map((SkiaPerfPoint p) => p.gitHash),
<String>[kFrameworkRevision1]);
for (int i = 0; i < 2; i += 1) {
expect(points[0].jsonUrl, startsWith('https://'));
}
}
Future<void> skiaPerfGcsIntegrationTestWithEnginePoints() async {
final SkiaPerfGcsAdaptor skiaPerfGcs = SkiaPerfGcsAdaptor(testBucket);
final String testObjectName = await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterEngineRepo, engineRevision);
await skiaPerfGcs.writePoints(testObjectName, <SkiaPerfPoint>[
SkiaPerfPoint.fromPoint(enginePoint1),
SkiaPerfPoint.fromPoint(enginePoint2),
]);
final List<SkiaPerfPoint> points =
await skiaPerfGcs.readPoints(testObjectName);
expect(points.length, equals(2));
expectSetMatch(
points.map((SkiaPerfPoint p) => p.testName),
<String>[engineMetricName, engineMetricName],
);
expectSetMatch(
points.map((SkiaPerfPoint p) => p.value),
<double>[engineValue1, engineValue2],
);
expectSetMatch(
points.map((SkiaPerfPoint p) => p.githubRepo),
<String>[kFlutterEngineRepo],
);
expectSetMatch(
points.map((SkiaPerfPoint p) => p.gitHash), <String>[engineRevision]);
for (int i = 0; i < 2; i += 1) {
expect(points[0].jsonUrl, startsWith('https://'));
}
}
// To run the following integration tests, there must be a valid Google Cloud
// Project service account credentials in secret/test_gcp_credentials.json so
// `testBucket` won't be null. Currently, these integration tests are skipped
// in the CI, and only verified locally.
test(
'SkiaPerfGcsAdaptor passes integration test with Google Cloud Storage',
skiaPerfGcsAdapterIntegrationTest,
skip: testBucket == null,
);
test(
'SkiaPerfGcsAdaptor integration test with engine points',
skiaPerfGcsIntegrationTestWithEnginePoints,
skip: testBucket == null,
);
// `SkiaPerfGcsAdaptor.computeObjectName` uses `GithubHelper` which requires
// network connections. Hence we put them as integration tests instead of unit
// tests.
test(
'SkiaPerfGcsAdaptor integration test for name computations',
() async {
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1),
equals(
'flutter-flutter/2019/12/04/23/$kFrameworkRevision1/values.json'),
);
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterEngineRepo, kEngineRevision1),
equals('flutter-engine/2019/12/03/20/$kEngineRevision1/values.json'),
);
expect(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterEngineRepo, kEngineRevision2),
equals('flutter-engine/2020/01/03/15/$kEngineRevision2/values.json'),
);
},
skip: testBucket == null,
);
test('SkiaPerfDestination correctly updates points', () async {
final SkiaPerfGcsAdaptor mockGcs = MockSkiaPerfGcsAdaptor();
final GcsLock mockLock = MockGcsLock();
final SkiaPerfDestination dst = SkiaPerfDestination(mockGcs, mockLock);
await dst.update(<MetricPoint>[cocoonPointRev1Metric1]);
await dst.update(<MetricPoint>[cocoonPointRev1Metric2]);
List<SkiaPerfPoint> points = await mockGcs.readPoints(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1));
expect(points.length, equals(2));
expectSetMatch(
points.map((SkiaPerfPoint p) => p.testName), <String>[kTaskName]);
expectSetMatch(points.map((SkiaPerfPoint p) => p.subResult),
<String>[kMetric1, kMetric2]);
expectSetMatch(
points.map((SkiaPerfPoint p) => p.value), <double>[kValue1, kValue2]);
final MetricPoint updated =
MetricPoint(kValue3, cocoonPointRev1Metric1.tags);
await dst.update(<MetricPoint>[updated, cocoonPointRev2Metric1]);
points = await mockGcs.readPoints(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision2));
expect(points.length, equals(1));
expect(points[0].gitHash, equals(kFrameworkRevision2));
expect(points[0].value, equals(kValue3));
points = await mockGcs.readPoints(
await SkiaPerfGcsAdaptor.computeObjectName(
kFlutterFrameworkRepo, kFrameworkRevision1));
expectSetMatch(
points.map((SkiaPerfPoint p) => p.value), <double>[kValue2, kValue3]);
});
Future<void> skiaPerfDestinationIntegrationTest() async {
final SkiaPerfDestination destination =
SkiaPerfDestination(SkiaPerfGcsAdaptor(testBucket), testLock);
await destination.update(<MetricPoint>[cocoonPointRev1Metric1]);
}
test(
'SkiaPerfDestination integration test',
skiaPerfDestinationIntegrationTest,
skip: testBucket == null,
);
}
dev/benchmarks/metrics_center/test/utility.dart deleted 100644 → 0
View file @ 120966f9
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:convert';
import 'dart:io';
import 'common.dart';
// This will be used in many of our unit tests.
void expectSetMatch<T>(Iterable<T> actual, Iterable<T> expected) {
expect(Set<T>.from(actual), equals(Set<T>.from(expected)));
}
// May return null if the credentials file doesn't exist.
Map<String, dynamic> getTestGcpCredentialsJson() {
final File f = File('secret/test_gcp_credentials.json');
if (!f.existsSync()) {
return null;
}
return jsonDecode(File('secret/test_gcp_credentials.json').readAsStringSync())
as Map<String, dynamic>;
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

HIAST - GitLab