e2e 0.4.0

  • Readme
  • Changelog
  • Example
  • Installing
  • 99

e2e #

This package enables self-driving testing of Flutter code on devices and emulators. It adapts flutter_test results into a format that is compatible with flutter drive and native Android instrumentation testing.

iOS support is not available yet, but is planned in the future.

Usage #

Add a dependency on the e2e package in the dev_dependencies section of pubspec.yaml. For plugins, do this in the pubspec.yaml of the example app.

Invoke E2EWidgetsFlutterBinding.ensureInitialized() at the start of a test file, e.g.

import 'package:e2e/e2e.dart';

void main() {
  E2EWidgetsFlutterBinding.ensureInitialized();
  testWidgets("failing test example", (WidgetTester tester) async {
    expect(2 + 2, equals(5));
  });
  exit(result == 'pass' ? 0 : 1);
}

Test locations #

It is recommended to put e2e tests in the test/ folder of the app or package. For example apps, if the e2e test references example app code, it should go in example/test/. It is also acceptable to put e2e tests in test_driver/ folder so that they're alongside the runner app (see below).

Using Flutter driver to run tests #

E2EWidgetsTestBinding supports launching the on-device tests with flutter drive. Note that the tests don't use the FlutterDriver API, they use testWidgets instead.

Put the a file named <package_name>_e2e_test.dart in the app' test_driver directory:

import 'dart:async';

import 'package:e2e/e2e_driver.dart' as e2e;

Future<void> main() async => e2e.main();

To run a example app test with Flutter driver:

cd example
flutter drive test/<package_name>_e2e.dart

To test plugin APIs using Flutter driver:

cd example
flutter drive --driver=test_driver/<package_name>_test.dart test/<package_name>_e2e.dart

You can run tests on web in release or profile mode.

First you need to make sure you have downloaded the driver for the browser.

cd example
flutter drive -v --target=test_driver/<package_name>dart -d web-server --release --browser-name=chrome

Android device testing #

Create an instrumentation test file in your application's android/app/src/androidTest/java/com/example/myapp/ directory (replacing com, example, and myapp with values from your app's package name). You can name this test file MainActivityTest.java or another name of your choice.

package com.example.myapp;

import androidx.test.rule.ActivityTestRule;
import dev.flutter.plugins.e2e.FlutterRunner;
import org.junit.Rule;
import org.junit.runner.RunWith;

@RunWith(FlutterTestRunner.class)
public class MainActivityTest {
  @Rule
  public ActivityTestRule<MainActivity> rule = new ActivityTestRule<>(MainActivity.class, true, false);
}

Update your application's myapp/android/app/build.gradle to make sure it uses androidx's version of AndroidJUnitRunner and has androidx libraries as a dependency.

android {
  ...
  defaultConfig {
    ...
    testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
  }
}

dependencies {
    testImplementation 'junit:junit:4.12'

    // https://developer.android.com/jetpack/androidx/releases/test/#1.2.0
    androidTestImplementation 'androidx.test:runner:1.2.0'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
}

To e2e test on a local Android device (emulated or physical):

./gradlew app:connectedAndroidTest -Ptarget=`pwd`/../test_driver/<package_name>_e2e.dart

Firebase Test Lab #

If this is your first time testing with Firebase Test Lab, you'll need to follow the guides in the Firebase test lab documentation to set up a project.

To run an e2e test on Android devices using Firebase Test Lab, use gradle commands to build an instrumentation test for Android.

pushd android
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=<path_to_test>.dart
popd

Upload the build apks Firebase Test Lab, making sure to replace <PATH_TO_KEY_FILE>, <PROJECT_NAME>, <RESULTS_BUCKET>, and <RESULTS_DIRECTORY> with your values.

gcloud auth activate-service-account --key-file=<PATH_TO_KEY_FILE>
gcloud --quiet config set project <PROJECT_NAME>
gcloud firebase test android run --type instrumentation \
  --app build/app/outputs/apk/debug/app-debug.apk \
  --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk\
  --timeout 2m \
  --results-bucket=<RESULTS_BUCKET> \
  --results-dir=<RESULTS_DIRECTORY>

You can pass additional parameters on the command line, such as the devices you want to test on. See gcloud firebase test android run.

iOS device testing #

You need to change iOS/Podfile to avoid test target statically linking to the plugins. One way is to link all of the plugins dynamically:

target 'Runner' do
  use_frameworks!
  ...
end

To e2e test on your iOS device (simulator or real), rebuild your iOS targets with Flutter tool.

flutter build ios -t test_driver/<package_name>_e2e.dart (--simulator)

Open Xcode project (by default, it's ios/Runner.xcodeproj). Create a test target (navigating File > New > Target... and set up the values) and a test file RunnerTests.m and change the code. You can change RunnerTests.m to the name of your choice.

#import <XCTest/XCTest.h>
#import <e2e/E2EIosTest.h>

E2E_IOS_RUNNER(RunnerTests)

Now you can start RunnerTests to kick out e2e tests!

0.4.0 #

  • Breaking change Driver request_data call's response has changed to encapsulate the failure details.
  • Details for failure cases are added: failed method name, stack trace.

0.3.0+1 #

  • Replace deprecated getFlutterEngine call on Android.

0.3.0 #

  • Updates documentation to instruct developers not to launch the activity since we are doing it for them.
  • Renames FlutterRunner to FlutterTestRunner to avoid conflict with Fuchsia.

0.2.4+4 #

  • Fixed a hang that occurred on platforms that don't have a MethodChannel listener registered..

0.2.4+3 #

  • Fixed code snippet in the readme under the "Using Flutter driver to run tests" section.

0.2.4+2 #

  • Make the pedantic dev_dependency explicit.

0.2.4+1 #

  • Registering web service extension for using e2e with web.

0.2.4 #

  • Fixed problem with XCTest in XCode 11.3 where the testing bundles were getting opened multiple times which interfered with the singleton logic for E2EPlugin.

0.2.3+1 #

  • Added a driver test for failure behavior.

0.2.3 #

  • Updates E2EPlugin and add skeleton iOS test case E2EIosTest.
  • Adds instructions to README.md about e2e testing on iOS devices.
  • Adds iOS e2e testing to example.

0.2.2+3 #

  • Remove the deprecated author: field from pubspec.yaml
  • Migrate the plugin to the pubspec platforms manifest.
  • Require Flutter SDK 1.10.0 or greater.

0.2.2+2 #

  • Adds an android dummy project to silence warnings and removes unnecessary .gitignore files.

0.2.2+1 #

  • Fix pedantic lints. Adds a missing await in the example test and some missing documentation.

0.2.2 #

  • Added a stub macos implementation
  • Added a macos example

0.2.1+1 #

  • Updated README.

0.2.1 #

  • Support the v2 Android embedder.
  • Print a warning if the plugin is not registered.
  • Updated method channel name.
  • Set a Flutter minimum SDK version.

0.2.0+1 #

  • Updated README.

0.2.0 #

  • Renamed package from instrumentation_adapter to e2e.
  • Refactored example app test.
  • Breaking change. Renamed InstrumentationAdapterFlutterBinding to E2EWidgetsFlutterBinding.
  • Updated README.

0.1.4 #

  • Migrate example to AndroidX.
  • Define clang module for iOS.

0.1.3 #

  • Added example app.
  • Added stub iOS implementation.
  • Updated README.
  • No longer throws errors when running tests on the host.

0.1.2 #

  • Added support for running tests using Flutter driver.

0.1.1 #

  • Updates about using androidx library.

0.1.0 #

  • Update boilerplate test to use @Rule instead of FlutterTest.

0.0.2 #

  • Document current usage instructions, which require adding a Java test file.

0.0.1 #

  • Initial release

example/lib/main.dart

import 'my_app.dart' if (dart.library.html) 'my_web_app.dart';

// ignore_for_file: public_member_api_docs

void main() => startApp();

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  e2e: ^0.4.0

2. Install it

You can install packages from the command line:

with Flutter:


$ flutter pub get

Alternatively, your editor might support flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:e2e/e2e.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
98
Health:
Code health derived from static analysis. [more]
100
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
100
Overall:
Weighted score of the above. [more]
99
Learn more about scoring.

We analyzed this package on Apr 3, 2020, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.7.1
  • pana: 0.13.6
  • Flutter: 1.12.13+hotfix.8

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.1.0 <3.0.0
flutter 0.0.0
flutter_driver 0.0.0
flutter_test 0.0.0
Transitive dependencies
archive 2.0.11 2.0.13
args 1.5.2 1.6.0
async 2.4.0 2.4.1
boolean_selector 1.0.5 2.0.0
charcode 1.1.2 1.1.3
collection 1.14.11 1.14.12
convert 2.1.1
crypto 2.1.3 2.1.4
file 5.1.0
fuchsia_remote_debug_protocol 0.0.0
image 2.1.4 2.1.12
intl 0.16.0 0.16.1
json_rpc_2 2.1.0
matcher 0.12.6
meta 1.1.8
path 1.6.4
petitparser 2.4.0 3.0.2
platform 2.2.1
process 3.0.12
pub_semver 1.4.2 1.4.4
quiver 2.0.5 2.1.3
sky_engine 0.0.99
source_span 1.5.5 1.7.0
stack_trace 1.9.3
stream_channel 2.0.0
string_scanner 1.0.5
term_glyph 1.1.0
test_api 0.2.11 0.2.15
typed_data 1.1.6
vector_math 2.0.8
vm_service_client 0.2.6+2 0.2.6+3
web_socket_channel 1.1.0
xml 3.5.0 4.1.0
Dev dependencies
pedantic ^1.8.0 1.8.0+1 1.9.0