CSE431

Mobile Programming

MODULE 8:
Mobile App Testing
Testing Flutter Apps
The more features your app has, the harder it is to test manually. Automated tests help ensure that your
app performs correctly before you publish it, while retaining your feature and bug fix velocity.

Automated testing falls into a few categories:

• A unit test tests a single function, method, or class.

• A widget test (in other UI frameworks referred to as component test) tests a single widget.
• An integration test tests a complete app or a large part of an app.

Generally speaking, a well-tested app has many unit and widget tests, tracked by code coverage, plus enough
integration tests to cover all the important use cases
Testing
Flutter
Unit tests Widget tests Integration tests
Apps • A unit test tests a single function,
method, or class. The goal of a unit
• A widget test (in other UI
frameworks referred to
• An integration test tests a complete
app or a large part of an app. The
test is to verify the correctness of a as component test) tests a single goal of an integration test is to
unit of logic under a variety of widget. The goal of a widget test is verify that all the widgets and
conditions. External dependencies to verify that the widget's UI looks services being tested work together
of the unit under test are and interacts as expected. Testing a as expected. Furthermore, you can
generally mocked out. Unit tests widget involves multiple classes and use integration tests to verify your
generally don't read from or write requires a test environment that app's performance.
to disk, render to screen, or receive provides the appropriate widget • Generally, an integration test runs
user actions from outside the lifecycle context. on a real device or an OS emulator,
process running the test. such as iOS Simulator or Android
Emulator. The app under test is
typically isolated from the test
driver code to avoid skewing the
results.
Continuous integration services
Continuous integration (CI) services allow you to run your tests automatically when pushing new code changes.

This provides timely feedback on whether the code changes work as expected and do not introduce bugs.

For information on running tests on various continuous integration services, see the following:

•Continuous delivery using fastlane with Flutter

•Test Flutter apps on Appcircle
•Test Flutter apps on Travis
•Test Flutter apps on Cirrus
•Codemagic CI/CD for Flutter
•Flutter CI/CD with Bitrise
Unit Testing in Flutter
Unit Testing in Flutter
Writing and Running Unit Tests

Use the test package for writing unit tests in Flutter. Create test files with the _test.dart suffix, and annotate test
functions with test.

import 'package:flutter_test/flutter_test.dart';

void main() {
test('Addition test', () {
expect(2 + 2, 4);
});
}

https://medium.com/@dihsar/the-complete-guide-to-flutter-app-testing-with-examples-4ed7b8188bce
Unit Testing in Flutter
Use the test package for writing unit tests in Flutter. Create test files with the _test.dart suffix, and annotate test
functions with test.

Example: Testing a Utility Function

import 'package:flutter_test/flutter_test.dart';

int add(int a, int b) => a + b;

void main() {
test('Add function test', () {
expect(add(2, 3), 5);
});
}
Unit Testing in Flutter - Steps
1. Add the test dependency

The test package provides the core functionality for writing tests in Dart. This is the best approach when writing
packages consumed by web, server, and Flutter apps.

To add the test package as a dev dependency, run flutter pub add:

flutter pub add dev:test

https://docs.flutter.dev/cookbook/testing/unit/introduction
Unit Testing in Flutter - Steps
2. Create a test file
In this example, create two files: counter.dart and counter_test.dart.

The counter.dart file contains a class that you want to test and resides in the lib folder. The counter_test.dart file
contains the tests themselves and lives inside the test folder.

In general, test files should reside inside a test folder located at the root of your Flutter application or package.
Test files should always end with _test.dart, this is the convention used by the test runner when searching for
tests.

When you're finished, the folder structure should look like this:
counter_app/
lib/
counter.dart
test/
counter_test.dart
Unit Testing in Flutter - Steps
3. Create a class to test

Next, you need a "unit" to test. Remember: "unit" is another name for a function, method, or class. For this
example, create a Counter class inside the lib/counter.dart file. It is responsible for incrementing and
decrementing a value starting at 0.

class Counter {
int value = 0;

void increment() => value++;

void decrement() => value--;

}
Unit Testing in Flutter - Steps
4. Write a test for our class

Inside the counter_test.dart file, write the first unit test. Tests are defined using the top-level test function, and
you can check if the results are correct by using the top-level expect function. Both of these functions come
from the test package.

// Import the test package and Counter class

import 'package:counter_app/counter.dart';
import 'package:test/test.dart';

void main() {
test('Counter value should be incremented', () {
final counter = Counter();

counter.increment();

expect(counter.value, 1);
});
}
 Unit Testing in Flutter - Steps
5. Combine multiple tests in a group import 'package:counter_app/counter.dart';
import 'package:test/test.dart';
If you want to run a series of related tests, void main() {
use the flutter_test package group function group('Test start, increment, decrement', () {
to categorize the tests. Once put into a test('value should start at 0', () {
group, you can call flutter test on all tests in expect(Counter().value, 0);
that group with one command. });
test('value should be incremented', () {
final counter = Counter();
6. Run the tests counter.increment();
expect(counter.value, 1);
Now that you have a });
Counter class with tests in test('value should be decremented', () {
place, you can run the tests. final counter = Counter();
counter.decrement();
flutter test test/counter_test.dart expect(counter.value, -1);
});
});
}
 Unit Testing in Flutter - Steps
5. Combine multiple tests in a group import 'package:counter_app/counter.dart';
import 'package:test/test.dart';
If you want to run a series of related tests, void main() {
use the flutter_test package group function group('Test start, increment, decrement', () {
to categorize the tests. Once put into a test('value should start at 0', () {
group, you can call flutter test on all tests in expect(Counter().value, 0);
that group with one command. });
test('value should be incremented', () {
final counter = Counter();
6. Run the tests counter.increment();
expect(counter.value, 1);
Now that you have a });
Counter class with tests in test('value should be decremented', () {
place, you can run the tests. final counter = Counter();
counter.decrement();
flutter test test/counter_test.dart expect(counter.value, -1);
});
});
}
 Mock dependencies using Mockito
Sometimes, unit tests might depend on classes that fetch data from live web services or databases. This is
inconvenient for a few reasons:

• Calling live services or databases slows down test execution.

• A passing test might start failing if a web service or database returns unexpected results. This is known as a
"flaky test."
• It is difficult to test all possible success and failure scenarios by using a live web service or database.

• Therefore, rather than relying on a live web service or database, you can "mock" these dependencies.

• Mocks allow emulating a live web service or database and return specific results depending on the situation.

• Generally speaking, you can mock dependencies by creating an alternative implementation of a class. Write
these alternative implementations by hand or make use of the Mockito package as a shortcut.

https://docs.flutter.dev/cookbook/testing/unit/mocking
 Mock dependencies using Mockito
Steps

1. Add the package dependencies.

2. Create a function to test.

3. Create a test file with a mock http.Client.

4. Write a test for each condition.

5. Run the tests.

https://docs.flutter.dev/cookbook/testing/unit/mocking
 Mock dependencies using Mockito
1. Add the package dependencies

To use the mockito package, add it to the pubspec.yaml file along with the flutter_test dependency in the
dev_dependencies section.

This example also uses the http package, so define that dependency in the dependencies section.

mockito: 5.0.0 supports Dart's null safety thanks to code generation. To run the required code generation, add
the build_runner dependency in the dev_dependencies section.

To add the dependencies, run flutter pub add:

flutter pub add http dev:mockito dev:build_runner

https://docs.flutter.dev/cookbook/testing/unit/mocking
Mock dependencies using Mockito

2. Create a function to test

In this example, unit test the fetchAlbum function Future<Album> fetchAlbum(http.Client client) async {
from the Fetch data from the internet recipe. To test final response = await client
this function, make two changes: .get(Uri.parse('https://jsonplaceholder.typicode.com/albums/1'));

Provide an http.Client to the function. This allows if (response.statusCode == 200) {

providing the correct http.Client depending on the // If the server did return a 200 OK response,
situation. For Flutter and server-side projects, provide // then parse the JSON.
an http.IOClient. For Browser apps, provide an return Album.fromJson(jsonDecode(response.body) as Map<String,
http.BrowserClient. For tests, provide a mock dynamic>);
http.Client. } else {
Use the provided client to fetch data from the // If the server did not return a 200 OK response,
internet, rather than the static http.get() method, // then throw an exception.
which is difficult to mock. throw Exception('Failed to load album');
The function should now look like this: }
}

https://docs.flutter.dev/cookbook/testing/unit/mocking
 Mock dependencies using Mockito
3. Create a test file with a mock http.Client
Next, create a test file. import 'package:http/http.dart' as http;
Following the advice in the Introduction to unit testing import 'package:mocking/main.dart';
recipe, create a file called fetch_album_test.dart in the root import 'package:mockito/annotations.dart';
test folder.
// Generate a MockClient using the Mockito package.
Add the annotation @GenerateMocks([http.Client]) to the // Create new instances of this class in each test.
main function to generate a MockClient class with mockito. @GenerateMocks([http.Client])
void main() {
The generated MockClient class implements the http.Client }
class. This allows you to pass the MockClient to the
fetchAlbum function, and return different http responses in
each test.
Next, generate the mocks running the following command:
The generated mocks will be located in
fetch_album_test.mocks.dart. Import this file to use them. dart run build_runner build

https://docs.flutter.dev/cookbook/testing/unit/mocking
 import 'package:flutter_test/flutter_test.dart';

Mock dependencies using Mockito import 'package:http/http.dart' as http;

import 'package:mocking/main.dart';
import 'package:mockito/annotations.dart';
import 'package:mockito/mockito.dart';

import 'fetch_album_test.mocks.dart';
4. Write a test for each condition
// Generate a MockClient using the Mockito package.
// Create new instances of this class in each test.
@GenerateMocks([http.Client])
The fetchAlbum() function does one of two things: void main() {
group('fetchAlbum', () {
test('returns an Album if the http call completes successfully', () async {
1. Returns an Album if the http call succeeds final client = MockClient();

// Use Mockito to return a successful response when it calls the

// provided http.Client.
2. Throws an Exception if the http call fails when(client
.get(Uri.parse('https://jsonplaceholder.typicode.com/albums/1')))
.thenAnswer((_) async =>
Therefore, you want to test these two conditions. http.Response('{"userId": 1, "id": 2, "title": "mock"}', 200));

expect(await fetchAlbum(client), isA<Album>());

});
Use the MockClient class to return an "Ok" response
test('throws an exception if the http call completes with an error', () {
for the success test, and an error response for the final client = MockClient();
unsuccessful test. // Use Mockito to return an unsuccessful response when it calls the
// provided http.Client.
when(client
Test these conditions using the when() function .get(Uri.parse('https://jsonplaceholder.typicode.com/albums/1')))
.thenAnswer((_) async => http.Response('Not Found', 404));
provided by Mockito:
expect(fetchAlbum(client), throwsException);
});
});
To run the test: flutter test test/fetch_album_test.dart }
Widget Testing in Flutter
 Widget Testing in Flutter

The flutter_test package provides the following tools for testing widgets:

• The WidgetTester allows building and interacting with widgets in a test environment.
• The testWidgets() function automatically creates a new WidgetTester for each test case, and is
used in place of the normal test() function.
• The Finder classes allow searching for widgets in the test environment.
• Widget-specific Matcher constants help verify whether a Finder locates a widget or multiple
widgets in the test environment.

https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter

Steps

1. Add the flutter_test dependency.

2. Create a widget to test.
3. Create a testWidgets test.
4. Build the widget using the WidgetTester.
5. Search for the widget using a Finder.
6. Verify the widget using a Matcher.

https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter

1. Add the flutter_test dependency

Before writing tests, include the flutter_test dependency in the dev_dependencies section of
the pubspec.yaml file.

If creating a new Flutter project with the command line tools or a code editor, this dependency
should already be in place.

dev_dependencies:
flutter_test:
sdk: flutter

https://docs.flutter.dev/cookbook/testing/widget/introduction
 class MyWidget extends StatelessWidget {
const MyWidget({
Widget Testing in Flutter super.key,
required this.title,
required this.message,
});
2. Create a widget to test
final String title;
final String message;

@override
Next, create a widget for testing. For Widget build(BuildContext context) {
this example, create a widget that return MaterialApp(
displays a title and message. title: 'Flutter Demo',
home: Scaffold(
appBar: AppBar(
title: Text(title),
),
body: Center(
child: Text(message),
),
),
);
https://docs.flutter.dev/cookbook/testing/widget/introduction }
}
 Widget Testing in Flutter
3. Create a testWidgets test

With a widget to test, begin by writing your first void main() {

test. Use the testWidgets() function provided by
the flutter_test package to define a test. // Define a test. The TestWidgets function also provides a WidgetTester
// to work with. The WidgetTester allows you to build and interact
// with widgets in the test environment.
The testWidgets function allows you to define a
widget test and creates a WidgetTester to work testWidgets('MyWidget has a title and message', (tester) async {
with.
// Test code goes here.
This test verifies that MyWidget displays a given
title and message. It is titled accordingly, and it });
will be populated in the next section. }

https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter
4. Build the widget using the WidgetTester

Next, build MyWidget inside the test environment by using the pumpWidget() method provided by WidgetTester.

The pumpWidget method builds and renders the provided widget.

Create a MyWidget instance that displays "T" as the title and "M" as the message.

void main() { testWidgets('MyWidget has a title and message', (tester) async {

// Create the widget by telling the tester to build it.

await tester.pumpWidget(const MyWidget(title: 'T', message: 'M')); }); }

https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter
5. Search for our widget using a Finder
With a widget in the test environment, search through the widget tree for the title and message Text widgets using
a Finder. This allows verification that the widgets are being displayed correctly.

For this purpose, use the top-level find() method provided by the flutter_test package to create the Finders. Since
you know you're looking for Text widgets, use the find.text() method.

void main() { testWidgets('MyWidget has a title and message', (tester) async {

await tester.pumpWidget(const MyWidget(title: 'T', message: 'M'));

// Create the Finders.
final titleFinder = find.text('T’);
final messageFinder = find.text('M’);
});
}

https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter
6. Verify the widget using a Matcher
Finally, verify the title and message Text widgets appear on screen using the Matcher constants provided by
flutter_test. Matcher classes are a core part of the test package, and provide a common way to verify a given value
meets expectations.

Ensure that the widgets appear on screen exactly one time. For this purpose, use the findsOneWidget Matcher.
void main() {
testWidgets('MyWidget has a title and message', (tester) async {
await tester.pumpWidget(const MyWidget(title: 'T', message: 'M'));
final titleFinder = find.text('T'); final messageFinder = find.text('M’);
// Use the `findsOneWidget` matcher provided by flutter_test to verify
// that the Text widgets appear exactly once in the widget tree.
expect(titleFinder, findsOneWidget);
expect(messageFinder, findsOneWidget);
});
}
https://docs.flutter.dev/cookbook/testing/widget/introduction
 Widget Testing in Flutter
Additional Matchers

In addition to findsOneWidget, flutter_test provides additional matchers for common cases.

findsNothing
Verifies that no widgets are found.

findsWidgets
Verifies that one or more widgets are found.

findsNWidgets
Verifies that a specific number of widgets are found.

matchesGoldenFile
Verifies that a widget's rendering matches a particular bitmap image ("golden file" testing).

https://docs.flutter.dev/cookbook/testing/widget/introduction
Integration Testing Concepts

https://docs.flutter.dev/testing/integration-tests
 Integration Testing Concepts
Integration tests verify the behavior of the complete app. This test can also be called end-to-end testing or GUI
testing.

Dependent package

To run integration tests, add the integration_test package as a dependency for your Flutter app test file.

Tests written with the integration_test package can perform the following tasks.

• Run on the target device. To test multiple Android or iOS devices, use Firebase Test Lab.

• Run from the host machine with flutter test integration_test.

• Use flutter_test APIs. This makes integration tests similar to writing widget tests.

https://docs.flutter.dev/testing/integration-tests
 Check app functionality with an integration test

Integration tests verify the behavior of the complete app. This test can also be called end-to-end testing or GUI
testing.

Steps:

1. Create an app to test.

2. Add the integration_test dependency.
3. Create the test files.
4. Write the integration test.
5. Run the integration test.

https://docs.flutter.dev/testing/integration-tests
Check app functionality with an integration test

This example uses the built-in Counter App example that Flutter produces when you run the flutter create
command. The counter app allows a user to tap on a button to increase a counter.

1. To create an instance of the built-in Flutter app, run the following command in your terminal:

flutter create counter_app

2. Change into the counter_app directory.

3. Open lib/main.dart in your preferred IDE.
4. Add a key parameter to the floatingActionButton() widget with an instance of a Key class with a string value of
increment.
floatingActionButton: FloatingActionButton(
key: const ValueKey('increment’),
onPressed: _incrementCounter,
tooltip: 'Increment’,
child: const Icon(Icons.add), ), 5. Save your lib/main.dart file.
https://docs.flutter.dev/testing/integration-tests
Check app functionality with an integration test

Add the integration_test dependency

• You need to add the testing packages to your new app.

• To add integration_test and flutter_test packages as dev_dependencies using sdk: flutter, run following
command.
flutter pub add 'dev:integration_test:{"sdk":"flutter"}’

• Updated pubspec.yaml file

# ...
dev_dependencies:
# ... added depencies
flutter_test:
sdk: flutter
flutter_lints: ^4.0.0
integration_test:
sdk: flutter
# ...
https://docs.flutter.dev/testing/integration-tests
Check app functionality with an integration test

Create the integration test files

Integration tests reside in a separate directory inside your Flutter project.

• Create a new directory named integration_test.

• Add empty file named app_test.dart in that directory.

The resulting directory tree should resemble the following:

counter_app/
lib/
main.dart
integration_test/
app_test.dart

https://docs.flutter.dev/testing/integration-tests
 Check app functionality with an integration test integration_test/counter_test.dart
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:how_to/main.dart';
import 'package:integration_test/integration_test.dart';
Write the integration test
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
• The integration test file consists of a Dart code file with
dependencies on integration_test, flutter_test, and your group('end-to-end test', () {
testWidgets('tap on the floating action button, verify counter',
app's Dart file. (tester) async {
// Load app widget.
await tester.pumpWidget(const MyApp());
• Open your integration_test/app_test.dart file in your
preferred IDE. // Verify the counter starts at 0.
expect(find.text('0'), findsOneWidget);

• This example goes through three steps: // Finds the floating action button to tap on.
final fab = find.byKey(const ValueKey('increment'));
1. Initialize IntegrationTestWidgetsFlutterBinding. This
singleton service executes tests on a physical device. // Emulate a tap on the floating action button.
await tester.tap(fab);
2. Interact and test widgets using the WidgetTester
class. // Trigger a frame.
await tester.pumpAndSettle();
3. Test the important scenarios.
// Verify the counter increments by 1.
expect(find.text('1'), findsOneWidget);
Test on a mobile device: flutter test integration_test/app_test.dart });
});
https://docs.flutter.dev/testing/integration-tests }
Test Lab
End of Module 8