Key Concepts
End-to-End Testing
By this, we mean testing the Mobile App, together with an instance of the Safe Places software. It is possible (and sometimes desirable) to test the Mobile App in isolation, or Safe Places in isolation - See this article for some notes on what other options are available: Testability of Safe Places - PathCheck GPS Mobile App interactions
Integration Points
There are 2 key functions of the Mobile App that interact with Safe Places
Location Data Sharing. This requires that a key is generated by the Safe Places user, and entered into the Mobile App by the user who wants to share their data. The data can then be viewed and managed within Safe Places.
Exposure Notifications. This depends on the Mobile App downloading a set of “points of concern” from the Health Department. These points of concern are published by Safe Places.
SafePlaces Endpoints
SafePlaces Endpoints are URLs that offer a particular service. For end-to-end testing, there are 2 key Endpoints, corresponding to the 2 integration points above.
Location Data Sharing uses the Ingest API URL
Exposure Notifications use the Cursor URL
These URLs will take different values for each Safe Places instance.
YAML file
YAML files contain the data about the Health Departments that are available, including the Safe Places instance and API Endpoints that are available.
We have several YAML files, for example
Production: https://github.com/Path-Check/trusted-authorities/blob/master/production/authorities.1.0.1.yaml (used in Production)
Staging: https://github.com/Path-Check/trusted-authorities/blob/master/staging/authorities.1.0.1.yaml (used in Testing)
Each YAML file lists a number of Health Departments, each of which has a Safe Places instance, and details of the API Endpoints used by that Safe Places Instance.
The “public_api” parameter points to the Ingest API for the Safe Places instance
The “cursor_url” parameter points to the Cursor URL for the Safe Places instance
Which YAML file does the Mobile App use?
The YAML file to use is hardcoded in the App.
Alpha, Beta and Production builds all point to the Production YAML file
Developer & Develop builds point to the Staging YAML file.
Note that while these builds can easily be distributed on Android as APKs, there are technical difficulties distributing these builds on iOS.
Custom builds can be created to point to bespoke YAML files.
It is also possible, using a Feature Flag, to manually conigure any YAML file in the app. When this is configured, this YAML file is used in addition to to the data in the hardcoded YAML file.
Privacy and Safe Places
At the time of writing (July 8 2020) Safe Places is implemented in such a way that any user who has access to the system, has access to all the location data that has been uploaded to the system.
While this is not a huge concern in production (where only vetted Contact Tracers should have log-in details), on test systems this means that any tester who has access to the system, has access to all the location data uploaded to the system.
For Privacy reasons, we want to take some care not to give wide open access to a very large number of testers.
Therefore, when we bring in crowdsource test organizations like Applause, to protect both their testers, and our testers, we prefer to have them work with their own private Safe Places instance.
Individual testers with particular privacy concerns may prefer to work with a personal Safe Places instance, rather than a shared instance. If you’d like help or support here, or have other privacy-related concerns, please talk to Diarmid Mackenzie or Adam Leon Smith
Testing End-to-End
Key decisions
To test end to end, you need to make some key decisions:
Which Safe Places instance am I using?
Which YAML file will I use to provide the App woth the details of this Safe Places instance
How will the App be set up to use this YAML file?
Example 1 - Testing with a HD’s Production Deployment
This is a straightforward set-up:
You will be using the HD’s Production Safe Places instance
You will use the Production YAML
Any Alpha, Beta or Production App build will automatically read the production YAML file.
So setup is easy - however this kind of testing will quickly hit restrictions:
When you come to share data, you will need someone from the HD to provide you with an access code for data sharing. (we assume that HDs will not be giving out Safe Places login credentials to PathCheck testers!).
If you want to test Exposure Notifications, you’ll only be able to test against the data that the Health Department publishes (i.e. real data about real COVID patients). This will limit Exposure Notification testing, especially if you are not in the geographic area served by the HD.
Example 2 - Testing with a HD’s Pre-production Deployment
Some HDs may be doing testing with their own pre-production Safe Places instance.
Depending on the situation…
Testers may have access to the Safe Places system
It may be OK for testers to push test points of concern (i.e. points that don’t correspond to a real COVID infection) to the Safe Places published points of concern.
In terms of the 3 key setup questions…
The tester will be using the HD’s Pre-production Safe Places instance
The details of this Safe Places instance won’t be in the Production YAML, so they will need to be included in some other YAML file. This could be…
A bespoke YAML file created just for this HD. It just needs to be hosted somewhere where it can be downloaded over HTTPS - e.g. GitHub, or any other web server with HTTPS
An entry in the Staging YAML file. (note this file is in version control, so changes require a PR, which is not super-convenient).
There are a few options here:
Use a Development build, which natively points towards the Staging YAML file.
Using any build, use the Feature Flag capability, and manually enter the URL of the YAML file. This will work whatever YAML file is used (even with the Staging or Production YAML files).
Note that Feature Flags are available in all Mobile App builds, even the one you get from the production App Store.
Example 3 - Testing with our internal Safe Places instances
We have 3 internal Safe Places instances
Dev - intended for us by the Development team. Might be unstable.
Staging - intended for use by Test/QA. We try to keep this fairly stable.
Demo - intended for use for Demos. Must remain very stable. Do not fill this with junk data!
Access details for these can be found on Slack in a pinned post in the “scrum_team_safe_places” channel, and also the “safe_places_testing” channel.
This includes details of the API endpoints, but you shouldn’t need to worry about those details, because they should all be kept up-to-date in the Staging YAML.
To test with one of these…
Pick the Safe Places instance that is suitable for your needs. Typically Staging for testers, but there may be good reasons to use Dev or Demo.
The Staging YAML file contains all the details you need.
If you use an Alpha, Beta or Production build, this will natively point to the Production YAML file, so the best solution is to use the Feature Flag to manually add the Staging YAML file as well.
Here’s a URL to copy/past for that purpose:
(you can get this by clicking the “Raw” button in the top right of the fiel viewer in GitHub)
Example 4 - Testing with a private Safe Places instances (e.g. Applause)
By “private” Safe Places instance, we mean an instance which is reserved for use by a specific known group of testers. This is distinct from a “personal” Safe Places instance (see below), which is used by just a single tester.
For Applause, we have set up a private Safe Places instance. We may do the same for other groups of testers that we want to partition from the main test setup, for mutual privacy reasons.
For these testers
They will use a Private Safe Places instance
We create a bespoke YAML file for these testers
These testers are directed to configure this YAML file using the available Feature Flag.
In future we may move to an alternative design whereby:
This system is included in the Staging YAML file
These testers have a build which natively points to the Staging YAML file.
This would simplify things for these testers (no need to configure a YAML file manually, and no need to use Feature Flags. However it complicates our build system…
Until we have the ability to provide a build that points natively to the Staging YAML, there’s no point in putting instances like the Appluase instance into the Staging YAML: those testers will have to configure a manual YAML file anyway, and it’s as easy for them to configure a bespoke YAML file with just their Safe Places instance.
Example 5 - Testing with a personal Safe Places instance
To be completed….