This is a template that can be copied and duplicated for all following builds. Simply Plug in the information and check off the list as you go.

Build Validated for Release Candidate?

Validated by QA Lead

Released Live

Summary:

This is a checklist for the QA lead to run through the tests of the app and make sure they don’t miss any important steps

Test Started: Dec 8, 2020

Current available HAs:

PR, Guam, MN, LA

iOS -

Device used:

HA used:

Android -

Device used:

HA used:

*There are significant differences between Android and iOS and the different versions of the GAEN api, some (non-exhaustive) are noted below:

iOS: Bluetooth and EN
Android 10 or below: Bluetooth, EN, and Location
Android 11: Bluetooth and EN

Verification General Test (only need one Health Authority per Operating System)

ONBOARDING / BLUETOOTH / LOCATION

Goal: a user can download the app and make it to the core features.

--

MANUALLY DETECT EXPOSURES

Goal: a user can manually prompt the app to check for exposures

#	Instruction	Check (iOS)	Check (Android)
1	Select the "Exposures" tab from the bottom menu	Button will select correct page	Button will select correct page
2	Click the "Check for exposures" button	"Success" banner shown, after "last 14 days" text is updated to reflect latest check	"Success" banner shown, after "last 14 days" text is updated to reflect latest check

--

EXPOSURE HISTORY

Goal: a user can access their exposure history populated with exposures

#	Instruction	Check (iOS)	Check (Android)
1	In Settings, Use the EN Debug menu to simulate 1 or more exposures	EN is generated & success message shows	EN is generated & success message shows
2	Navigate to Exposure History	Exposure History shows expected number of exposures	Exposure History shows expected number of exposures
3	Tap on an exposure	Relevant Next Steps info is shown	Relevant Next Steps info is shown

--

GAEN (Google Apple Exposure Notifications) Permissions

Goal: an onboarded user can enable required permissions as expected.

#	Instruction	Check (iOS)	Check (Android)
*	Disable Exposure Notifications (device OS), Enable	Exposure Notifications changes from red X to green check	Exposure Notifications changes from red X to green check
*	Disable Location *(if applicable)** on device / Re-enable		Location changes from red X to green check

--

VERIFICATION CODE & KEY SUBMISSION

Goal: an onboarded user can submit a verification code as expected.

*Code provided from verification server, if you do not have access and are an internal QA please contact @Arthur Gibson

--

MANUALLY DELETE APP DATA

Goal: a user can manually delete their data stored by the app

#	Instruction	Check (iOS)	Check (Android)
1	Access the Settings page	Page shows with no issues	Page shows with no issues
2	Press "Delete My Data"	Page "Delete My Data" shows, text is shown with no issue	Page "Delete My Data" shows, text is shown with no issue
3	Press "Delete My Data" red banner	Pop-up confirmation is shown	Pop-up confirmation is shown
4	Press “Confirm” on pop-up	Intro onboarding page is shown - no previous data is shown in main pages	Intro onboarding page is shown - no previous data is shown in main pages

--

APP UPDATE

Goal: a user can update the app and verify that simulated exposure is still shown

--

GAEN EXPOSURE NOTIFICATION (END-TO-END VERIFICATION)

Goal: a user can receive an exposure notification and see it on the exposure history.

NOTE: Two (or more) phones running app w/ Exposure Notifications ON required.

For testing purposes, we will use an iOS and an Android device to run this test (provides a full end-to-end experience). The Android device will be device #1 (this is the one that a positive test case will be reported on) and the iOS device will be device #2 (the device that an exposure notification should appear on).

HA Used for End-To-End:

When generating a Verification Code: You need to make sure to use a Symptoms onset and Testing date that is 24 hours or less from the actual contact of the two devices we are testing with. Example: if it’s Dec. 11 and the phones have been in contact for the past 24 hours, I would use Dec 10 as my testing and Symptoms onset date.

Symptoms Onset Date used:Dec 10, 2020

Testing Date Used:Dec 10, 2020

Last Date of contact for testing Devices:Dec 11, 2020

Unique identifier (NOT the short code) :

#	Instruction	Device #2 (iOS)	Device #1 (Android)
1	Let phone sit for AT LEAST 1 hour (app in background)	Debug menu > Show Local Diagnosis Keys has keys	Debug menu > Show Local Diagnosis Keys has keys
2	Move phone to within 6 ft of another testing phone (at least 15 minutes)
3	"Report positive test result" > "Start" > enter code > share keys		No errors
4	Wait (up to 24 hours)	Exposure Notification appears on second testing phone
5	(On 2nd phone) Open Exposure History	Possible exposure is listed, Next Steps button leads to relevant HA website (if applicable)

Interface General Check

General UI/UX	iOS	Android
Goal: a user can navigate the app with any supported text or language settings
Large Font: execute any of the above tests with device's font set to maximum*	All screens are readable & navigable	All screens are readable & navigable
Translation: execute any of the above tests after selecting (and toggling) a non-English language	All screens are readable & translations make sense	All screens are readable & translations make sense

GAEN Permissions*	iOS	Android
Goal: an onboarded user can enable required permissions as expected.
Disable Exposure Notifications (device OS), Enable Proximity Tracing (in app)	EN Check on homescreen changes from red X to green check	EN Check on homescreen changes from red X to green check
Disable Location (if applicable) on device / Re-enable		Location changes from red X to green check

+3 Day Symptom Tracking Test

Test skipped for this build.

This test is not always necessary. We will work with the developers to determine which builds and which HAs would require this. Only do if the release effects exposure notifications, we will work on a better way to indicate this for future testing.

This is to verify that phones DO NOT receive an EN (Exposure Notification) while in the 14 day window of tracking IF the Exposure Reporter (person who tested positive) denoted that the onset of symptoms did not occur until 3 days after the interaction.

Example:

Person 1 (Who does not have symptoms at the time) is within six (6) feet of Person 2, and Person 3. Lets call this ‘Day A’
Three (3) days go by after the occurrence of Persons 1, 2, and 3 within six (6) feet and Person 1 then starts having symptoms. This is ‘Day B’
Within 14 days of ‘Day A’, Person 1 is tested and receives a positive test. This is ‘Day C
Person 1 reports to a tracker, when the tracker asks when the test happened, Person 1 denotes Day C, the tracker then asks when symptoms started, Person 1 denotes Day B, and the tracker then gives Person 1 their Verification Code.
Person 1 then inputs their Verification Code into the App, it is registered with the server and completes the transition of data
--
Because Day B is 3 (or more) days before Person 1’s symptoms appear, Persons 1 and 2 are NOT notified.

Steps for Testing:

Please make sure that phones have been wiped of any previous data.
- If on iOS make sure to access the App’s Settings tab, and click “Delete my data”, uninstall the app THEN make sure to have turned off Exposure Notifications in the Settings > Exposure Notifications page to purge any previous data.
-If on Android, while app is still installed, access Settings > Exposure Notifications > Exposure Checks > Delete exposure checks, THEN in the App’s Settings Tab, click “Delete my data” and uninstall the app.

Access 3 phones (Android or iOS)
Install the app, make sure they're on the same HA
Set all phones to a date (hereto referred to as DATE A) - Make sure this is within the 14 day window, but also have the ability to give 3 days after for symptoms.
[EX: if today is 11/18, make sure phones are set to a 'DATE A' of 11/9 (within 14 days, but room within that window to add 3 days for the first day symptoms are reported)]
Allow phones to sit at least 1 hour (1.5 hours if possible)
Change all phones to current date
Access the HA's Test Verification Interface, login, choose correct realm
Create a verification code, make sure to set:
A) Symptom Onset to 3 days past DATE A (hereto referred to as DATE B) [Ex: if DATE A is 11/9, DATE B is 11/12]
B) Testing Date is within 14 days of DATE A
Select one of the phones to be the "Reporter", Input Verification Code generated
Sequester “Reporter” phone out of Bluetooth distance, or turn the phone OFF)
Allow adequate time for server to relay exposure notifications (up to 24 hours)

+3 Day Test Follow Up

+3 Day Test Follow Up
Test Passed Test Skipped Test Failed
Notes:

#	Instruction	Check (iOS)	Check (Android)
1	Install App	No Errors	No Errors
2	Proceed through onboarding wizard (5 screens)	Text displays and is readable	Text displays and is readable
3	Open "How we protect your privacy"	Text displays and is readable	Text displays and is readable
4	Try to Activate Exposure Notifications	Should prompt to open settings Click “No thanks” button	Should prompt to open settings Click “No thanks” button
6	Enable Notifications	Screen “App Setup Incomplete” is shown	Screen “App Setup Incomplete” is shown
7	Access Home Page	Exposure Detection is red - shows OFF	Exposure Detection is red - shows OFF
8	Enable Location (only necessary on older Android OS's)**		“Location” shows ON
9	Verify Exposure Detection is ON	Card on Home screen shows as green and ON	All three fields show as green and ON

#	Instruction	Check (iOS)	Check (Android)
1	Generate verification code* On Health Authority’s “Issue Code” page, input required data for positive results and click “Create Verification Code”	Code is provided from HA server	Code is provided from HA server
2	Access App, select Report Result area	Graphic / Text show	Graphic / Text show
3	Enter verification code
4	"Next" > "Share" > "I understand and consent"	No errors after selecting any of the buttons	No errors after selecting any of the buttons
*	Offline: disable devices network connectivity and execute above test	Error should appear after "Next" and prevent code submission	Error should appear after "Next" and prevent code submission
*	Invalid Code: Submit a non-valid verification code	Error should appear after "Next" and prevent code submission	Error should appear after "Next" and prevent code submission

#	Instruction	Check (iOS)	Check (Android)
1	Install build 1.0.x (older) from AppCenter	Older build installs without issue	Older build installs without issue
2	Navigate to debug menu and tap "simulate exposure"	exposure simulation occurs	exposure simulation occurs
3	Navigate to Exposure history screen and view exposure	Exposure is shown in Exposure History	Exposure is shown in Exposure History
4	Download Staging build 1.0.y (newer) from AppCenter	Newer build installs without issue	Newer build installs without issue
5	Return to updated app’s “Exposure History” Tab	Previous Exposure is still shown in Exposure History	Previous Exposure is still shown in Exposure History

Testing Checklist