Author: Diarmid Mackenzie
Last updated by: Diarmid Mackenzie, Sunday 19 April, 12pm EDT
Testing Activity
Safe Paths Android: v0.9.4 in test
Various individual volunteers
Luca Dusi has a team of ~20 Italian testers, with a focus on:
Italian language
A wide range of devices & form factors
Error conditions
Usability
Safe Paths iOS: Testing is limited until we get through Beta App Review, which will enable Open Beta (hopefully by Monday)
Ali Raizin testing
We have limited spaces for other iOS testers. Contact Diarmid Mackenzie if you are keen to start immediately.
Jonathan Wright can repackage / resign the IPA file to deploy to any iOS devices (just need peoples device IDs (uid) and apple ids).
Luca Dusi team of ~20 testers lined up to test iOS when available.
Safe Places
Adam Leon Smith has been developing an initial test plan
Starting Monday, we should have a team from PQA Test, starting testing.
Lead contact TBC
Safe Paths Automation
Team from PQA Test starting tomorrow.
Lead contact: TBC
Safe Places Automation
Team from PQA Test starting tomorrow.
Lead contact: TBC
Test Data / Location Data
Adam Leon Smith is looking at a number of solutions for Test Data
His own Google location history. 5 years of data rebased into 86 different versions of March 2020. He has tools to do this manipulation if you want to (see the test plan doc)
If you are using your own long-term location history for testing, be mindful of privacy concerns, and keep the data to yourself (do not share with the project) unless you have explicit agreement otherwise.
He has a ML contact who can use this as seed data to generate many more similar trails.
Huw Price previously solved this problem for smart cities project, Jonathan Wright has reached out for the 5billion historical journeys previously generated (GPX format).
We have a lead on a tech that allows a user to program alternate GPS coordinates into their phone, to simulate being in different locations. Could be a powerful enabler, as testers will be able to use test data from anywhere in the world.
Jonathon Wright is qualifying this opportunity. Working with Eran Kinsbruner around using the same stack we used to test smart city mobile apps (https://youtu.be/wuQt97rTa1Y)
Test Data / Mock HAs for Automation. Problem that will need solving. Probably give to PQA Test as part of their Automation mission - DIarmid to brief on Monday.
Testing in Production
Jonathon Wright is taking the lead on Testing in Production - figuring out what we need to do, and how to make it happen.
Working with Todd DeCapua around Testing in Production (TiP) / Telemetry Data (Firebase)
Overall MVP Quality Readout
Diarmid owns this for now, trying to present a position based on the Quality Map document
What aren’t we doing?
Here are the things that we probably ought to be doing more of. Plus why we aren’t doing them
Testing Community support (Not Blocked - needs someone’s time & energy! Who can step up?)
Somebody taking an explicit “Scrum master”-like role in how we are all working together, what’s effective, what’s ineffective & how we can improve
Identifying & resolving team impediments
Of course, everyone can do this, but it would be great to have one or more people making this a real focus.
Security / Privacy (Blocked)
No active testing in this space.
Issues we are aware of are not getting prioritized by Dev, so not clear it is worth putting effort into finding more.
Diarmid Mackenzie has a string of 30 Privacy & Security concerns about the project that need to be channeled to the right people for review. He’s figuring out how to handle.
Adam Leon Smith has raised a feature request for a brief architectural deployment/security note on this for Safe Places
Key impediments for the team
These include a raised date (baselined at 4/19), to provide some accountability in terms of resolving impediments quickly.
(4/19) Too many gaps in documentation of requirements, and detailed product behaviour
Diarmid hoping to drive something here, but anyone else who could take this on would be amazing.
(4/19) No available domain expert for Contact Tracing
Kyle is working on a contact through Alina
Is one enough? Or do we need more?
(4/19) No signatory to make a contract with Applause
Christian is trying to find us somebody - until we have this in place we can’t use Applause
Diarmid to share PoC - Du Tri will take it forward for Partnership to action
(4/19) #430 means we still don't have the ability to import data from Health Agencies. This blocks a lot of important App testing.
Escalated to Abhishek Singh who is making sure it gets the attention it needs.
(4/19) No clear path for escalation of very high-level issues identified in Test - e.g. Privacy / Security concerns
Diarmid asking Yasaman, Christian & Kyle for guidance, but not path forward yet.
Du Tri will take forward - he has a link to the doc.
(4/19) Dev bandwidth & MVP 1.0 crunch is a blocker in terms of engaging more in areas that are likely to need significant time from Dev
E.g. Security / Privacy
(4/19) Very limited testing for iOS until we get into Open Beta
We are in Beta Review right now - hopefully just a few days.
(4/19) PractiTest not yet embedded as a tool.
Outdated documentation points to spreadsheets rather than PractiTest
Limited test cases documented in PractiTest
No clear patterns for using PractiTest for more exploratory forms of testing
(4/19) Onboarding has a 3d backlog
Diarmid has permission to bypass onboarding by directly sharing a link to the key info. This has been key to getting certain people on board fast.
(4/19) No current process for raising reqs/spec issues uncovered in Test
We’ve been asked not to file these in GitHub issues.
We believe we’ll be moving to Jira, but not quite ready yet
We need to update guidance for testers.
(4/19) Not clear map for new arrivals about what works, what doesn’t work, where to focus testing attention
Diarmid envisages a coverage / status dashboard based on the Quality Map. Work to do to make this a reality
??? what else - please add your impediments here
Who owns resolving impediments? We all do! I (Diarmid) am very happy to brainstorm about forward paths on any of these - reach out on Slack.
Other useful docs: