This is a consultation document intended for use between Test, Dev and PM regarding quality targets for the MVP. Early Draft, under review - do not take any content here as final.
...
BRIEF UPDATE: After initial conversation with Sam Zimmerman his feedback is that we are going to have to push forward the quality on Security & Privacy faster than we are at the moment - i.e. not acceptable for them to be Red and we need to get them moving towards Green. Anyone with a particular interst or expertise in Security or Privacy, please get in touch with Diarmid Mackenzie The rest is probably about right. Some details on overall stratregy and why this is the right prioritization of wuality to follow shortly….
...
It is generated from the latest Quality Map (Apr 21 2020) by Test.
Based on Test’s understanding of the current levels of function in tha App it represents a best case in terms of quality targets that could be achieved for MVP, subject to fixes to address the problems bugs we will inevitably find in Test The need to ship the MVP quickly and pragmatically means that it is llikely quite likely that the final shipped MVP we ship will in fact fall short of what is outlined in this document.
...
It is not an assessment of the quality currently delivered by the App (our expectation is that the app falls short of this positon today on multiple aspects, but we have not yet made a full assessmentassessment is ongoing).
By highlighting certain cells red & yellow we aim to provide::
Direction for PM, that our current plan is expected to leave deficiencies in these areas vs. the eventual requirements for the product.
Direction for Test, that it would be unhelpful to raise bugs in these areas (except for bugs of extremely high severity) since it is acknoowledged and expected that these aspects of the product will be deficient.
Direction for Dev in terms of which bug fixes to target for MVP, and which bugs can be left.
...
Attribute | Illustrative statement (not comprehensive) | Safe Paths | Non-App Dependencies |
Precision of Info | When I overlapped with someone I want to get specific information (arrow time window, specific place) | We should meet this expectation | |
Accuracy (False Positives) | I don’t want to be warned about encounters where there was no plausible risk of infection | There will be False positives (e.g. when alone in a car) | Also dependent on redaction practices |
Accuracy (False Negatives) | If I have an encounter with someone that risks infection, I want to be notified | There will be many false negatives, but at least some true positves | Also depends on other users sharing data |
Reliability (foreground) | I want the app to work when I try to use it. No hangs, crashes etc. | We should meet this expectation | |
Reliability (background) | Background operations of the app should work all the time - no silent failures | We should meet this expectation | |
Clarity | I want it to be easy to understand what the app is telling me, and what actions I should take. | We should meet this expectation | May be additonal supporting guidance on websites |
Usability | It should be easy to do what I want to do in the app without too much thought. | We should meet this expectation | |
Privacy | My personal data should not be exposed without my consent | There are known shortcomings here: see Privacy Roadmap - Input from Testing team & Software Testing Community | Also depends on overall privacy architecture |
Security | The app should not expose me to any source of harm | There are known shortcomings here | Also depends on overall security architecture |
Trust | I’m putting my trust in this app. I don’t want it undermining my confidence in it. I want it to feel like a high quality product.. | We should meet this expectation | Dependency on quality of data available to App, availablity of HAs |
Cost | The app should not cost me much to run & use (ideally it should be free). | Data volume downloaded is not optimized | Free requires agreements with network operators |
Complete data | I want most infected users to share their data (actually a specific category of “False Negatives” but important enough to pull out separately) | N/A | Depends on public campaigning |
...
Attribute | Illustrative statement (not comprehensive) | Safe Paths | Non-App Dependencies |
Reliability | I want the app to work when I try to use it. No hangs, crashes etc. | We should meet this expectation | |
Non-intrusivity | When I am not trying to use the app, I don’t want it to bother me, or cause me any issues | We should meet this expectation | |
Clarity | I want it to be easy to understand what the app is telling me, and what actions I should take. | We should meet this expectation | |
Usability | It should be easy to do what I want to do in the app without too much thought. | We should meet this expectation | |
Privacy | My personal data should not be exposed without my consent | There are known shortcomings here. See: Privacy Roadmap - Input from Testing team & Software Testing Community | Also depends on overall privacy architecture |
Security | The app should not expose me to any source of harm | There are known shortcomings here | Also depends on overall security architecture |
Trust | I’m putting my trust in this app. I don’t want it undermining my confidence in it. I want it to feel like a high quality product.. | We should meet this expectation | |
Cost | The app should not cost me much to run & use (ideally it should be free). | Data volume uploaded is not optimized | “Free” requires agreements with network operators |
...