Safe Paths Quality Map

Initially written by Diarmid Mackenzie, Weds 15 April. Minor feedback from Steve Penrods applied. More feedback welcome (add comments, or edit directly - new content, or clarifications: just go ahead. If you want to delete or substantially rewrite existing stuff please comment & discuss first).

At the time, I was very focussed on Safe Paths app. Does this need more extension to reflect all stakeholders/value from Safe Places? (almost certainly, yes).

What is this doc & why does it exist?

This document is created by testers, and its purpose is:

• To help testers to think about quality goals for Safe Paths, and in particular to think about coverage of threats to quality

• To help Product Managers to understand what we know - and what we still don’t know about the quality of the product

• As a framework for anyone else who might be interested in thinking about product quality.  Devs in particular may be interested.

Introduction

In Context Driven Testing, we often explain “quality” as “value to someone who matters”

This leads to 2 more questions:

1. Who matters?  We call them stakeholders

2. What do they value in Safe Paths?

This “Quality Map” tried to identify all the key stakeholders in Safe Paths (there are more than you might think).

It also tried to identify & explain the key attributes of Safe Paths that matter to them.

We hope for this to be as complete as possible - though if the lists get too long, we might split based on priority so we don’t lose focus on what matters most.

How is this supposed to be useful?

For testers:

• Inspiration for test ideas.  Pick an area, and think about ways in which the product might fail to deliver that value.  Now you are starting to visualize possible bugs: think of the possible triggers, and you are starting to flesh out same space for productive exploratory testing.

• Evaluation of coverage.  To report on the status of our testing, we need to talk about coverage, as well as the problems we find.  “We tested and we didn’t find any bugs” doesn’t imply anything if your coverage was poor. This Quality Map gives a top-level framework for talking about coverage (within any individual cell on the map, you could talk about various dimensions of coverage too, but at a high level, this gives us some sense of what we know, and where the biggest unknown still lie).

For product managers

• Fundamentally the value is that it underpins good test reporting (in particular reporting about coverage) - and that’s valuable for PMs because it helps to ensure we keep aware of what we don’t know, as well as what we do know.

• This map may also be useful in helping PMs to think about how we are serving all our stakeholders, and to review for stakeholders or specific attributes valued by those stakeholders, that we are overlooking in our roadmap development.

See also - this (living) sheet where I try to map out the current status of these quality goals

Building & Maintaining the Quality Map

We’ll build the quality map in 3 stages:

1. One person (happens to be Diarmid) scratches his head and comes up with all the key stakeholders & things they value that he can think of.  Includes reviewing a selection of important bugs, to check whether the values those bugs threaten are things we’ve captured.

2. Doc put out for wide review, by PMs, Devs, other testers.  Crowdsource all the things Diarmid missed & correct his mistaken ideas & assumptions. We are in this stage right now

3. Maintenance mode.  We use this. We match bugs against it.  Over time we spot further gaps, and we adjust and move forwards.  (or maybe we do a more substantial overhaul).

One person needs to have responsibility for maintaining the overal integrity of the doc.  For now that’s also Diarmid

Let’s get Mapping!  Who are our Stakeholders?

Here’s a list of Stakeholders.  These are people that matter, in the context of Safe Paths.  Think of them as (classes of) people who could potentially be impacted by a defect in Safe Paths.

Further in the doc, we’ll get into analyzing what matters to each of these groups.

• App users (diagnosed & undiagnosed)

• Health Authorities

• Internet Service Providers

• The Local Community (including non-users of the app)

• Local Businesses

• Testers

• Developers

• Product Managers

• Sales (do we call them Sales?  I mean the folks who own our relationships with the Healthcare Providers) & Marketing

• Privacy Advocates

• Safe Paths Volunteers

• … 

EXTERNAL STAKEHOLDERS

Users (Undiagnosed)

Users (Diagnosed)

All these attributes are attributes for undiagnosed users too,  We pull them out because explicitly considering these attributes in the context of a diagnosed user leads to different insights - and the threats to quality are fundamentally different.

Note: some of the attributes that are important to Undiagnosed users are not important to diagnosed users, because they use the app differently - so not all rows are duplicated here.

Health Authority

Community

The wider community in which the app is deployed.  They may or may not be users of the app.

Local Businesses

Businesses such as restaurants could be significantly damaged by information that connects their location to users diagnosed positive.

Internet Service Providers

Partners

Partners are any organizations that have a relationship with Safe Paths, and are therefore stakeholders in our success.

Privacy Advocates

By positioning ourselves as privacy-first, we encourage the support of privacy advocates, campaigners & influencers.

By supporting us, they become stakeholders in our success, and could be harmed if we underdeliver against our promises.

INTERNAL STAKEHOLDERS

Testers 

Ripped from James Bach / Michael Boton Rapid Software Testing.  I can’t make this better by re-writing it.

Open

Developers

Product Managers

Sales & Marketing

Attributes that make for a compelling demo.

Attributes that make for compelling press.  The product may not be able to deliver these values by itself, but it is worth considering what the app can do to facilitate creation of this value.

Safe Paths Volunteers

I’ve run out of steam here.  This is definitely a valid stakeholder group - but do they need anything from the product that’s not already covered under one of the other stakeholders?

I can’t think of any, so I don’t see much value in mapping this out.

Ideas for identifying gaps in this document

Learn about standard KPIs.  WHat do Mobile APp creators typically monitor?  Why? Which stakeholders do these KPIs serve?

https://www.smartlook.com/blog/top-15-most-important-mobile-app-kpis-to-measure-the-performance/

Review GitHub issues (both open and closed).  For each bug, consider what value that bug threatens.

• Which stakeholder receives that value?

• Is it mapped above already?  If not, add it.  

• Is the value mapped, but the bug is radically different from the illustrated example?  If so, consider adding a 2nd illustrative example, to encourage broader thinking. Or perhaps sub-divide the attribute to highlight the multiple aspects.

Appendix; Functional Areas of the product

In test planning, it can be useful to consider any of the value attributes idetified in this document, in the context of individual areas of product function.

Drawing up a matrix of attributes vs. functional areas, and mapping the cells can be a helpful way to assess coverage and generate test ideas.

Key functional areas of MVP 1.0 app:

• Install / setup

• Import location data

• Export location data

• Load HA data

• Cross-matching local data vs. HA data

• Providing guidance based on collected data

• Newsfeed