Safe Paths Test/QA Team Strategy

This is our strategy as a test team. That’s different from our Product Test Strategy (which is … coming soon!). This strategy talks about what kind of team we are building, what it’s for, and how we are doing it.

1st draft created by DIarmid Mackenzie - 21 April 2020. Comments are welcome. If you want to make edits, please discuss with me first.

Mission & Vision

Mission: To provide the best information we can to the rest of the Safe Paths team (at all levels) about threats to the goals of Safe Paths project.

The best information is (among other things):

• Clear

• Timely

• Explicit about its limitations
  
  

Vision: All levels of the Safe Paths organization have good information and understanding of the quality & nature of what they have built (and are building), to inform their decision-making.

Key Pillars

The key pillars of our strategy are:

• People

• Focus on Value

• Connections to other teams

• Self-Direction

• Diversity of Approaches

• Transparency

• Support & Feedback

People

We are particularly looking for people with the following attributes. We seek out contributions from those who have these attributes already, and we encourage everyone on the project to develop these attributes

• Commitment to the craft of testing

• Specific relevant domain or tooling expertise & experience

• Curious, inquisitive, open to new ideas

• Clear & candid communications

• Generous & supportive community participants

We attract these people to the project, and the test team, though

• A clear articulation of the Safe Paths project goals, and the test team’s role within that

• Outreach using existing channels in the Test Community, and any other connections we may have

• Continuously working to create an environment in which people can do their very best work.

Focus on Value

As far as possible, we try to relate the information, concerns & insights we share with the rest of the Safe Paths team, towards the value that this project is trying to deliver.

This includes providing support for both:

• Maximizing the benefits that this project can deliver

• Minimizing the harm that this project can do. And we must all be aware that this project is at risk of doing significant more harm than good to individuals and communities. It is our responsibility to support the rest of the organization in ensuring that does not happen.

There are many artifacts that we will use to support our testing: specifications, test plans, UI mock-ups etc. etc. However, we never lose sight of the fact that these artifacts are fallible, and that delivering value to project stakeholders is more important than conformance to any of these artifacts.

Our Safe Paths Quality Map (which everyone is invited to help develop and shape) helps us to stay aligned with this principle.

Connections to Other Teams

We consciously build and develop connections with other teams on the Safe Paths project, so that we can serve them as trusted providers of information about what we are really managing to build and deliver.

• Our relationships with the Product and Dev teams are fundamental

• However, we also aim to build relationships with folks in Communications, Implementation, and the overall Leadership team.

We build these relationships through

• Direct outreach to individuals in these teams, at all levels. We seek to understand how, as Testers, we can best serve & support them in their goals.

• The creation and sharing of useful information and models, that can support them in doing their jobs.

We recognize that people from different disciplines and backgrounds may use very different language & terminology from us, and we do our best to reach out the them in terms they understand, rather than expecting them to understand the “language of testing”.

Self-Direction

We are a non-hierarchical self-organizing team. While certain individuals can (and must) take the lead on particular initiatives, there are no structures dictated to the organization.

We may offer suggestions and ideas to people for how they can help us, but these are suggestions, not orders.

Three behaviors are fundamental to this working successfully:

• We don’t wait to be told. If we see something that’s not being done, that we believe is important, we tell other people what we are proposing to do, give others a reasonable chance to express any thoughts or concerns, and then we do it.

• We do what we say we are going to do, If we change our minds later, we tell people so that they can choose to step in and take over.

• When we need help and support, we ask for it. Everybody in this team is going to be taking on challenges they have never done before. We all need help and guidance, and asking for it is a sign of strength, not weakness.

As a team, we hold each other to account on all of these behaviors.

Diversity of Approaches

In the Software Testing community, there are a wide range of different ideas about how to approach testing, and some of these differences can lead to heated, long-running and seemingly irresolvable debates.

We aim to create a community where people with a different range of approaches can work alongside each other, inspire and learn from each other.

Approaches that are welcome here include (but are not limited to)

• Manual scripted testing (or “checking”), measuring the product against a tightly defined set of expectations.

• Testing that follows a plan, derived from key product documents such as Requirements and Specifications

• Exploratory Testing, which may be tightly focused in a given area, or de-focused and holistic.

• Technical testing, using software tools (whether 3rd party or homegrown) to deeply explore the product (e.g. Security Scans, HVAT)

• Developing, running and maintaining suites of Automated Tests (or “checks” if you prefer)

• Testing in Production.

The one shared principle that we hope that all testers will try their best to align with is the “Focus on Value” principle described above.

Transparency

We believe that the quality of this product can be increased, and the risks of harm greatly decreased, by being as open and transparent as possible in what we are do. This is a key principle not just of the Test Team, but the entire project, going right back to the decision to develop our products as Open Source.

While we have many talented people contributing on this project, there are also people outside the project who can offer extremely valuable insights, even if they would not choose to formally volunteer on the project. Here’s just one example of this.

As well as the open GitHub repo (including all our bugs, past & present), this Confluence space is also publicly visible, and open to public comments (although only project members can edit content).

Over time, we expect to do more to increase the visibility of, and engagement with, these public spaces, to build a strong community of interest outside the project, to complement the efforts of those working inside.

Working in public may creates additional challenges, but the light that it shines on our work will be invaluable in holding us to the highest of standards on the work we do.

Support and Feedback

We are a community, and we can achieve more as a collective than we can as individuals.

As well as a focus on our own work, we look around us for others who may be struggling or in need of support, and offer to help them in whatever way would be useful to them.

Many people are putting in long hours of their own personal time towards this project, as well as continuing to meet their commitments to their work, their family, and their community - not to mention the additional stresses of lockdown, such as social isolation, school closures etc.

Before offering feedback, we always take care to check that the other person is open to hearing some feedback at that point in time,

Public artifacts created by team members are assumed to be open for feedback by default. Where possible, we explain to team members how we’d like that feedback - e.g. should they add comments to the document, or should they just go ahead and edit?