These flows are being developed in late-July 2020, and are intended for use with the SAF and PLACES Jira products (i.e. the GPS Mobile App, and Safe Places).
This is an early draft of the processes we hope to follow. Feedback welcome.
Contents:
Basic FAQ for Testers
What testing work should I work on?
As a tester, you’ll work on two key types of Jira ticket:
Bug retests, when fixes come back from Dev. Tracked in “Bug” tickets.
Testing for new features or functionality. Tracked in sub-tasks labelled “QA:”
What should I work on first?
For inexperienced testers, or those new to the project, bug retests are typically going to provide an easier point of entry. Test queues for bugs that are ready to test are here:
https://pathcheck.atlassian.net/issues/?filter=10059 (Apps only)
https://pathcheck.atlassian.net/issues/?filter=10060 (SafePlaces only)
Ticket Types
This section covers the different types of ticket we use to track testing work in Jira, and the main flows that we use.
Bugs workflow
Bugs are raised as described here: How to raise bugs found in Testing
Once a bug is fixed, Developers should:
Set status to READY FOR TEST
Include some details of the fix:
Which build will contain the fix
Any particular concerns or risks that need testing
Assign back to the original raiser (unless they know the original raiser is no longer on the project)
If the original raiser is no longer on the project, they should assign to “Triage QA”
When a Tester receives a bug in READY FOR TEST state, they should:
Assign themselves to the ticket
Review the notes on the ticket and add further clarifying information
Once you being to work on the bug fix, transition status to IN TEST
Once you have completed testing, transition status to DONE
Update the testing you completed for the bug fix in the ticket
If you find a problem with the bug fix, then transition status to IN PROGRESS and assign back the the Developer
If you get stuck on a ticket and cannot progress, assign the ticket to “Triage QA” where one of the leads will pick up and review
Review the notes & form a judgment whether they can do the testing
If they can test in the next few days, they should do so
If they can’t test in the next few days, but the testing is fairly straightforward, they should
Add clarifying notes explaining what testing needs to be done
Assign the ticket to “Test Queue”
If they can’t test in the next few days, and the testing is complex or specialised, they should
Add clarifying notes explaining what testing needs to be done, and highlighting the specific difficulties
Assign the ticket to “Triage QA”
Once a tester begins working on testing a bug fix, they should transition the status to IN TEST
Once a tester completes testing of a bug fix, they should transition the status to DONE
If the tester finds problems with the bug fix, they should transition status to IN PROGRESS, and assign back to the Developer. In some cases, it will be worth reaching out on slack as well, to ensure the Dev & Tester are in sync.
New Features workflow
New Features are tracked by either Stories or Epics (with just a couple of minor differences between them in terms of process).
Once a feature is ready for test, the Developer should:
Set status to READY FOR TEST
Include some details of the new feature:
Which build will contain the new feature
What the design of the new feature is - including links to Figma resources & any other design reference material.
Any particular concerns or risks that need particular attention from the QA team.
Assign to “Triage QA”
When a Tester receives a new feature in READY FOR TEST state, they should:
Assign themselves to the ticket
Review the notes on the ticket and add further clarifying information
Once you being to work on the new feature, transition status to IN TEST
Once you have completed testing, transition status to DONE
Update the testing you completed for the new feature in the ticket
If you find a problem with the new feature, then transition status to IN PROGRESS and assign back the the Developer
If you get stuck on a ticket and cannot progress, assign the ticket to “Triage QA” where one of the leads will pick up and review
New stories and epics assigned to Triage QA will be monitored by a team of QA product leads.
When a card is assigned to “Triage QA”, one of these leads will:
Review the information provided by Dev and clarify any missing info
Identify any particular issues that may exist with testing this function, any blockers etc.
Provide detailed set of test guidance for the feature, that a tester will be able to follow
Create one or more sub-task tickets tracking the specific testing that needs to be done:
These sub-tasks always have a name beginning “QA:”
Each sub-task includes detailed guidance on the testing to be done.
See below for details of the life-cycle of these sub-tasks.
The state of the card remains “READY FOR TEST” - testing status is tracked under the individual sub-tasks.
The above applies equally to Epics, Stories, and occasionally to bugs (where the test lead determines that substantive testing is needed for the bug fix, over & around just validating the original bug scenario).
Note that when the card in question is an Epic, an additional child Story card must be created first, and the sub-task created under that.
”QA:“ sub-tasks workflow and tasks
“QA:” sub-tasks are created by product test leads detailing the testing required for an Epic, Story or complex bug
They have a life-cycle as follows:
Initially created as OPEN, with an owner of either:
The test lead; who will detail the testing to be done
or “Triage QA” if there’s no-one immediately available to articulate the detail of the testing.
Once the required testing is defined, and ticket is suitable for testing, the card is updated:
Status moved to READY FOR TEST
Owner is set to either an individual tester if available, or “Test Queue” if not
Now the ticket is ready to test, the tester picks up ticket from “Test Queue” then;
Assign themselves to the ticket
Performs the testing outlined in the ticket
Raises any issues / concerns about the scope of testing with the Test Lead
Raises any bugs as necessary and they involve the Dev team and/or a QA lead in the event of any issues
Records details of testing performed in the ticket
For lightweight notes, this can be directly in the ticket
For more extensive notes, they can link to a Test Report filed in Confluence (E.g. here: Test Reports)
Once testing is complete, before transitioning the ticket to DONE, they have a debriefing conversation with the relevant test lead, covering:
What has been learned in testing, and whether any further testing should be planned
Whether the documentation of the testing done is adequate
Any testability issues, or impediments, that we should be looking to resolve
Once this debrief is completed, the ticket can be transitioned to DONE.
Note that a testing ticket can be transitioned to DONE even if there are open bugs raised in that testing, as long as the tester & the debriefer are happy that the bug retests to follow will adequately cover the remaining risks. This will typically be the case for minor non-blocking bugs, but will typically not be the case for major blocking bugs.
Roles & Responsibilities
This section explains the key roles within the QA team, and what should be done by people in each of these roles, to keep the flows described above working.
Tester
Our intention is that Testers should have a fairly simple workflow.
A tester should:
Assign tickets to themselves that they are working on
Always add detail of what you tested, and what you didn’t for any ticket, and review with the test lead
If they don’t have any work assigned, review work in the “Test Queue”, and assign this work to themselves, and do it! Where set, use ticket priority as a hint as to what order to tackle things in.
Test queue is here:
Brand New testers will probably find it easier to start with Bugs, and then move on the “QA:” sub-tasks when they have a bit more familiarity with the products.
If the tester cannot complete the ticket in a reasonable timeframe, and that work could reasonably be assigned to someone else, pass that work back to “Test Queue” so that someone else can pick it up
If a tester finds work assigned to “Test Queue” that is not clearly defined enough to progress, they should comment to explain the issue and assign back to “Triage QA” with comments explaining why
Test Leads
Product Test Leads need to regularly (at least once a day) perform the following.
Manage the “Triage QA” list
Review all Bugs, Epics and Stories in “Triage QA”
Process and create “QA:” sub-tasks, clarifying them to the point where they can be set READY FOR TEST
The Triage list is here:
https://pathcheck.atlassian.net/issues/?filter=10056 (All Products)
https://pathcheck.atlassian.net/issues/?filter=10057 (SAF only)
https://pathcheck.atlassian.net/issues/?filter=10058 (PLACES only)
Monitor (and manage when necessary) the “Test Queue” list (BUGS)
Keep en eye on the length of the “Test Queue” list, and ensure that we are progressing through the required testing at a pace that will meet our commitments to Dev / PM.
Look out for items that are “stuck” on the “Test Queue” list. Look for any comments that indicate people are struggling to progress particular items. Also look out for items where nobody is commenting, but everyone seems to be avoiding!
Test queues are here:
Current Test Leads
Current Test Leads are Diarmid Mackenzie and Stella Nelson , currently sharing work across both SAF and PLACES.
Other experienced testers who would like to contribute as test leads, please do let us know.