A place to log possible risks of harm that could result from the Safe Paths App. Intention is that we grow this list as we encounter new considerations about how the product could lead to harm. Also that we regularly review this list, and use it to help product direction such that we mitigate these risks as far as possible.
All project participants are invited to add concerns to this list
Identified Risks of Harm
Increased social inequality
Poorer communities have less access to smartphones
Poorer communities live more densely, and may be more prone to false positives & false negatives than more privileged communities
COVID-19 is disproportionately hitting ethnic minority communities in the US. Poor people will use this app (and therefore be at risk of any harms it may cause), while rich people never will.
Misuse / abuse of data by Health Authorities of other Government Agencies. E.g. Concern that data heat maps could lead to “kettling” of deprived communities to contain the spread.
Corruption
Contact tracer may redact certain business from traces, in exchange for compensation from them.
Harm to small businesses
Concern about harm to small businesses, because of information being spread that there was a COVID infection there. Larger businesses will be able to cope, but small businesses will have less resilience.
Health status stamps
Could be problematic, and lead to the App becoming de facto mandatory, even if that was not our original vision.
Unregulated Health Authorities (mostly outside the US)
May not respect our vision that private data should only be shared with consent.
Reckless beahviour due to misinformaton (False Negatives)
Particularly if the app is not very careful to indicate the limits of information it has (e.g. perhaps there is no HA data downloaded, or perhaps no location data has been recorded by the App).
Fatigue due to excess False Positives.
If the App “cried wolf” this could lead to disregarding not only info from the app, but also possibly from other sources of guidance as well.
Reckless behaviour due to false sense of safety
Even if the app does not have high levels of false positives or false negatives, there is a risk that people substitute use of the APp for other more-effective controls of spread, e.g. hand-washing.
Problems caused by leaks of personal data, e.g. location information
Individuals attacked or discriminated against having been identified as spreaders of the disease
Individuals exposed to harm following leaks of location info to another person (e.g. Domestic Violence)
Leaked location information used inappropriately by security services or law enforcement.
“Griefing” - deliberate injection of false COVID infection reports into the system, for any number of reasons:
Business competition
Entertainment
Civil disobedience / disruption
Foreign attackers
Organized crime
Actions we could take to identify further Risks of Harm
STRIDE threat modelling - see “Security” pages.
“Abusability Testing” -
https://www.wired.com/story/abusability-testing-ashkan-soltani/
https://www.usenix.org/node/226468
Diversity - Open Source Software has a diversity issue. More diversity in our project volunteers might help identify more risks of harm.
Other Risk-related brainstorming activities