Efficacy Measurement Capabilities
- Diarmid Mackenzie
List of suggested measures from Kyle + comments + actions. Plus some more potential measures.
This focuses on what we can realistically meaure, not what targets might be - that is for separate consideration.
Metric | Comments | Actions |
|---|---|---|
Daily Downloads iOS | Can we measure this by country? What about in a given county/state in the US? | Look at what metrics are available in App Store |
Daily Downloads Android | Can we measure this by country? What about in a given county/state in the US? | Look at what metrics are available in Play Store |
Cumulative Downloads iOS | As per daily |
|
Cumulative Downloads Android | As per daily |
|
Daily Active Users ?? | We can measure how many downloads of JSON data there are from the HA - but this is HA infrastructure not our software | Explore how we might get download counts for JSON data. |
Daily New Cases | We will need HA to provide this info | Add to list of asks for the HA |
Number of Cases Contact Traces usind Safe Places. | We should be able to get this from Safe Places | Discuss with Safe Places team how we could get this |
Number of New Cases added to HA JSON data | We should be able to get this from Safe Places (even though it’s not derivale from the JSON data) | Discuss with Safe Places team how we could get this |
Daily number of “points of concern” added to HA JSON data | We should be able to get this from Safe Places, or JSON data | Discuss with Safe Places team how we could get this |
Number of data points redacted during Contact Trace interviews. | We should be able to get this from Safe Places, or JSON data | Discuss with Safe Places team how we could get this |
How many diagnosed cases had app | We will need HA to provide this info | Add to list of asks for the HA |
How many contacted PHA because of app | We will need HA to provide this info | Add to list of asks for the HA |
% of people notified with exposure test positive | We will need HA to provide this info. Because positive test may come some days after exposure notification, they will need appropriate record keeping to provide this info | Add to list of asks for the HA |
Total exposure counts for each patient that contacts the HA. To allow us to test correlation of exposure notifications with +ve / -ve test results. | We will need to ask the HAs for this data - they will need to ask the patients. | Add to list of asks for the HA - sensitive as it requires collection of additional data from patient. |
For a patient contact traced using Safe Places, how many of their data points matched existing “points of concern” | Safe Places should be able to generate this. | Check technically possible with Safe Places. |
Location history analysis of new infected carriers compared to hotspots | Didn’t understand this - aren’t the “hotspots” by definition the places where the most new infected carriers have been? | Check with Kyle. |
% of contact tracers trained | We will need HA to provide this info | Add to list of asks for the HA |
% positive test rate | We will need HA to provide this info | Add to list of asks for the HA |
HA Contacts / day relative to new cases | Just a ratio of other values already being gathered. |
|
HA Beacon Data (Refreshed per day) | Didn’t understand tis - does this relate to BLE? | Check with Kyle. |
Devices registered against HA endpoint (selected the JSON) | I think this is a dupe of “active users” above (if active users = “opened the app” than (1) we can’t determine that without something like firebase, and (2) it’s not how the app is used anyway. |
|
iOS App uninstalls | I don’t think we can measure this - will have to infer from drops in pulls of JSON HA data |
|
Android App uninstalls | I don’t think we can measure this - will have to infer from drops in pulls of JSON HA data |
|