23 May & 26 May - Testing with updated Android build

Owned by Diarmid Mackenzie
Last updated: May 26, 2020
5 min read

Intial test results from testing with an updated Android build, that intends to address reliability & accuracy issues with GPS.

Headlines:

  • Testing so far is limited, but it looks like we have made significant progress on both aspects

  • Some concerns remain - I plan more testing to try to better understand the specific issues that may remain.

Summary of changes

Android Apkhttps://tinyurl.com/covidsafepath

Notes

- With the current build, we are fetching the current location every 5 minutes interval. Within 5 minute intervals, we are taking 5 samples of locations and checking for the < 20 m accuracy.

- The location with the best accuracy from 5 samples will be stored in the realm database.- Out of 5 samples, if we did not find any location with < 20 m accuracy. Then we are not saving location for that 5-minute interval. That means we will have a gap for now for that interval.

- If the user forcefully stops the application from app info, then it's impossible to trace the location.

- If the user restarts the device, then we are having a known issue of not restarting location service again. But we are working on this and trying to get this resolved as soon as possible.

One important note:- It might be possible that after 2 min if the user stop app and start after 2 hours then those 2 hours will not be recorded and can not be treated and that user was there at the same location.

Test Results

These results are after 16 hours. During this time, the phone was left powered on in a single location (indoors).

Initially the app was open, but was closed after an hour or so - I don’t have a record of the precise time.

Reliability

The chart on the left shows the time gaps between GPS logs (on a log scale) - which is much more reliable than the data from the previous build (20 May) on the right (albeit over a slightly shorter time period).

Remaining points of concern on reliability.

  • It looks like background performance (with the app closed) is where the most improvement is seen

  • During the initial period, the app was in the foreground, and we still seem to have a lot of delays in the ~10 min region.

  • There is a single worrying data point right at the end. At 10:26 UTC, I started the app, and exported location data. At this point, I had a 58 min data gap, and a 39m leap in position. Given that data collection had been rock solid before this, I am suspicious that some data may have been discarded - I have no data from 09:24 UTC, even though I did not start the app up until just before 10:26.

  • (though it may be that I was using the phone for something else from 09:24 UTC onwards - I hadn’t been paying close enough attention.

So more testing needed to analyze cases where:

  • App is in foreground

  • App is in background, but phone is in use for other activities

  • App is brought to foreground & data exported.

Accuracy

Distance variation (left) is improved massively from previous test results (right)

My records on the previous test (right) aren’t graet (sorry!). It may be that the phone was in 2 different locations, approx 20m apart over the course of the test.

But even so, we have clearly eliminated a large number of wayward points, and variation of position is well within the 20m accuracy target.

The only immediate points of concern would be:

  • The final data point, which shows a sudden move of 39m. The phone had remained stationary. This correlates to the app being starting up, and also seems to correlate to a loss of about an hour’s data (see reliability above). I will aim for some more testing around these operations.

  • We have only tested with a stationary phone. Next step would be to test the app with 2 or 3 phones moving around together, and compare accuracy.

Battery usage

My phone reports 2% of battery usage by the app over the last 16 hours.

My phone is a Moto G7 Power, with a massive 5,000 mAh battery.

However, previously I had seen COVID Safe Paths report 4% battery usage over an 8 hour period.
26 April - Evaluation of GPS Logger (Basic Air Data)

So there’s certainly no evidence so far that battery consumption has got any worse, and it may even have got better. More testing needed, but no immediate concerns here.

 

Update - 26 May

Further testing from 6pm on Saturday 23 May to evening on Tuesday 26 May suggests improvements may be less significant than initially thought.

My phone has been logging in the background over the weekend. After a phone reboot on Saturday afternoon, logging had stopped (don’t yet have the fix for that problem running). I restarted the app at 6pm.

Since then, I have been moving about so can’t give a reliable position on accuracy.

But reliability has been poor - no better than before the fixes.

Here’s a graph of the data (log scale so that we can see both small & large delays on a single scale)

This looks much like it did previously.

Over the period, we had

  • Just 61% of the expected data points (535 of 879)

  • 22% > 6 mins apart

  • 17% > 9 mins

  • 3% > 14 mins

  • 2% > 29 mins.


Notable long delays:

Saturday night:

  • 00:50:43, 01:56:41, 03:56:21, 01:42:50

Sunday night:

  • 02:54:05, 05:48:28

Monday night:

  • No big gaps (steady 5 min logs throughout).

Daytime gaps > 20 mins:

  • Sun: 26:45, 53:41, 34:54, 20:56

  • Mon: 24:59,

  • Tue: 29:59, 42:28, 51:40.

 

So we are still seeing:

  • Very long gaps at night

  • Occasional long (30-60 min) delays in the day.

  • Plenty of occasions where logs are 6-10 mins apart.

Overall, I see no significant improvement from the previous software. Which is odd, because the 1st 24 hours after install of the new software seemed much better.

Either that was just a lucky period of time - or there is something unusual abotu a fresh install of the app.