Coverage & Comparison Testing

[bardram]

• iOS coverage
• Android coverage
• iOS comparing phone to server
• Android comparing phone to server
• Make blog post on homepage

[bardram]

I've collected data on iOS over 2 days, which has lead to findeing a couple of issues - #470 #471. But - there are still a lot of issues - it seems like only 1/3 of the data is uploaded to CAWS. Will continue to look into this.

[bardram]

Did an over-night coverage test on Android - Samsung Galaxy A54 - Android OS 14. This worked fine, until the app crashed due to this issue #474.

But till then, the comparison looks like this:

which overall is fine. Most of the missing data in CAWS is still in the local database, while not uploaded before the app crashed.

Though, it seems like error measurements are still not uploaded - #470

[bardram]

Coverage on Android looks like this on local SQLite DB data:

And like this from the JSON from CAWS:

[bardram]

Did sampling of data on Android (like above) for 4-5 days:

Deployment :  15c12adb-b6ce-408f-99a0-f54c944416d9
Start      :  2025-02-27 18:54:34.292609
End        :  2025-03-05 08:27:10.579898

which gave this result:

                   SQLite    CAWS Delta Percentage
Measurement                                       
heartbeat            4576    4593    17       0.37
triggeredtask        6189    6220    31        0.5
completedtask          34      35     1       2.86
stepcount           19931   19987    56       0.28
ambientlight         1019    1023     4       0.39
timezone               39      40     1        2.5
screenevent           184     188     4       2.13
freememory           5102    5124    22       0.43
batterystate        14201   14211    10       0.07
currentlocation      5139    5160    21       0.41
weather               332     334     2        0.6
airquality            331     333     2        0.6
survey                 37      38     1       2.63
deviceinformation       2       2     0        0.0
activity              640     644     4       0.62
polar.hr           150078  150477   399       0.27
error                   4       4     0        0.0
mobility                1       1     0        0.0

This shows that there is actually MORE measurements in CAWS than in the local SQLite DB....?

But this is due to that the app was reinstalled - and hence the local DB deleted - a couple of times on the start of the study.

So - all in all - it seems like all data is uploaded safely to CAWS (except for #470).

[bardram]

Below is the coverage analysis for the first 29 hours of the Android data set collected above.

There are the following observations:

• Collection of Polar HR data keeps the app running and collecting data. When the polar device is connected the sampling rate for all other measures (air, light, location, etc.) are 100%.
• When the Polar device is disconnected (at time 28-05:00), coverage drops to 0%
• Charging the phone (at time 28-14:00) activates the app and sampling is 100% (even though the Polar device is not connected (yet))
• Mobility is not calculated or collected - Mobility Features are Not Collected #477

Note that on this phone nothing has been done to optimize for background sampling.

[bardram]

Following the tips on the wiki and the Reddit thread on optimization of background app on Samsung, I did the following:

• Disable battery restrictions: Do the following from the Setting > Battery screen:
  • Turn off Power Saving.
  • Click on Power Saving, click on the 3 dot menu in the top right, and select Adaptive power saving. Turn it off.
  • Click on Background usage limits and turn off Put unused apps to sleep. Under Never auto sleeping apps add the app you want to keep alive (e.g. CARP Studies app) by clicking the + button on the top (need to check up on this - CARP Studies app is not listed???)
• Unrestricted Battery Usage: Go to the app's App Info page.
  • Set Remove permissions if app is unused to off.
  • Go to Battery and set it to Unrestricted.
• Excluded Memory Apps: Go to Device Care > Memory > Excluded Apps and the app you want to allow to be cached. This isn't super reliable, but some apps will benefit.
• Keep Open: In the Android multitasking view click the icon above the app and select Keep open.

I did all of this, but according to the Reddit thread, it seems like the last option -- to mark the app as "Keep open" in the multitasking view -- is the best option.

[bardram]

After running the app with the improvement above implemented for 31 hours, I get this picture:

There are no significant improvements to background sensing ☹️:

• The app is kept alive while the HR is samples and the BLE device is connected (07-11:00 to 07-18:00) and sampling is 100%
• Once disconnected during the night, sampling is reduced to 13-17%
• The next morning when surveys are filled in once per hour (08-09:00) sampling is increased but only to 50-60%
• Once the app is no longer used for surveys (09-14:00), sampling is decreased to 20-30%

Notes:

• the phone was not charged during this period (which is something that affects the background mode).
• the many missing weather and airquality measurements are due to the phone not being on the Internet most of the time. The phone does not have a SIM card.

[bardram]

Will try to add the location measure to the protocol - this collects location in the "background mode" of the location plugin. Will also change the configuration of the LocationService to a higher precision.

Testing if this improves things.

[bardram]

Here is the data from the iOS testing (iPhone 12Pro, iOS 18).

We observe:

• The connected HR device keeps the app alive also during the first night
• The app is inactive during the second night since the HR device is disconnected.
• Using the app to fill in surveys once per hour seems to keep the app running.
• Mobility features are collected (see Mobility Features are Not Collected #477)
• Heartbeat measurements are double - 200%?

[bardram]

Will try to add the location measure to the protocol - this collects location in the "background mode" of the location plugin. Will also change the configuration of the LocationService to a higher precision.

Testing if this improves things.

Now I configure the LocationService to a higher precision and sampling rate, like this:

  var locationService = LocationService(
    accuracy: GeolocationAccuracy.high,
    distance: 0,
    interval: Duration(seconds: 1),
  );

I.e., high precision, zero distance filter and one second sampling interval.

This gave this picture:

Hence:

• a good coverage during day time when the phone is used and moves around
• mobility features are being sampled on a regular basis
• still no sampling during the night when the phone is sitting still
• heartbeat is below 100% during the day, which is strange?