Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Apple Health #47

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ Here you can download all of the data that Google has stored on you: <https://ta

To use this script, you only need to select and download your "Location History", which Google will provide to you as a JSON file by default. KML is also an output option and is accepted for this program.

You can also import [GPS Exchange Format (GPX)](https://en.wikipedia.org/wiki/GPS_Exchange_Format) files,
e.g. from a GPS tracker.

If you are using Apple Watch, you can export track data using the Health app on your iPhone:
- Open the Health app on your iPhone
- Tap on the profile icon in the top right corner
- Select "Export Health Data"

### 3. Clone This Repository

On <https://github.com/luka1199/geo-heatmap>, click the green "Clone or Download" button at the top right of the page. If you want to get started with this script more quickly, click the "Download ZIP" button, and extract the ZIP somewhere on your computer.
Expand All @@ -39,9 +47,11 @@ python geo_heatmap.py <file> [<file> ...]

Replace the string `<file>` from above with the path to any of the following files:

1. The `Location History.json` JSON file from Google Takeout
2. The `Location History.kml` KML file from Google Takeout
3. The `takeout-*.zip` raw download from Google Takeout that contains either of the above files
- The `Location History.json` JSON file from Google Takeout
- The `Location History.kml` KML file from Google Takeout
- The `takeout-*.zip` raw download from Google Takeout that contains either of the above files
- [GPS Exchange Format (GPX)](https://en.wikipedia.org/wiki/GPS_Exchange_Format) files
- Zip files exported from Apple Health

#### Examples:

Expand Down Expand Up @@ -80,16 +90,16 @@ python geo_heatmap.py --min-date 2017-01-02 --max-date 2018-12-30 locations.json
#### Usage:

```
usage: geo_heatmap.py [-h] [-o] [--min-date YYYY-MM-DD]
[--max-date YYYY-MM-DD] [-s] [--map MAP]
file [file ...]
usage: geo_heatmap.py [-h] [-o] [--min-date YYYY-MM-DD] [--max-date YYYY-MM-DD] [-s] [--map MAP] file [file ...]

positional arguments:
file Any of the following files:
1. Your location history JSON file from Google Takeout
2. Your location history KML file from Google Takeout
3. The takeout-*.zip raw download from Google Takeout
that contains either of the above files
- Your location history JSON file from Google Takeout
- Your location history KML file from Google Takeout
- The takeout-*.zip raw download from Google Takeout
that contains either of the above files
- A GPX file containing GPS tracks
- An Export.zip file from Apple Health

optional arguments:
-h, --help show this help message and exit
Expand Down
73 changes: 68 additions & 5 deletions geo_heatmap.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
import ijson
import json
import os
from progressbar import ProgressBar, Bar, ETA, Percentage
import posixpath
from progressbar import ProgressBar, NullBar, Bar, ETA, Percentage, Variable
from utils import *
import webbrowser
from xml.etree import ElementTree
Expand Down Expand Up @@ -97,7 +98,30 @@ def loadKMLData(self, file_name, date_range):
self.updateCoord(coords)
pb.update(i)

def loadZIPData(self, file_name, date_range):
def loadGPXData(self, file_name, date_range, make_progress_bar=ProgressBar):
"""Loads location data from the given GPX file.

Arguments:
file_name {string or file} -- The name of the GPX file
(or an open file-like object) with the GPX data.
date_range {tuple} -- A tuple containing the min-date and max-date.
e.g.: (None, None), (None, '2019-01-01'), ('2017-02-11'), ('2019-01-01')
make_progress_bar -- progress bar factory (specify NullBar to suppress progress bar)
"""
xmldoc = minidom.parse(file_name)
gxtrack = xmldoc.getElementsByTagName("trkpt")
w = [Bar(), Percentage(), " ", ETA()]

with make_progress_bar(max_value=len(gxtrack), widgets=w) as pb:
for trkpt in pb(gxtrack):
lat = trkpt.getAttribute("lat")
lon = trkpt.getAttribute("lon")
coords = (round(float(lat), 6), round(float(lon), 6))
date = trkpt.getElementsByTagName("time")[0].firstChild.data
if dateInRange(date[:10], date_range):
self.updateCoord(coords)

def loadGoogleTakeOutZIPData(self, file_name, date_range):
"""
Load Google location data from a "takeout-*.zip" file.
"""
Expand Down Expand Up @@ -133,6 +157,40 @@ def loadZIPData(self, file_name, date_range):
raise ValueError("unsupported extension for {!r}: only .json and .kml supported"
.format(file_name))

def loadAppleHealthData(self, file_name, date_range):
"""
Load location data from an Apple Health Export file.
"""
with zipfile.ZipFile(file_name) as zip_file:
namelist = zip_file.namelist()
all_routes = fnmatch.filter(namelist, "apple_health_export/workout-routes/*.gpx")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this path shouldn't be hard coded if it changes in the future.

filtered_routes = [name for name in all_routes if dateInRange(posixpath.basename(name)[6:16], date_range)]

w = [Bar(), Percentage(), " ", Variable('name', width=30), " ", ETA()]
with ProgressBar(max_value=len(filtered_routes), widgets=w) as pb:
for i, name in enumerate(filtered_routes):
pb.update(i, name=posixpath.basename(name))
with zip_file.open(name) as read_file:
self.loadGPXData(read_file, date_range, make_progress_bar=NullBar)

def loadZIPData(self, file_name, date_range):
"""
Load location data from a zip file.

Supported zip files:
- Google Takekout
- Apple Health export
"""
with zipfile.ZipFile(file_name) as zip_file:
namelist = zip_file.namelist()
if fnmatch.filter(namelist, "Takeout/*.html"):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really like that the files are detected by their names. Can't we detect them with file and say html is google and xml is apple or something like that?

self.loadGoogleTakeOutZIPData(file_name, date_range)
elif fnmatch.filter(namelist, "apple_health_export/Export.xml"):
self.loadAppleHealthData(file_name, date_range)
else:
raise ValueError("unsupported ZIP file {!r}: only Google Takeout and Apple Health supported"
.format(file_name))

def updateCoord(self, coords):
self.coordinates[coords] += 1
if self.coordinates[coords] > self.max_magnitude:
Expand Down Expand Up @@ -184,6 +242,8 @@ def run(self, data_files, output_file, date_range, stream_data, tiles):
self.loadJSONData(json_file, date_range)
elif data_file.endswith(".kml"):
self.loadKMLData(data_file, date_range)
elif data_file.endswith(".gpx"):
self.loadGPXData(data_file, date_range)
else:
raise NotImplementedError(
"Unsupported file extension for {!r}".format(data_file))
Expand All @@ -203,9 +263,12 @@ def run(self, data_files, output_file, date_range, stream_data, tiles):
parser = ArgumentParser(formatter_class=RawTextHelpFormatter)
parser.add_argument(
"files", metavar="file", type=str, nargs='+', help="Any of the following files:\n"
"1. Your location history JSON file from Google Takeout\n"
"2. Your location history KML file from Google Takeout\n"
"3. The takeout-*.zip raw download from Google Takeout \nthat contains either of the above files")
"- Your location history JSON file from Google Takeout\n"
"- Your location history KML file from Google Takeout\n"
"- The takeout-*.zip raw download from Google Takeout \n that contains either of the above files\n"
"- A GPX file containing GPS tracks\n"
"- An Export.zip file from Apple Health"
)
parser.add_argument("-o", "--output", dest="output", metavar="", type=str, required=False,
help="Path of heatmap HTML output file.", default="heatmap.html")
parser.add_argument("--min-date", dest="min_date", metavar="YYYY-MM-DD", type=str, required=False,
Expand Down