Timezone correcting Fitbit sleep data using Google Timeline
Published February 12, 2024
Hello friends, I purchased an Apple Watch and (hopefully) escaped from the hellhole that is the Fitbit data environment — forever!
Custom Apple Watch strap that turns it into a tilted driver watch pic.twitter.com/Ktp1nmskyw
— Andy (@oldestasian) February 11, 2024
To cap off all my Fitbit posting, I exported my data one last time and noticed that they have restructured the data and ruined my stress method for timezone correction, rendering all my latest data unusable. Since I've collected more Tetris and Stroop data that I wanted to correlate, I decided to try one final method that I've always thought about but never cared enough to do — syncing my Google Location with Fitbit's data to get UTC timestamps. I'll describe the steps verbally and the code will be at the end.
Method
First I loaded my Google location data with this neat tool by Scarygami which produces a massive CSV of all the longitude and latitude and everything else. Here I visualized part of it using plotly's scatter_mapbox function.
From each longitude and latitude, you can get the timezone name using the Python library timezonefinder, which works at a rate of ~2ms/1000 coordinates without needing to do GET requests. I converted the timezone strings into timezone offsets using some code from this SO post.
Then I realized that I needed to also account for daylight savings time across two regions, Europe and America, which for some reason have DST dates that are about a week off from each other. I can't wait for them to abolish DST so I'll have to add a bool which determines if DST correction is necessary in my timezone correction pipeline /s
Some sleep sessions have no "close" longitude/latitude coordinates, so I toss them. Close is defined arbitrarily, I use 6 hrs as my cutoff.
My success rate with this method is ~60%, and most of the loss comes from me neglecting to turn on Google Timeline on my iPhone until sometime mid-2021. In total, I recovered 600+ days of accurate Fitbit sleep data and can now produce fun sleep graphs which I can trust. For instance, here is my true Tetris speed vs. sleep graph:
Turns out sleep is good for you, actually.
Code
Code is provided with no guarantees or user-friendly comments. Use at your own risk, etc. etc.