GPS Data Corruption in XLS Format for Longer Measurements

[physken_JPN]

I am a high school teacher from Japan. Thank you for developing phyphox. In my classes, it's an indispensable tool for small-scale experiments.

Issues:

When saving long-duration GPS measurements as xls, the file gets corrupted.
I instruct my students to analyze the GPS data and then upload it to Google Drive and work on it in Google Spreadsheets.
Data for about 10 minutes seems to display without issues, and I had no problems in my preliminary tests.
However, I've received reports from some students that their files are corrupted and can't be opened on Google Drive. One student's file had about 900 rows of data. When I downloaded it on a Windows PC and repaired the file in Excel, it was restored to normal.
I'm hoping for an output file format that won't get corrupted.
If immediate improvements are challenging, could you provide information on how much data or duration might lead to this issue?

Looking forward to further developments of phyphox. Thank you.

[Jens Noriʇzsɔɥ]

Thank you for the report. The data is exported to the old binary Excel format that has some issues, however, the amount of data appears to be far from its limits (we have received reports about the 65k something row limit in .xls files).

I will do a test run on my smartphone, 900 rows at one row per second should be about 15'…

It looks that I could reproduce it: see post below. For future issues: If they have saved their experiments to the menu*, you could share one of those with broken output by mail (GPS, for instance, is likely personal data, so it should not be publicly available in this forum).

* in the experiment: tap on the three dots, select “save experiment state”; in the menu: tap on the three dots in the menu entry, select “share”

[Jens Noriʇzsɔɥ]

Ok, that's weird: I have (accidentally) recorded almost one hour of data, 3k+ rows. Excel on Mac complains about the file, however, correctly reads it. Neither Numbers nor Excel on iOS report any issues… ?

The .csv export should be a safe alternative. You should check which of the formats both, Google Spreadsheet and Excel, could deal with. In Germany, just semicolon and decimal comma work out-of-the-box in Excel. The dot is used as a decimal separator in Japan, isn't it. So, likely semicolon and decimal dot could be ok…

[physken_JPN]

Ok, that's weird: I have (accidentally) recorded almost one hour of data, 3k+ rows. Excel on Mac complains about the file, however, correctly reads it. Neither Numbers nor Excel on iOS report any issues… ?

The .csv export should be a safe alternative. You should check which of the formats both, Google Spreadsheet and Excel, could deal with. In Germany, just semicolon and decimal comma work out-of-the-box in Excel. The dot is used as a decimal separator in Japan, isn't it. So, likely semicolon and decimal dot could be ok…

Hello,

Thank you for your helpful information.

I will check if students have saved their experiment data and try to reproduce the issue. I've spoken to 30 students, and 3 faced problems on their iOS devices. However, some could save 30-minute data without issues.

I'm looking into it, thinking it might be an iOS version issue. For the .xls file problems, reopening with iOS Numbers seemed to help. But any data corruption stops us from editing on Google Drive.

Thanks for the .csv advice. Using .zip on iPads is a bit tricky, but I'll give it a try.

I'll keep you updated. Thanks for your support.

[Jens Noriʇzsɔɥ]

The problem with .zip files on iPads is local storage? I am mainly on iDevices and have not observed any issues with them…

(On a side note as it will most likely not help in your case: if you tap on graphs you could export the graph data via “More tools” and the resulting .csv file is not zipped.)