How can I trim a log file down to the ~20 seconds of really interesting flight? ArduPlane/ChibiOS on latest master…

Kelly

I’m not aware of any programs which trim log-files.

[In case anyone is interested, here are some related ideas:]
Basically every log message is tagged with a timestamp. If you only want folks reviewing the log to focus on a particular section, you could just tell them the start and ending timestamps, and to ignore everything else.

If you analyze logs in MATLAB, @Georacer and I developed a custom dataflash log tool. We implemented the ability to extract a “slice” from the log, but I would recommend you save it as a MATLAB-only filetype, rather than re-codify back to the original dataflash binary format. I’d be glad to share more detail. The code is here: https://github.com/georacer/ardupilog

[A reason that such a log-slice extraction may be more difficult than anticipated:]
There are FMT log messages at the beginning of the log which define how to correctly decode the binary data in the log. Even if you were to just cut-off the unnecessary parts of the dataflash file before-and-after your period of interest, you’d lose the FMT messages (as well as possibly other important “general data”) which makes anyone else viewing the log difficult/impossible.

So if you sliced a log, you’d want a program which decodes it, selects what to keep and what to toss, and then re-encodes the new log beginning with FMT messages. You’d probably want to keep a lot (all?) of the MSG messages too, even if they’re not in the window-of-interest.

Hope any of that helps somebody!

Thanks gents. I’ve thought about cutting out the majority of the file, leaving the formatting stuff at the front. I converted a .bin to text to explore that. This shouldn’t be that hard.

Kelly

This would definitely work, except for a couple of pitfalls:

If you cut the transmission of mission files in the middle, they would
obviously become corrupt and invalid

In large logs, the plain-text .log file can be several hundred megabytes.

But if it works for you… then that’s fine!

George,

I don’t fully understand your first point. I have a long file (30+min) and want only 20 seconds that I have determined are of interest. This is only for post-morten graphing. Are you saying that deleting a significant middle portion of the text file would make the file unusable to MP graphing?

Kelly

You got me wrong.
I was referring specifically to those MAVLink messages which are used to
transmit the Mission sequence (for use in AUTO mode).

Each Mission sequence is sent using multiple MAVLink messages and ALL of
these messages must be read (e.g. from MP) to decode and understand a
Mission sequence.

Regarding your use case, you will face 2 (not very serious) problems:

You won’t be able to see the Mission sequence waypoints if you don’t
include them in your log slice

You won’t be able to know the initial Mode, because a mode-change
MAVLink message will have occurred before the start of your slice.

Thanks George. Neither of those are concerns for my purposes here. Thank you for clarifying.

Kelly

Try this:

mavlogdump.py --condition "XKF1.TimeUS > 127000000 and XKF1.TimeUS < 147000000" source.bin --parms --quiet --meta --output destination.bin
It will preserve the parameters and be quiet
For more information do:
mavlogdump.py --help
EDIT: To be on the safe side you need a pymavlink version newer than Feb 2024. Previous versions had a bug.