feat: add support of comma-separated param files with comments and empty lines by flyasky · Pull Request #402 · AvyFly/parasect

flyasky · 2026-02-06T04:35:49Z

Added support for Mission Planner format with comments and empty lines.
ulog parser is disabled because it is a subset of MissionPlanner format, and we have SyntaxError: Protocol ambiguous, fits more than one parser. exception.

flyasky · 2026-03-31T17:28:46Z

@Georacer could you please review this MR?

Georacer

Thank you very much for the PR and sorry for the delay.
We'll get this in eventually, one way or another.

But I have a first round of questions here. Perhaps a few more later, when I look at this again.
Could you provide some answers?

Georacer · 2026-03-31T21:15:01Z

+ARDUPILOT_DEFAULT_PARM = Path(ARDUPILOT_ASSETS_PATH) / "sitl_copter_defaults.parm"
+ARDUPILOT_DEFAULT_PARAM = Path(ARDUPILOT_ASSETS_PATH) / "sitl_plane_defaults.param"


Is there any actual reason to add both a copter and a plane parameter file? Do you use both in the autotests?

Also these two names are too similar, they need more appropriate names.

Plane and copter defaults are different. As I generally work on planes, so I only had plane defaults to commit here=).

In the tests we need to cover both .parm (tab separator) and .param (comma separator) file types. Maybe let me replace it with the file for copter. I guess if I replace tabs to commas it will work out

Keep in mind that Parasect is meant to be agnostic of the actual parameters meaning, so copter and plane defaults won't and should not make any difference to how they are parsed.
Unless there are syntactic differences, which to my understanding are not, for the moment.

Georacer · 2026-03-31T21:17:43Z

    def test_qgc_rare_types(self):
        """Ensure that value types 2,4,6,9 are correctly read."""
-        parameter_list = _helpers.read_params(utils.ARDUPILOT_ODD_PARAM_VALUES_FILE)
+        parameter_list = _helpers.read_params(utils.ARDUPILOT_ODD_PARAMS_VALUES_FILE)


Is there really a need to rename this constant?

I think yes, as it has file type in the name related to .params extension. This is a different type produced by QGroundControl.

Ah, now I understand. I did not mean for this variable name to match the extension of the filetype.

I'll consider if this is important in the context of these util attribute names.

Georacer · 2026-03-31T21:22:08Z

+        return f"{meal.name}.parm"
+    elif format == Formats.mp:


You're re-routing the .param format to the Mission Planner one, and throwing .parm to .apm and .apj.
Can you explain this change a bit?

Both apj and mavproxy(apm) use the format key<tab>value (PARM)
This format does not allow comments and empty lines. Kept this part as is.

Mission planner (MP) exports parameters in the format key<comma>value (PARAM).
Also MP is able to read format key<comma>value with any number of empty lines, comments at the end of line, and full-line comments.

Basically I replaced/extended ulog type to more flexible MP format. And made it possible to open for comparison.

https://ardupilot.org/dev/docs/apjtools-intro.html
maybe apj format should be updated separately, as it supports 4 elements in the row (including @readonly flag).

I'll try to disentangle the definitions of the different extensions, but keep in mind that, AFAIK, there isn't a hard rule for a 1:1 match between extension name and file format.

For example, the ArduPilot apj tool can probably handle .parm and .param files. I haven't tried it and I probably should.

Georacer · 2026-03-31T21:33:21Z

+    if len(item) < 3:
+        reasoning = None
+    else:
+        reasoning = item[2]


The interface of the function says that the item list should have 3 items. Is there a good reason to assume it doesn't? Perhaps the caller that needs this change should be the one to change?

I see, removed. As I have

if len(value_reasoning) <= 1: params.append("") else: params.append(value_reasoning[1].strip())

in the split_missionplanner_row() it should provide 3 items

…pty lines

flyasky · 2026-04-12T11:03:25Z

also fixed some tests

flyasky · 2026-04-30T19:26:20Z

@Georacer could you please have another look =)

Georacer

You've moved around quite a bit of code and features. Some are not strictly relevant to the spirit of the PR, which is a problem because it makes the review more expansive and it takes longer.

If I understand you correctly, as you mention in the title, what you really want is to add support for comparing and building MP parameter files with comments at the end, comment lines and empty lines.
That's clear and reasonable.

Since you mention that MP is a superset of ULOG, what I'd like to do first is to make a table of all the various parameter formats we have and their related software. Then see where this MP format fits.

I may or may not start from scratch, but for sure I can't merge your PR in its current form. It's a bit too pervasive for my taste. Don't get me wrong, I appreciate your effort immensely and one way or another I'll credit your work.

Thank you!

Georacer · 2026-05-01T20:43:44Z

+        return f"{meal.name}.parm"
+    elif format == Formats.mp:


I'll try to disentangle the definitions of the different extensions, but keep in mind that, AFAIK, there isn't a hard rule for a 1:1 match between extension name and file format.

For example, the ArduPilot apj tool can probably handle .parm and .param files. I haven't tried it and I probably should.

Georacer · 2026-05-01T20:56:19Z

+    try:
+        float(params[1])
+    except ValueError as e:
+        raise SyntaxError("MP: First row element must be a parameter name string.") from e


This text is wrong. It should refer to the second element, the parameter value.

Georacer · 2026-05-01T21:01:42Z


    try:
-        with open(filepath) as f:
+        with open(filepath, encoding='utf-8') as f:


I'm curious why this was necessary.

for supporting Cyrillic characters in the comments -)

Georacer · 2026-05-05T20:18:54Z

@flyasky I started looking into this.

Here are my findings so far:

Output formats:

GCS	Separator	Extension
Mission Planner	Comma	param
QGC	QGC-style	params
MAVProxy	Tab	parm
mavparams.py	Tab/QGC	N/A

Input format compatibility:

Format	Mission Planner	QGC	MAVProxy
Comma	Y	N	Y
Comma with empty lines	Y	N	Y
Comma with inline #	Y	N	N (ignores lines with comments)
Comma with full line #	Y	N	Y
Tab	Y	N	Y
Tab with empty lines	Y	N	Y
Tab with inline #	Y	N	N
Tab with fullline #	Y	N	Y
QGC	N	Y	N
QGC with empty lines	N	Y	N
QGC with inline #	N	N Buggy, ignores first change	N
QGC with fullline #	N		N

Next I'll try all of these in Parasect, and then try to add support to whatever's missing.

Maybe there's room for a parameter converter feature here!

flyasky marked this pull request as draft February 6, 2026 12:09

flyasky force-pushed the feature/ardupilot-file-format-with-comments branch from 48bef53 to e1e59f3 Compare February 8, 2026 17:50

flyasky marked this pull request as ready for review February 8, 2026 17:51

Georacer reviewed Mar 31, 2026

View reviewed changes

flyasky force-pushed the feature/ardupilot-file-format-with-comments branch from e1e59f3 to d7e65ed Compare April 12, 2026 10:21

feat: add support of comma-separated param files with comments and em…

6a255d7

…pty lines

flyasky force-pushed the feature/ardupilot-file-format-with-comments branch from d7e65ed to 6a255d7 Compare April 12, 2026 11:02

Georacer requested changes May 1, 2026

View reviewed changes

flyasky marked this pull request as draft May 2, 2026 19:48

		ARDUPILOT_DEFAULT_PARM = Path(ARDUPILOT_ASSETS_PATH) / "sitl_copter_defaults.parm"
		ARDUPILOT_DEFAULT_PARAM = Path(ARDUPILOT_ASSETS_PATH) / "sitl_plane_defaults.param"

Conversation

flyasky commented Feb 6, 2026

Uh oh!

flyasky commented Mar 31, 2026

Uh oh!

Georacer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyasky Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyasky Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyasky commented Apr 12, 2026

Uh oh!

flyasky commented Apr 30, 2026

Uh oh!

Georacer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Georacer commented May 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

flyasky Apr 12, 2026 •

edited

Loading

flyasky Apr 12, 2026 •

edited

Loading