Add flatbuffer support for COM configuration using flatbuffers by OliverHeilwagen · Pull Request #112 · eclipse-score/communication

OliverHeilwagen · 2025-12-04T08:08:57Z

resolves #110

This initial version was focused on working ipc bridge test using flatbuffers, quality aspects are not yet covered.
It is recommended to break down the issue into smaller tasks and setup a merge strategy, PR is therefore draft only.

Pending decisions:

Runtime switch between JSON and Flatbuffers, currently based upon file extention .json, .bin
Common approach for:
- Buffer version identification on target (file_identifier + common version info)
- Schema evolution handling (example)

Changes (rough outline)

Added generate_com_config rule (bazel/tools)
- Convert existing json to flatbuffer friendly version (key snake_case, enum friendly values)
- Creates buffer from converted json
Abstracted config_parser with config_loader
- Switch between JSON, FlatBuffer depending on file extension (see above)
Added FlatBufferConfigLoader
- Initial version tested via:
  - unit_test_runtime_single_exec
  - ipc bridge example

Open points:

Quality Artifacts not touched (Requirements, Architecture, ...)
Tracing configuration is still json only
File loading (open, mmap) should be moved to separate library (baselibs? I only saw fstream based abstraction)
Flatbuffer schema check can't check for value ranges
- generate schema from *.fbs via flatc using --jsonschema
- extend schema and add verification to generate_com_config rule
Add option to generate_com_config to update tracing config file from json to bin
Add version info to COM FlatBuffer format, to allow for version identification on target.

LittleHuba · 2025-12-09T09:09:34Z

Thank you for sharing this initial version of the flatbuffer support.
We will give this an initial review and get back to you with feedback as soon as we can.

LittleHuba · 2025-12-09T09:13:51Z

See also general discussion here: #110

OliverHeilwagen · 2025-12-09T09:49:10Z

+{
+    if (ShouldUseFlatBuffer(path))
+    {
+        // Warning is used to see usage in the default ipc bridge test setup


Change back to info before merging PR. Warning is currently only used to see effect in ipc bridge test.
Same applies to json log.

Signed-off-by: KaitoKid <thomas.mikhael@icloud.com>

OliverHeilwagen · 2026-01-14T16:48:25Z

    }

-    auto config = configuration::Parse(runtime_configuration.GetConfigurationPath().Native());
+    auto config = configuration::Load(runtime_configuration.GetConfigurationPath().Native());


@LittleHuba @castler @hoe-jo:
I guess we should first decide on the scope of this PR.
We could for example leave out this change to not yet allow for actual usage of flatbuffers by a user of COM and only introduce a initial set of changes e.g.:

mw/com/impl/configuration

ara_com_config.fbs

flatbuffer_config_loader.h/cpp

including the respective quality artifacts architecture, ..., tests

bazel/tools

Please coordinate with @Thomas-Mikhael he will takeover the PR.

OliverHeilwagen · 2026-01-20T10:32:33Z

+    """
+
+    # Always use the COM FlatBuffer schema
+    schema = "//score/mw/com/impl/configuration:ara_com_config.fbs"


@Thomas-Mikhael: Please check all references in *.bzl, which is used outside of this module, we have to specify the full bazel path e.g.:

Suggested change

schema = "//score/mw/com/impl/configuration:ara_com_config.fbs"

schema = "@score_communication//score/mw/com/impl/configuration:ara_com_config.fbs"

ipc_eample_rs didn't compile for QNX since there was no definition of max_align_t. Add one locally and use it. GIT_ORIGIN_SPP_REV_ID: 6317783d4a6a81edc49459e5c1c923763dea6824

Since there have been some changes to the SamplePtr for Lola, these also have to be reflected in the Rust FFI binding. GIT_ORIGIN_SPP_REV_ID: e7c8d5e74b34be1bdbbce3820a863bbb45b6867f

* Consumer FFI Binding with com-api GIT_ORIGIN_SPP_REV_ID: 648c4790c8e3a55527ef3468ed693200ee7c4f84

* Review comment fix GIT_ORIGIN_SPP_REV_ID: a81d4bc7706cbae53d59e00feb826b1438383a46

* Used SamplePtr instead of type pointer for Lolabinding type * ManageProxyBase added for proxy resource management * Removed the id from Sample type * Added send and sync trait for HandleContainer * Added Debug trait for HandleContainer * Removed ANY related specification from runtime GIT_ORIGIN_SPP_REV_ID: f1ba79ffdcf3c4d4c4a24f7939b9e7d2469ab4de

* Addressed review comment * Added Safety comment * Max_sample check added GIT_ORIGIN_SPP_REV_ID: 3bbc282ab27628cb64501130e8dbc3da9e48bfdd

* Added resource manager wrapper for proxy * Subscribe event error handing GIT_ORIGIN_SPP_REV_ID: aab6d4183b4cb39c2ebc3c24315f40f2b8e31314

* Type identifier added with Subscriber new api GIT_ORIGIN_SPP_REV_ID: bc10e5bafce467d7308229d36b5f5970f4bc5d81

* Fixed the clippy error of com repo GIT_ORIGIN_SPP_REV_ID: 891fbd39bd6c76438e426eb3beeabd4ff13c86e5

Replace shorthand 'stub' with explicit @score_baselibs target reference.

libc++ doesn't fully support fancy pointers (like offset pointers) in std::basic_string, which causes compilation errors when using PolymorphicOffsetPtrAllocator for shared memory.

GIT_ORIGIN_SPP_REV_ID: 601313c8029694c7da86a3fe5944af4663ab4e7f

If we want to make sure that the rules C++ and Python loaded are the version we defined and not the one in Bazel, we need to explicitly add the load statement. This also becomes mandatory in Bazel 9. These changes have been done automatically with buildifier fix after adding native-cc and native-py. GIT_ORIGIN_SPP_REV_ID: 6796832a7640ee19010d3d78a4d6b7ac57b6c794

issue: SWP-244458 GIT_ORIGIN_SPP_REV_ID: c37dc8e5bee23c354ca6b6fd3076bc4037a805cf

GIT_ORIGIN_SPP_REV_ID: 8706c44fabadd2689641486c73be0bb90b1333c4

GIT_ORIGIN_SPP_REV_ID: 4e55c93ae5f8ca5bd30472a0face20dc1e7b50ff

GIT_ORIGIN_SPP_REV_ID: d213476bef523e854a54d4aaf1929de2cb84a7fb

These functions are currently private since they will be called only by guard objects that will be returned when registering the handlers. GIT_ORIGIN_SPP_REV_ID: ad4b46c206ef788d0e6cbabd2a1b45688e548d33

Skeleton::PrepareStopOffer now cleans up any registered OnServiceMethodSubscribedHandlers and MethodCallHandlers (stored in the child SkeletonMethods). This is achieved by returning RAII guard objects from the registration calls which will call unregister on destruction. This pattern avoids that we accidentally forget to unregister handlers and also avoids that we erroneously try to unregister handlers which were never registered. GIT_ORIGIN_SPP_REV_ID: 3c83781b949b28db77eb428a38a952009daa7144

When a skeleton is processing a Proxy subcription, we should unsubscribe any methods that were successfully subscribed during the subscription process if one of the methods fails to subscribe. GIT_ORIGIN_SPP_REV_ID: cec2c157caaf703acffb5b023e99df938f21c4b6

GIT_ORIGIN_SPP_REV_ID: 51f50001dc3b6339498af57330ab823978006bff

GIT_ORIGIN_SPP_REV_ID: 3bcc9a1d882cf234b67c6382b95536472803d55a

The test target sometimes times out in the CI when building with spp_memcheck. This issue will be investigated in SWP-141169 and the test will be marked again as short timeout. GIT_ORIGIN_SPP_REV_ID: 97bd4704ca1fc2d297117971a640787e3f817581

With this changset, it is possible to run CodeQL in a sandboxed manner, for our codebase. e.g. `bazel run //quality/static_analysis:codeql_lint -- --target=//score/mw/com/impl/configuration:quality_type` This approach is not _cached_! We first implemented a solution via aspects, but it turns out, that this is performing at least 3x worse then this solution. We now, setup the environment variables as expected for CodeQL. To follow this, you can read in the CodeQL documentation. This command produces in `bazel-out` two files: * codeql.sarif * codeql.csv These files are update for the last execute run and then can be used by your favorite IDE to display the results. In order to supress warnings, one needs to define in `quality/static_analysis/coding-standards.yaml` first deviation argumentations, and assign an `code-identifier` that is then used in the respective line. For more information about that please read: https://github.com/github/codeql-coding-standards/blob/main/docs/user_manual.md#applying-deviations At the current point, we are aware that the MISRA C++:2023 support is under-development and not safety certified. But the achievements by the team behind and milestone show, that this seems to be the best alternative to perform Misra static analysis in the open. Besides this solution, we have looked into: * SonarQube * Coverity Both require a paid plan, to execute Misra checks, which is why we did not put them into the solution space for now.

If we want to make sure that the rules C++ and Python loaded are the version we defined and not the one in Bazel, we need to explicitly add the load statement. This also becomes mandatory in Bazel 9. These changes have been done automatically with buildifier fix after adding native-cc and native-py. GIT_ORIGIN_SPP_REV_ID: c820f1811d8c6d42332706badb1e1a8a6f9b2dbc

Two small updates: * Lifetime of shared resources for client connections is changed to what is implemented; * Clarification how it is possible to postpone/serialize open() handlers in servers is provided. GIT_ORIGIN_SPP_REV_ID: 736649eb0425dcda656d5b5b61aa040ef8c435ff

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: 98e9ecd77628a83864914472dbd1cab77228a4d0

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: 5a93c06053affd9d9fd38be26a96f917666a13d8

GIT_ORIGIN_SPP_REV_ID: 59be6be63cb3dae039dcf934a78bd5f126814921

GIT_ORIGIN_SPP_REV_ID: 1a3369ba2f400285da83efe2959de75923095e61

Calling the interface with anything other than a `std::string&&` is deprecated, and leads to a warning. As on S-CORE builds warnings are treated as errors, this prevents building other components, such as ConfigManagement. This fix uses the non-deprecated overload, allowing it to move a copy of the value retrieved from the JSON. GIT_ORIGIN_SPP_REV_ID: f36fc7b6ec542e74d2d35c907b7b021589d45b3f

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: c3f6b797eb2e1041312633cdab382852d9a46c76

Using OffsetPointerSpan allows us to store the size information of the call queue together with its starting position. This helps us avoid misconfigured call queues. Add tests for the correct size calc for InArgs and Return value queues issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: 968806ae8d546978554f15481d0ac45ab721b1e9

crimson11 previously approved these changes Dec 9, 2025

View reviewed changes

OliverHeilwagen commented Dec 9, 2025

View reviewed changes

Comment thread score/mw/com/impl/configuration/ara_com_config.fbs

OliverHeilwagen commented Dec 9, 2025

View reviewed changes

Comment thread bazel/tools/generate_com_config.bzl Outdated

OliverHeilwagen added 2 commits December 9, 2025 13:47

COM configuration using flatbuffers - initial version.

18805d6

Add lola method configuration to flatbuffer config

75dcfcc

OliverHeilwagen force-pushed the flatbuffer-configuration branch from 46dea4b to 75dcfcc Compare December 9, 2025 13:54

Thomas-Mikhael and others added 2 commits January 14, 2026 16:36

Flatbuffers PR, fine tunings & cleanup

524c211

Merge branch 'main' into flatbuffer-configuration

6464d38

Signed-off-by: KaitoKid <thomas.mikhael@icloud.com>

OliverHeilwagen marked this pull request as ready for review January 14, 2026 16:30

OliverHeilwagen requested review from LittleHuba, castler and hoe-jo as code owners January 14, 2026 16:30

OliverHeilwagen commented Jan 14, 2026

View reviewed changes

OliverHeilwagen commented Jan 20, 2026

View reviewed changes

OliverHeilwagen dismissed crimson11’s stale review via 6464d38 March 16, 2026 10:09

anmittag added this to COM - Communication FT Mar 16, 2026

github-project-automation Bot moved this to Backlog in COM - Communication FT Mar 16, 2026

saksham-chawla mentioned this pull request Mar 18, 2026

Add flatbuffer support for COM configuration using flatbuffers propel-gtm/communication#20

Closed

darkwisebear and others added 9 commits April 13, 2026 13:48

Introduce max_align_t for QNX

92d2390

ipc_eample_rs didn't compile for QNX since there was no definition of max_align_t. Add one locally and use it. GIT_ORIGIN_SPP_REV_ID: 6317783d4a6a81edc49459e5c1c923763dea6824

Update the Rust binding for SamplePtr

1ef6b87

Since there have been some changes to the SamplePtr for Lola, these also have to be reflected in the Rust FFI binding. GIT_ORIGIN_SPP_REV_ID: e7c8d5e74b34be1bdbbce3820a863bbb45b6867f

Rust::com Consumer APIs binding with FFI

cfaa12a

* Consumer FFI Binding with com-api GIT_ORIGIN_SPP_REV_ID: 648c4790c8e3a55527ef3468ed693200ee7c4f84

Rust::com Consumer API Implementation

2eb3b50

* Review comment fix GIT_ORIGIN_SPP_REV_ID: a81d4bc7706cbae53d59e00feb826b1438383a46

Rust::com Consumer implementation for com-api

98dfec9

* Addressed review comment * Added Safety comment * Max_sample check added GIT_ORIGIN_SPP_REV_ID: 3bbc282ab27628cb64501130e8dbc3da9e48bfdd

Rust::com Cosumer API implementation for com-api

df61111

* Added resource manager wrapper for proxy * Subscribe event error handing GIT_ORIGIN_SPP_REV_ID: aab6d4183b4cb39c2ebc3c24315f40f2b8e31314

Rust::com Consumer APIs implementation for com-api

6b4d886

* Type identifier added with Subscriber new api GIT_ORIGIN_SPP_REV_ID: bc10e5bafce467d7308229d36b5f5970f4bc5d81

Rust::com Clippy warning fix

8ed6abd

* Fixed the clippy error of com repo GIT_ORIGIN_SPP_REV_ID: 891fbd39bd6c76438e426eb3beeabd4ff13c86e5

Rahul-Sutariya and others added 28 commits April 13, 2026 13:49

update tracing_library flag

940a980

Replace shorthand 'stub' with explicit @score_baselibs target reference.

fix: Use libstdc++ instead of libc++ for LLVM toolchain

517498f

libc++ doesn't fully support fancy pointers (like offset pointers) in std::basic_string, which causes compilation errors when using PolymorphicOffsetPtrAllocator for shared memory.

Update test priority metadata for requirement linking.

493e706

GIT_ORIGIN_SPP_REV_ID: 601313c8029694c7da86a3fe5944af4663ab4e7f

mw/com: Microbenchmark for GetNewSamples

065de5e

issue: SWP-244458 GIT_ORIGIN_SPP_REV_ID: c37dc8e5bee23c354ca6b6fd3076bc4037a805cf

lola: Add tests for SkeletonMethod::IsRegistered

322cfd2

GIT_ORIGIN_SPP_REV_ID: 8706c44fabadd2689641486c73be0bb90b1333c4

lola: Add log stream overloads for identifier classes

d472822

GIT_ORIGIN_SPP_REV_ID: 4e55c93ae5f8ca5bd30472a0face20dc1e7b50ff

lola: Add missing source file

75e4b7c

GIT_ORIGIN_SPP_REV_ID: d213476bef523e854a54d4aaf1929de2cb84a7fb

lola: Add functions to unregister methods-related handlers

6c81483

These functions are currently private since they will be called only by guard objects that will be returned when registering the handlers. GIT_ORIGIN_SPP_REV_ID: ad4b46c206ef788d0e6cbabd2a1b45688e548d33

lola: Update comments in MessagePassingServiceInstance

208180e

GIT_ORIGIN_SPP_REV_ID: 51f50001dc3b6339498af57330ab823978006bff

lola: Add tests for operator<<

20b56f3

GIT_ORIGIN_SPP_REV_ID: 3bcc9a1d882cf234b67c6382b95536472803d55a

fetch the latest baselibs

b0b976e

git: Ignore further Clion directories

7b65572

mw/com: Fix clang-tidy bugprone-use-after-move finding

335bc93

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: 98e9ecd77628a83864914472dbd1cab77228a4d0

mw/com: Suppress clang-tidy cppcoreguidelines-pro-type-reinterpret-cast

87b38c1

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: 5a93c06053affd9d9fd38be26a96f917666a13d8

lib/message_passing: block SIGTERM in internal thread

6852bff

GIT_ORIGIN_SPP_REV_ID: 59be6be63cb3dae039dcf934a78bd5f126814921

lib/message_passing: improve comments for SIGTERM blocking

48de89a

GIT_ORIGIN_SPP_REV_ID: 1a3369ba2f400285da83efe2959de75923095e61

mw/com: Suppress clang-tidy bmw-banned-function for legitimate use

4c206f1

issue: SWP-239907 GIT_ORIGIN_SPP_REV_ID: c3f6b797eb2e1041312633cdab382852d9a46c76

COM configuration using flatbuffers - initial version.

8596793

Flatbuffers PR, fine tunings & cleanup

708b83d

Thomas-Mikhael requested review from bemerybmw and limdor as code owners April 13, 2026 12:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flatbuffer support for COM configuration using flatbuffers#112

Add flatbuffer support for COM configuration using flatbuffers#112
OliverHeilwagen wants to merge 243 commits intoeclipse-score:mainfrom
etas-contrib:flatbuffer-configuration

OliverHeilwagen commented Dec 4, 2025 •

edited

Loading

Uh oh!

LittleHuba commented Dec 9, 2025

Uh oh!

LittleHuba commented Dec 9, 2025

Uh oh!

Uh oh!

OliverHeilwagen Dec 9, 2025

Uh oh!

Thomas-Mikhael Jan 19, 2026

Uh oh!

Uh oh!

OliverHeilwagen Jan 14, 2026

Uh oh!

OliverHeilwagen Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

	schema = "//score/mw/com/impl/configuration:ara_com_config.fbs"
	schema = "@score_communication//score/mw/com/impl/configuration:ara_com_config.fbs"

Conversation

OliverHeilwagen commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LittleHuba commented Dec 9, 2025

Uh oh!

LittleHuba commented Dec 9, 2025

Uh oh!

Uh oh!

OliverHeilwagen Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Thomas-Mikhael Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

OliverHeilwagen Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

OliverHeilwagen Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

OliverHeilwagen commented Dec 4, 2025 •

edited

Loading