Skip to content

P183 modbus RTU#5390

Open
flashmark wants to merge 46 commits intoletscontrolit:megafrom
flashmark:P183_Modbus_registers
Open

P183 modbus RTU#5390
flashmark wants to merge 46 commits intoletscontrolit:megafrom
flashmark:P183_Modbus_registers

Conversation

@flashmark
Copy link
Copy Markdown
Contributor

Initial version of a simple Modbus RTU plugin. This plugin handles a Modbus device as a set of holding registers. Up to 4 holding registers can be read into the 4 values of the plugin. The plugin number is 183 as agreed elsewhere.

Comment thread src/_P183_modbus.ino
@TD-er
Copy link
Copy Markdown
Member

TD-er commented Aug 31, 2025

Can you look at the timing stats for this plugin you added?
I suspect the reading like this will take 100's of msec per register read.

Please take a look at what I did for the Eastron plugin, where I set a queue of registers to read.
This way the waiting time per call is nihil and thus not blocking the rest of the ESPeasy functionality.

My intention is to make all ESPEasy plugins using modbus use this approach to share access to the same Modbus bus.

Also the ESPEasySerial for (nearly) all plugins is using the same set of PCONFIG() for the serial config.
I did not check if you did this, so please make sure to use the same way as in the most recent plugins using a serial port.

@flashmark
Copy link
Copy Markdown
Contributor Author

I have been busy for some time. Try to pickup the comments soon. Indeed the exchange takes some time and I will look into creating a queue. Eastron was my inspiration, but I did not look in most of the details as it was mainly device specific handling. Serial settings were copied from Eastron. To share the modbus I need some more insights how the sharing is intended. I have too little experience with some details in ESPEasy.

@flashmark
Copy link
Copy Markdown
Contributor Author

A small introduction of myself. I started as embedded software developer using mainly C, but I am for quite some time software architect and not doing professional coding anymore. I am definitely not a C++ expert. I am working for a large company building complex machines. I like to pickup smaller embedded software projects and domotica. And I really like the way ESPEasy is set up.

I see some issues with the current P078 implementation. The modbus and Eastron device specific features are mixed over the various "layers". There is the "plugin" that takes care of the configuration and external data (variable, config, web representation). Then the "data_struct" to put the behavior of the plugin in a C++ friendly environment (away from .ino). And there is a "Eastron library" in SDM.cpp.

If understood you well the requirements are:

  • Multiple plugin instances of different plugin types can share the same physical Modbus. (Use the same link)
  • Multiple links should be possible in parallel
  • Modbus access shall not hold the CPU during message exchange

The P078 implementation uses a queue that can manage multiple plugins in theory. For me it is unclear how it can differentiate between multiple links. The plugin defines the serial link, if there are multiple plugins connected it seems the last plugin that initializes defines the link properties. Is this desired behavior?

The queue knows it is handling SDM messages and delivers the received holding register directly into the uservar: UserVar.setFloat(it->taskIndex, it->taskVarIndex, value);
It is a fast way to handle the data, but it makes it impossible to retrieve other data from the Modbus device.

My proposal would be to split the code into the following classes:

  • Plugin & optional data_struct to handle the user interaction as standard in ESPEasy
  • A Modbus class that handles one link. This includes a queue and Modbus packet coding/decoding.
  • A singleton Modbus "broker" that manages a list of Modbus links. The broker handles init/terminate requests from the plugins and tries to combine requests from multiple plugins in a single Modbus link.
  • Existing ESPEasy serial link class responsible for the data transport over the selected serial link

Broker and link should be outside any plugin as they can be shared by multiple plugin classes. As they are both Modbus specific they can be in a single file sharing a header file.

I still need to think about the details how to exchange data and fit the queue neatly in the design. What has to be in and how does it return results. The Modbus link should be able to handle both the RTU binary and ASCII format. It should be able to handle other Modbus message types. Both option for future only :-)

One thing to consider:
Do we want to have the serial link specified through the plugin or add Modbus links as dedicated resources to the hardware page? My preference would be to keep it as is. Will make the broker a bit more complex, but we can manage that.

@TD-er
Copy link
Copy Markdown
Member

TD-er commented Sep 6, 2025

Well hats off to you sir, as you seem to have a very good idea of the layers we have right now :)

Right now, I am working on another pull-request to do a complete rewrite for ESPEasy's WiFi/network implementation.
This does add a "Network" tab, just like the current "Devices" and "Controllers" tab we currently have.
And when I say "just like", it really is very similar.
So the first table is showing all network adapters and a short overview of its current state/config.

When clicking "Edit" on such an adapter, you get the specific settings, very similar to how controllers and tasks are being dealt with.

My next idea for a follow-up on this is to add a tab for "Interfaces" (or buses, name is not yet clear).
This way you can define interfaces like Modbus, SPI, I2C, 1-wire, etc. (maybe also extended GPIO pins)

Especially some of those interfaces which share a bus for multiple devices, need a handler to deal with all devices and pending transactions on the bus.
And also as you rightfully mentioned, some main (singleton) handler to handle all interfaces of the same type.
For example, when requesting a read or write on a modbus device, you must make sure you have completed this before something else is requested.

This idea is already implemented (in a bit too specific way) by keeping track of a list of registers to read and where to put the result.
This does work fine for Eastron devices, as it is all the same. You call for a register and interpret the result, then store the value somewhere.
However this already has some limitation as it only assumes float values. There are however some registers which do not return a float.

So to "fix" this, I imagine there might be some new PLUGIN_MODBUS_READ_RESULT call, which then should be implemented in those plugins which support Modbus communications.
Then in the event (ESPEasy EventStruct), there should be the taskindex set and probably some other values too and a pointer to the read data.
Those plugins then should request a read to the Modbus handler from the PLUGIN_READ call.
This way the read is already way more generic.

Then adding a main handler would probably be something for a next pull-request so we can think of a more generic way to manage modbus handlers.

The main advantage with this is that there is no longer a collision when accessing modbus devices on the same bus and there is no active blocking wait for a reply from the device, which may take quite some time.
For example the SenseAir S8 may take 200+ msec to reply and currently does actively wait.
The same for the ACU28x (or how those power meters are called...) and probably also for those PZEM meters.

@flashmark
Copy link
Copy Markdown
Contributor Author

Looks good. By the way, I am not in any hurry to push my branch. Please continue the framework and let me know where I can contribute for something modbus specific. A suggesting is to remove the word MODBUS in the event and keep is a bit more abstract like BUS_READ_RESULT. This can then be any pending bus transaction. I think that I2C could also benefit.

If we add this callback trough an event and a central bus or link administration the singleton management layer will be very simple. The plugin know the bus type and index and the manager returns the associated bus object. Maybe do some admin to check how many active plugins are coupled to the bus; and do initialization termination when the first joins or the last leaves.

By the way, I saw a Modbus_RTU file. Is this where the new bus management stuff should grow?

@TD-er
Copy link
Copy Markdown
Member

TD-er commented Sep 6, 2025

Not likely that this will remain the (file) structure.

For the WiFi/network rewrite, I'm introducing namespaces like ESPEasy/net/wifi

The idea of having a generic bus callback/event also seems OK, as long as the bus manager/handler does keep in mind which task may be expecting specific bus responses.
Like there are certain devices which can be used via various different bus types. For example the same sensor on I2C or SPI and/or UART.
But that's something to worry about later.

Comment thread src/_P183_modbus.ino Outdated
Comment thread src/src/Helpers/Modbus_mgr.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.h Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
@flashmark
Copy link
Copy Markdown
Contributor Author

Sorry it is still work in progress I wanted to set aside. I had some other duties and am just picking up this project. Will update it soon with a more crisp design. Keep in mind: I am not a C++ coder, any corrections and suggestions are welcomed.
Main design separation into several classes:

  • Plugin: Unique for a device and implementing the interaction with ESPEasy.
  • Modbus_device: Represents one modbus device and is coupled to a plugin. Does the modbus message (de)coding.
  • Modbus_link: Represents a physical modbus link (serial link). One link can serve multiple modbus_devices.
  • Modbus_mgr: Connects the devices and the links. This singleton object knows the devices and the links. Main task is to create a link when the first device wants to connect and connect a device to a link on request.
  • Queue: Each link has a queue of pending modbus transactions.

Main issues to resolve:

  • Serial port configuration is connected to a modbus_link. But I can only configure a plugin. Workaround: last device connecting to a link determines the properties. (If the ESPEasySerialPort type is different then it is a new link, still to think how to handle multiple "software serial")
  • Transaction takes too much time to wait/poll. Instead we use a queue to prepare transactions (one message exchange). As a result we need some callback mechanism to return the results.
  • For the device to link relation the is a real callback is implemented as a class function and the link knows the modbus_device that queued the transaction.
  • For the plugin to device relation I don't know how to properly implement a mechanism that can feed back the uint16 registers from the modbus reply. Converting them to plugin output values blurs the responsibilities of the plugin and the modbus_device. This will make reuse much more difficult (like the P078 plugin).
  • To keep the link busy it needs to poll the serial link for a complete reply. For now this is done by a ten_per_second trigger from each plugin. This is a bit heavy on CPU load. I am looking for a direct triggering of the link objects. The code should be prepared for that.

Is there a way to store design documentation with a plugin?

Comment thread src/_P183_modbus.ino Outdated
addLogMove(LOG_LEVEL_INFO, "P183 INIT AFTER INIT");
P183_ModbusDevice->setModbusTimeout(P183_MODBUS_TIMEOUT);

addLogMove(LOG_LEVEL_DEBUG, "AFTER TIMEOUT");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check to see if we have debug code (including this LOG_LEVEL_DEBUG define) included in the build.

#ifndef BUILD_NO_DEBUG
addLogMove(LOG_LEVEL_DEBUG, F("AFTER TIMEOUT"));
#endif

And also please use the F() macro as I showed here when handling strings.

Comment thread src/_P183_modbus.ino Outdated
Comment thread src/src/Helpers/Modbus_device.cpp Outdated
if (_modbus_link != nullptr) {
_modbus_link->freeTransactions(this);
ModbusMGR_singleton.disconnect(_deviceID);
_modbus_link = nullptr;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use some (weak) std::shared_ptr like structure for this?

Comment thread src/src/Helpers/Modbus_device.cpp Outdated
String log = F("---> ModbusDevice Init: Slave address = ");
log += slaveAddress;
log += F(", This = ");
log += (ulong)this;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this compile? And what is an ulong?
In C++ there is a special integer type which is by definition large enough to hold a pointer. This is the size_t
On our ESP boards, this happens to be 32-bit in size, so essentially it is the same as an uint32_t, which happens to be the same as unsigned int.
Though it is good practice to use size_t for these kind of things.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot about size_t :-(.

Comment thread src/src/Helpers/Modbus_device.cpp Outdated
} else {
// Invalid CRC
_state = ModbusQueueState_t::ERROR;
String log = F("---> Device callback: ");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No check to see if req is a valid pointer.

Comment thread src/src/Helpers/Modbus_device.cpp Outdated

log += req->_id;
log += F(", Message = ");
log += (uint8_t)req->_messageType;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please only use those C-style casts only when needed. (e.g. when literally casting a pointer to an int to get the address is a good example)
In C++ we should use static_cast<uint8_t>(....)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing. Please note this is a preliminary version. Don't integrate it in mega yet! I will go over the comments and fix them. But expect also more fundamental changes as it does not work as I want it to work. Debug stuff will be cleaned up before final pull request.

Comment thread src/src/DataTypes/SettingsType.cpp
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_mgr.h Outdated
Comment thread src/src/Helpers/Modbus_mgr.h Outdated
Comment thread src/_P183_modbus.ino Outdated

case PLUGIN_WEBFORM_LOAD:
{
addFormNumericBox(F("Modbus Link"), P183_LINK_ID_LABEL, P183_LINK_ID, 0, 3);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That magic number 3 should be based on the #define set elsewhere

# endif // ifndef DAT_OFFSET_CUSTOM_CONTROLLER
# endif
# ifndef DAT_MODBUS_INTERFACE_OFFSET
# define DAT_MODBUS_INTERFACE_OFFSET 6144 // each Modbus link config = 256 bytes, 4 max
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is different for ESP8266 and ESP32.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should enable the Modbus link on ESP8266 at all due to the strict size limitations. But that is a different discussion. Where should this land for the ESP8266? Where can we find the layout in a human readable/understandable format? I already fort how I came to this magic number

Copy link
Copy Markdown
Member

@TD-er TD-er Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'basic settings' are stored from the start, so you can base it on the DAT_BASIC_SETTINGS_SIZE

However for ESP8266 we do have some FEATURE_NON_STANDARD_24_TASKS (not for official builds, only for some custom builds), so I would suggest to take that into consideration for the defines.

Something like this:

  # ifndef DAT_MODBUS_INTERFACE_OFFSET
  #  ifdef FEATURE_NON_STANDARD_24_TASKS
  #    if defined(FEATURE_MODBUS_FAC) && FEATURE_MODBUS_FAC
  #     error  "Not yet defined where to store modbus data, see https://github.com/letscontrolit/ESPEasy/pull/5390#pullrequestreview-4124760094 "
  #    endif
  #  else
  #   define DAT_MODBUS_INTERFACE_OFFSET     DAT_BASIC_SETTINGS_SIZE  // Stored in the 1k right after the basic settings, each Modbus link config = 256 bytes, 4 max
  #  endif
  # endif

Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
Comment thread src/src/Helpers/Modbus_link.h Outdated
Comment thread src/src/Helpers/Modbus_link.cpp Outdated
int16_t ModbusLINK_struct::getSerialRX(void) const
{
return _easySerial != nullptr ? _easySerial->getRxPin() : -1;
return _easySerial != nullptr ? _easySerial->getRxPin() : -1;
Copy link
Copy Markdown
Contributor

@tonhuisman tonhuisman Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should isInitialized() be used here instead? Same for the other return values

Comment thread src/src/Helpers/Modbus_mgr.cpp Outdated
@@ -152,7 +145,7 @@ uint16_t P183_data_struct::readRegisterWait(uint16_t address) {

while (state == ModbusResultState::Busy) {
delay(50);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, delays should be avoided like the plague, processCommands() should be called in 50/sec or 10/sec events instead

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I am working on it. This is due to the fact that the bus takes time for a transaction. This was the reason to go for a separate link object that is triggered outside the device code. But for some functions the device shall return the result of the Modbus transaction.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but then the internal state of the object should be like "X_pending".
Some Modbus devices (e.g. SenseAir S8 and older devices) easily take upto 200 msec or more to give a reply.
This is way too long to literally wait and this is also why a sommand should be put into a queue and get notified when there is a reply.
Either via some PLUGIN_xyz function call or with some function pointer kept along with the pending request.

On a shared bus you simply can't expect to be immediately serviced as there may be pending calls already sent. At best you could set some priority like "this has to be the first in line"

@flashmark
Copy link
Copy Markdown
Contributor Author

Right now, I am working on another pull-request to do a complete rewrite for ESPEasy's WiFi/network implementation. This does add a "Network" tab, just like the current "Devices" and "Controllers" tab we currently have. And when I say "just like", it really is very similar. So the first table is showing all network adapters and a short overview of its current state/config.

Especially some of those interfaces which share a bus for multiple devices, need a handler to deal with all devices and pending transactions on the bus. And also as you rightfully mentioned, some main (singleton) handler to handle all interfaces of the same type. For example, when requesting a read or write on a modbus device, you must make sure you have completed this before something else is requested.

This idea is already implemented (in a bit too specific way) by keeping track of a list of registers to read and where to put the result. This does work fine for Eastron devices, as it is all the same. You call for a register and interpret the result, then store the value somewhere. However this already has some limitation as it only assumes float values. There are however some registers which do not return a float.

So to "fix" this, I imagine there might be some new PLUGIN_MODBUS_READ_RESULT call, which then should be implemented in those plugins which support Modbus communications. Then in the event (ESPEasy EventStruct), there should be the taskindex set and probably some other values too and a pointer to the read data. Those plugins then should request a read to the Modbus handler from the PLUGIN_READ call. This way the read is already way more generic.

Then adding a main handler would probably be something for a next pull-request so we can think of a more generic way to manage modbus handlers.

The main advantage with this is that there is no longer a collision when accessing modbus devices on the same bus and there is no active blocking wait for a reply from the device, which may take quite some time. For example the SenseAir S8 may take 200+ msec to reply and currently does actively wait. The same for the ACU28x (or how those power meters are called...) and probably also for those PZEM meters.

I got the Modbus engine running. Plugins can use de Modbus_device to queue transactions to the Modbus_link object. The singleton Modbus_manager manages all links and implements toe link configuration web page. Needs some cleaning, but it works.

Next task is to get rid of plugins waiting for the transaction result. Currently this only works for the plugin_read and causes a delay of one full sample time. I want to implement a "callback" using the event mechanism. Can we design a generic event for responses from autonomous modules like the Modbus link? I can imagine there are other processes that may take time to generate a value. Using a PLUGIN_MODBUS_READ_RESULT sounds very specific for the Modbus.

@TD-er
Copy link
Copy Markdown
Member

TD-er commented Apr 25, 2026

We currently have the PLUGIN_TASKTIMER_IN, which I guess is somewhat related.
This is typically called through scheduling (see setPluginTaskTimer ) which also makes it even more 'async' in behavior.

A typical work flow can be something like this:

  • Task adds request to Modbus engine's queue. (giving some priority or not)
  • Modbus engine sends out next request
  • Modbus engine receives reply ('happy flow' here, no timeout) and schedules a PluginTaskTimer with the response
  • Modbus engine sends out next request.
  • Task processes scheduled PluginTaskTimer call + adds new request to Modbus engine's queue.

N.B. the task does 'know' it is a modbus task, so there is no ambiguity in what's intended with the PLUGIN_TASKTIMER_IN

@flashmark
Copy link
Copy Markdown
Contributor Author

I will give it a try with PLUGIN_TASKTIMER_IN. Let's do some research about the event mechanism and structure.

@TD-er
Copy link
Copy Markdown
Member

TD-er commented Apr 25, 2026

I will give it a try with PLUGIN_TASKTIMER_IN. Let's do some research about the event mechanism and structure.

You can start by looking at systemTimerStruct to see if you can fit everything to let the task know the read result.
If not, please let me know, so we can see what should/could be extended.

Co-authored-by: Copilot <copilot@github.com>
addLogMove(LOG_LEVEL_ERROR, F("P183: Invalid output index in task timer event"));
return false;
}
UserVar.setFloat(event->TaskIndex, outputIndex, event->Par1); // Update the user variable with the value read from Modbus
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value may not always be a float.
So maybe we could have a look at how to deal with those.

However a 16-bit value can still be stored in a float without loosing bits.

@TD-er
Copy link
Copy Markdown
Member

TD-er commented May 2, 2026

I just approved running your GH Actions build, just to see if there are any build issues so far.

@TD-er
Copy link
Copy Markdown
Member

TD-er commented May 2, 2026

OK, that didn't run for long....

Compiling .pio/build/minimal_core_312_ESP8266_1M_OTA_FHEM_HA/src/src/Controller_config/C023_AT_commands.cpp.o
src/src/Commands_tmp/__tmpfile.cpp:91368:10: fatal error: P183_data_struct.h: No such file or directory

**************************************************************************
* Looking for P183_data_struct.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:P183_data_struct.h"
* Web  > https://registry.platformio.org/search?q=header:P183_data_struct.h
*
**************************************************************************

91368 | #include "P183_data_struct.h"
      |          ^~~~~~~~~~~~~~~~~~~~

As you may not yet know... For ESP8266 builds, all .cpp files are concatenated into a single file (well all except the new src/ESPEasy/net/.... part)
This is for 2 reasons:

  • The linker would other wise fail due to too long argument list with all .cpp.o files
  • The linker is better at removing unused functions resulting in smaller builds.

However this has a major consequence, where you need to make the include path a bit longer as the concatenated file will be in a different directory from the .h files.

So the #include "P183_data_struct.h" should be changed into #include "../DataStructs/P183_data_struct.h"
And probably on other includes too.


# include "../Helpers/Modbus_device.h"
# include "../Helpers/Modbus_mgr.h"
# include "Modbus_device.h"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefix include path with ../Helpers/

#if FEATURE_MODBUS_FAC

# include "Modbus_device.h"
# include "Modbus_link.h"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here also missing ../Helpers/ in the includes

#if FEATURE_MODBUS_FAC

# include <ESPeasySerial.h>
# include "Modbus_mgr.h"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here also missing ../Helpers/ in the include

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants