Skip to content

Fully asynchrounous host code (v2)#3490

Merged
hathach merged 18 commits intohathach:masterfrom
Precidata:non-blocking-host-v2
Mar 4, 2026
Merged

Fully asynchrounous host code (v2)#3490
hathach merged 18 commits intohathach:masterfrom
Precidata:non-blocking-host-v2

Conversation

@ceedriic
Copy link
Copy Markdown
Contributor

Describe the PR
This remove all calls to tusb_time_delay_ms_api() from inside tuh_task_ext().

Additional context

This is a cleaned-up version of #3482

Use of the host API was not really possible with OPT_OS_NONE due to blocking calls in tuh_task_ext().

With an OS, it will also improve the situation because:

  1. The timeout_ms parameter of tuh_task_ext() is now respected.
  2. The traffic with device 1 will not stop when device 2 is plugged.

I've split this in 3 main patches

  1. Long boring preparation patch to split the functions calling tusb_time_delay_ms_api().
  2. Executing the continuation functions asynchronously when CFG_TUH_TASK_USE_TIME_MILLIS_API is set.
  3. cleaning up the handling of deferred attachments with a separate queue (note: I've not really tested that last patch because with my hub and 2 devices for some reason I never got into this situation)

Comment thread src/host/usbh.c

#if OSAL_MUTEX_REQUIRED
#if CFG_TUH_HUB
osal_queue_delete(_usbh_daq);

Check warning

Code scanning / CodeQL

Expression has no effect Warning

This expression has no effect (because
osal_queue_delete
has no external side effects).
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Bot, this function will have side effects if CFG_TUSB_OS is different from OPT_OS_NONE

Comment thread src/host/usbh.c Fixed
Comment thread src/host/usbh.c Outdated
@ceedriic
Copy link
Copy Markdown
Contributor Author

ceedriic commented Feb 11, 2026

Hmm, any idea why hil-tinyusb would fail with this PR, but passed with #5484?
This should be exactly the same final diff (but against master)

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 22, 2026

Size Difference Report

Because TinyUSB code size varies by port and configuration, the metrics below represent the averaged totals across all example builds.

Note: If there is no change, only one value is shown.

Changes >1% in size

file .text .rodata .data .bss size % diff
usbh.c 4395 ➙ 4556 (+161) 60 ➙ 61 (+1) 63 ➙ 101 (+38) 954 ➙ 967 (+13) 5436 ➙ 5647 (+211) +3.9%
TOTAL 4395 ➙ 4556 (+161) 60 ➙ 61 (+1) 63 ➙ 101 (+38) 954 ➙ 967 (+13) 5436 ➙ 5647 (+211) +3.9%

Changes <1% in size

No entries.

No changes
file .text .rodata .data .bss size % diff
audio_device.c 2849 0 1248 1673 4518 +0.0%
cdc_device.c 1328 16 19 661 1988 +0.0%
cdc_host.c 6610 487 15 1539 8371 +0.0%
dcd_ch32_usbfs.c 1472 0 0 2444 3916 +0.0%
dcd_ch32_usbhs.c 1648 0 0 448 2096 +0.0%
dcd_ci_fs.c 1925 0 0 1290 3215 +0.0%
dcd_ci_hs.c 1762 0 0 1280 2530 +0.0%
dcd_da146xx.c 3067 0 0 144 3211 +0.0%
dcd_dwc2.c 4174 25 0 265 4463 +0.0%
dcd_eptri.c 2270 0 0 259 2529 +0.0%
dcd_khci.c 1953 0 0 1290 3243 +0.0%
dcd_lpc17_40.c 1470 0 0 648 1794 +0.0%
dcd_lpc_ip3511.c 1463 0 0 264 1639 +0.0%
dcd_mm32f327x_otg.c 1478 0 0 1290 2768 +0.0%
dcd_msp430x5xx.c 1796 0 0 176 1972 +0.0%
dcd_musb.c 2446 0 0 160 2606 +0.0%
dcd_nrf5x.c 2919 0 0 292 3211 +0.0%
dcd_nuc120.c 1093 0 0 78 1171 +0.0%
dcd_nuc121.c 1167 0 0 101 1268 +0.0%
dcd_nuc505.c 0 0 1529 157 1686 +0.0%
dcd_rp2040.c 859 20 604 655 2138 +0.0%
dcd_rusb2.c 2917 0 0 156 3073 +0.0%
dcd_samd.c 1032 0 0 266 1298 +0.0%
dcd_samg.c 1319 0 0 72 1391 +0.0%
dcd_stm32_fsdev.c 2557 0 0 291 2848 +0.0%
dfu_device.c 744 28 712 183 926 +0.0%
dfu_rt_device.c 156 0 134 0 156 +0.0%
dwc2_common.c 601 30 0 0 618 +0.0%
ecm_rndis_device.c 1037 0 1 2272 3310 +0.0%
ehci.c 2761 0 0 5970 7537 +0.0%
fsdev_common.c 180 0 0 0 180 +0.0%
hcd_ch32_usbfs.c 2484 0 0 498 2982 +0.0%
hcd_ci_hs.c 190 0 0 0 190 +0.0%
hcd_dwc2.c 4970 32 1 512 5516 +0.0%
hcd_khci.c 2442 0 0 449 2891 +0.0%
hcd_musb.c 3073 0 0 157 3230 +0.0%
hcd_pio_usb.c 262 0 240 0 502 +0.0%
hcd_rp2040.c 976 73 416 384 1849 +0.0%
hcd_rusb2.c 2923 0 0 245 3168 +0.0%
hcd_samd.c 2220 0 0 324 2544 +0.0%
hcd_stm32_fsdev.c 3282 0 1 420 3703 +0.0%
hid_device.c 1118 44 997 115 1233 +0.0%
hid_host.c 1206 0 0 1250 2456 +0.0%
hub.c 1235 8 8 29 1268 +0.0%
midi_device.c 1127 0 991 589 1714 +0.0%
midi_host.c 1353 7 7 3740 5097 +0.0%
msc_device.c 2518 108 2286 538 3056 +0.0%
msc_host.c 1589 0 0 394 1984 +0.0%
mtp_device.c 1689 22 1449 579 2275 +0.0%
ncm_device.c 1514 28 1408 5830 7358 +0.0%
ohci.c 1942 0 0 2414 4356 +0.0%
rp2040_usb.c 172 75 718 4 969 +0.0%
rusb2_common.c 160 0 16 0 176 +0.0%
tusb.c 429 ➙ 428 (-1) 0 368 3 430 +0.0%
tusb_fifo.c 843 0 477 0 838 +0.0%
typec_stm32.c 820 8 2 12 842 +0.0%
usbc.c 420 2 20 166 608 +0.0%
usbd.c 3191 57 89 276 3531 +0.0%
usbd_control.c 523 0 474 78 600 +0.0%
usbtmc_device.c 2176 24 69 291 2500 +0.0%
vendor_device.c 624 0 530 464 1087 +0.0%
video_device.c 4391 5 1851 472 4855 +0.0%
TOTAL 108915 ➙ 108914 (-1) 1099 16680 44557 155478 +0.0%

@hathach
Copy link
Copy Markdown
Owner

hathach commented Feb 25, 2026

Hmm, any idea why hil-tinyusb would fail with this PR, but passed with #5484? This should be exactly the same final diff (but against master)

usb subsystem on my VM sometime got "stuck" and/or timeout due to lots of usb attach/activity. May need to re-run or reset the whole system.

@ceedriic
Copy link
Copy Markdown
Contributor Author

Hmm, any idea why hil-tinyusb would fail with this PR, but passed with #5484? This should be exactly the same final diff (but against master)

usb subsystem on my VM sometime got "stuck" and/or timeout due to lots of usb attach/activity. May need to re-run or reset the whole system.

OK.

Anyway, after I did the [Merge branch 'hathach:master' into non-blocking-host-v2] two days ago, all tests passed.

Comment thread src/host/usbh.c Outdated
if (is_empty) {
return; // Exit if this is the only event in the queue, otherwise we loop forever
}
TU_ASSERT(osal_queue_send(_usbh_daq, &event, in_isr),);
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple queue is not currently supported. E.g freeRTOS will put running task() to suspended mode and switch to other task when trying to get from the queue. I am making some refactor, and also rever this for now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just ignore the third patch for now then...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(All three patches can be applied independently)

Copy link
Copy Markdown
Contributor Author

@ceedriic ceedriic Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: the second queue is only used with timeout=0, and only from the task() thread, so maybe the OS_NONE implementation could be used for that second queue?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh right, I overlook it, we can use queue receive with timeout=0, then. I did reverted, but will re-add that in the next push.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed now with latest push.

Comment thread src/host/usbh.c Fixed
Comment thread src/host/usbh.c Fixed
Comment thread src/host/usbh.c Dismissed
Comment thread src/tusb_option.h Outdated
#define CFG_TUH_TASK_USE_TIME_MILLIS_API 1
#endif
#endif

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My goal with that check was to enable the new asynchronous code with or without OS (because it is better in BOTH cases).
Unfortunately, somme platforms do not define tusb_time_millis_api() so I had to use this check to support them and get the CI green.
If tusb_time_millis_api() could be defined for all platforms, then I think this conditional code could just be removed. Unfortunately, adding the missing tusb_time_millis_api() is not something I can do.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most rtos support time api that return time since boot. I will add osal_time_millis() and make it official. It is currently implement in user (bsp in the example).

Copy link
Copy Markdown
Owner

@hathach hathach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfect, thank you @ceedriic for the PR. I make some changes

  • replace CFG_TUH_TASK_USE_TIME_MILLIS_API with CFG_TUSB_OS_HAS_SCHEDULER
  • rename usbh_wait_delay to usbh_call_after_ms() and also take arg. This is useful helper, maybe used in other places as well in the future
  • merge all delay/after callback to make it easier to maintain
    I also make some other changes while doing local tests as well. Let me know what you think, or do we need to make any other improvement.

Comment thread src/host/usbh.c

#if CFG_TUH_HUB
// Deferred attachment queue, only needed when using hub
OSAL_QUEUE_DEF(usbh_int_set, _usbh_daqdef, CFG_TUH_HUB, hcd_event_t);
Copy link
Copy Markdown
Owner

@hathach hathach Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these event is only generated one per hub, since we process hub port status 1 by 1. The only time it got another attachment is from other hub, or from same hub (with multiple device) when an enumerating device is set_configured(). I short, max size is CFG_TUH_HUB

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, great, this is why I couldn't test this code with a single hub :)

@ceedriic
Copy link
Copy Markdown
Contributor Author

I'm reviewing the latest code, give me an hour or so...

implement tusb_time_millis_api() with osal_time_millis() when OS is not NONE
@hathach hathach force-pushed the non-blocking-host-v2 branch from 0cbdf07 to e492748 Compare February 27, 2026 16:54
@hathach
Copy link
Copy Markdown
Owner

hathach commented Feb 27, 2026

I'm reviewing the latest code, give me an hour or so...

beware, there is a huge change list that replace board_millis() with tusb_time_millis_api() (which is required when OS=none). You can skip files in examples and hw/bsp

@hathach hathach force-pushed the non-blocking-host-v2 branch from e492748 to 8444c25 Compare February 27, 2026 17:01
@ceedriic
Copy link
Copy Markdown
Contributor Author

perfect, thank you @ceedriic for the PR. I make some changes

Thank you for reviewing/improving!

  • replace CFG_TUH_TASK_USE_TIME_MILLIS_API with CFG_TUSB_OS_HAS_SCHEDULER

This is the part I do not understand: I agree my option name (CFG_TUH_TASK_USE_TIME_MILLIS_API) was bad,
but if I understand the new define, you will not run the asynchronous code with a multitasking RTOS.
I only use OPT_OS_NONE at the moment, so this is fine for me, but why not running the asynchronous code with a RTOS?

  • rename usbh_wait_delay to usbh_call_after_ms() and also take arg. This is useful helper, maybe used in other places as well in the future
  • merge all delay/after callback to make it easier to maintain

Yes, both these changes make the code better.

I also make some other changes while doing local tests as well. Let me know what you think, or do we need to make any other improvement.

I've basically two small things:

  1. I see that you've added code I missed in tuh_task_event_ready() :)
    This is obviously correct, but I think it is a bit more complicated, and the following patch should be added (I can push it if you agree)
   #if CFG_TUH_HUB
-  if (!osal_queue_empty(_usbh_daq)) {
-    return true;
+  if (_usbh_data.enumerating_daddr == TUSB_INDEX_INVALID_8 && !osal_queue_empty(_usbh_daq)) {
+    return true; // Deferred attachment ready to run
+  }
+  #endif
+
+  #if CFG_TUSB_OS_HAS_SCHEDULER == 0
+  if (_usbh_data.call_after.func) {
+      int32_t ms = (int32_t)(_usbh_data.call_after.at_ms - tusb_time_millis_api());
+      if (ms <= 0)
+         return true; // Timer event ready to run
   }
   #endif
  1. You changed the following code:
  if (delay_cb) {
    int32_t ms = (int32_t)(_usbh_data.enum_wait_deadline - tusb_time_millis_api());
    if (ms <= 0) {
      // delay expired, run callback now
      TU_LOG_USBH("USBH run timer callback\r\n");
      _usbh_data.enum_wait_delay_cb = NULL;
      delay_cb();
    } else if (timeout_ms > (uint32_t)ms) {
      // reduce timeout accordingly
      timeout_ms = (uint32_t)ms;
    }
  }

Into a simplified version:

    if (after_cb) {
      uint32_t ms = tusb_time_millis_api();
      if (ms >= _usbh_data.call_after.at_ms) {
        TU_LOG_USBH("USBH run timer callback\r\n");
        _usbh_data.call_after.func = NULL;
        after_cb(_usbh_data.call_after.arg);
      }
    }

But I think it is wrong for two reasons:

  1. The comparaison ms >= _usbh_data.call_after.at_ms will fail when the timer rolls over after 50 days.
    if '_usbh_data.call_after.at_ms' equals 4'294'967'295 and 'ms' wrap to 0, then the new test will never fire anymore.
    But by converting to a signed integer first, ms will be equals to -1, and the test will always be correct by the magic of 2's complement.

  2. if in the original code ms > 0 (for example 10), it means that the next timer event will need to occur after 10ms.
    if the timeout timeout_ms given in tuh_task_ext was bigger than that (for example 100), then we need to reduce this timout from 100 to 10 to make sure that if no message are in the queue, we still exits tuh_task_ext() after 10ms, to run the next timer event on time. (I can also push it if you agree).

Other than that, at least for a quick look, I like your changes.

@hathach
Copy link
Copy Markdown
Owner

hathach commented Mar 2, 2026

@ceedriic you are right, we shouldn't rely on rtos delay since it blocks main event queue. I apparently does not have enough sleep, let me try to revert those change.

- call after (enum non-blocking delay) also support rtos now
Copy link
Copy Markdown
Owner

@hathach hathach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ceedriic I revert the changes to also support non-blocking on RTOS, also rename to usbh_defer_func_ms_async() and make it a helper, though usage outside of enumeration need some kind of mutex. We will address that later if needed. Let me know if you think it looks good now ?

Comment thread src/host/usbh.c
};

// process async delay in enumeration
static void enum_delay_async(uintptr_t state) {

Check notice

Code scanning / CodeQL

Unused static function Note

Static function enum_delay_async is unreachable (
process_enumeration
must be removed at the same time)
@ceedriic
Copy link
Copy Markdown
Contributor Author

ceedriic commented Mar 2, 2026

@ceedriic I revert the changes to also support non-blocking on RTOS, also rename to usbh_defer_func_ms_async() and make it a helper, though usage outside of enumeration need some kind of mutex. We will address that later if needed. Let me know if you think it looks good now ?

I took a quick look at the usbh.c change, and I think is is OK, I've just 3 comments (the only important is the first one)

  1. In the following hunk, I think the osal_queue_empty(_usbh_daq) added check inside CFG_TUH_HUB is incorrect and should be removed (if there are deferred attachments, we don't want to loop forever here)
+  #if CFG_TUSB_OS_HAS_SCHEDULER
+    // return if there are no more events, to allow application to run other backgrounds
+    if (osal_queue_empty(_usbh_q)
+    #if CFG_TUH_HUB
+        && osal_queue_empty(_usbh_daq)
+    #endif
+    ) {
+      return;
+    }
+  #endif

Thinking about it, I think the whole chunk here could also be simply replaced by a single line, without conditionals, to prevent sleeping a second time with an RTOS:

timeout_ms = 0;
  1. The following comment should be removed or fixed:
// For non-scheduler deferred callback. For scheduler OS: blocking delay then callback
bool usbh_defer_func_ms_async(uint32_t ms, tusb_defer_func_t func, uintptr_t param) {
  1. You've not updated the tuh_task_event_ready() function, I don't know if that's on purpose or not.

Comment thread src/host/usbh.c Outdated

// above after_cb() can re-schedule another function, we need to re-check and reduce timeout of
// the main event timeout to make sure we aren't blocking more than call_after timeout.
if (_usbh_data.call_after.func != NULL &&
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ceedriic there is a gotcha, the callback can re-schedule another function. If we miss that main queue can sleep forever, and then there is no event (since we need that callback to submit new transfer for new event).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, Yep, good catch!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just worried about the unsigned comparaisons with your latest patch.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to go for lunch but I will get back to you after

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hathach
What do you think about the following alternative patch?

I think it's simpler and follow the rule to exit tuh_task() after we've processed some events.

diff --git a/src/host/usbh.c b/src/host/usbh.c
index a674b040e..80f8e5672 100644
--- a/src/host/usbh.c
+++ b/src/host/usbh.c
@@ -666,6 +666,7 @@ void tuh_task_ext(uint32_t timeout_ms, bool in_isr) {
         TU_LOG_USBH("USBH invoke scheduled function\r\n");
         _usbh_data.call_after.func = NULL;
         after_cb(_usbh_data.call_after.arg);
+        timeout_ms = 0; // no sleeping after processing en event
       } else if (timeout_ms > (uint32_t)ms) {
         // reduce main event timeout to make sure we don't blocking more than call_after timeout
         timeout_ms = (uint32_t)ms;
@@ -786,16 +787,7 @@ void tuh_task_ext(uint32_t timeout_ms, bool in_isr) {
         break;
     }
 
-  #if CFG_TUSB_OS_HAS_SCHEDULER
-    // return if there are no more events, to allow application to run other backgrounds
-    if (osal_queue_empty(_usbh_q)
-    #if CFG_TUH_HUB
-        && osal_queue_empty(_usbh_daq)
-    #endif
-    ) {
-      return;
-    }
-  #endif
+    timeout_ms = 0; // no sleeping after processing en event
   }
 }

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ceedriic I have push a new update, that better address the signe comparison. Aso the timeout_ms = 0 is good one, just applied that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hathach:
in your latest patch if remain_ms <= 0, you also need to set timeout_ms = 0

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that is kind of unexpected for a new calback expired right after scheduled but it is possible

Comment thread src/host/usbh.c Dismissed
@ceedriic
Copy link
Copy Markdown
Contributor Author

ceedriic commented Mar 3, 2026 via email

Copy link
Copy Markdown
Owner

@hathach hathach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfect, thank you @ceedriic for this excellent PR. all issues are resolved, will merge when ci passed.

@hathach hathach merged commit 13e0b0c into hathach:master Mar 4, 2026
379 of 381 checks passed
@ceedriic ceedriic deleted the non-blocking-host-v2 branch March 5, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants