NE Derbyshire: Self-service lookup form removed, scraper completely broken?

[AndyNewsome]

Name of Council

North East Derbyshire

Issue Information

It appears that the council has completely removed their "Check your Bin Day" self-service lookup form. The URL https://myselfservice.ne-derbyshire.gov.uk/service/Check_your_Bin_Day now redirects to the council homepage. The fillform-frame-1 iframe that the scraper depends on no longer exists.

The bin collection dates page at https://www.ne-derbyshire.gov.uk/bins-and-recycling/bin-collection-dates now only provides downloadable PDF calendars — there is no web form at all.

Error:

selenium.common.exceptions.TimeoutException waiting for fillform-frame-1 iframe
The integration enters setup_retry with "Timeout while updating data" and never loads.

Additional info:

The council website is behind Cloudflare Turnstile, which blocks headless Chrome entirely and also blocks automated PDF downloads (even fetch() from within a non-headless session returns 403 on the /documents/ path)
The PDF calendars are split into Calendar A (North) and Calendar B (South) by area
The scraper will need a complete rewrite since the web form no longer exists — not just a fix to the existing code
Possibly related to #1861 which reports a different error on the same council
Environment: HA 2026.3.1, UK Bin Collection Data (latest via HACS), Selenium Grid 4.41.0

Error log:

This error originated from a custom integration.

Logger: custom_components.uk_bin_collection
Source: custom_components/uk_bin_collection/init.py:376
integration: UK Bin Collection Data (documentation, issues)
First occurred: 19:57:25 (2 occurrences)
Last logged: 19:58:25

[UKBinCollection] Timeout while updating data

Steps to reproduce the error:

Steps to reproduce:

1. Set up UK Bin Collection Data integration for NorthEastDerbyshireDistrictCouncil with a valid postcode and UPRN
2. Ensure Selenium Grid is running on port 4444
3. Reload or restart HA
4. Integration times out waiting for the fillform-frame-1 iframe which no longer exists
5. You can verify the form is gone by visiting https://myselfservice.ne-derbyshire.gov.uk/service/Check_your_Bin_Day — it redirects to the council homepage
6. The only bin collection info now available is via PDF calendars at https://www.ne-derbyshire.gov.uk/bins-and-recycling/bin-collection-dates

This might all be an error on the council's part, as most of their website still references the missing form though..

Verification

• I searched for similar issues at https://github.com/robbrad/UKBinCollectionData/issues?q=is:issue and found no duplicates
• I have checked my address/postcode/UPRN works on the council's website
• I have provided a detailed explanation of the issue as well as steps to replicate the issue
• I understand that this project is run by volunteer contributors therefore completion of this issue cannot be guaranteed

[m26dvd]

The North East Derbyshire website appears to be offline at the moment so I can't check.

#1861 reported a different issue which was fixed and pending release under #1841 however it may need to be completely re-written once the website comes back up

[m26dvd]

The site is back up and it appears the form has been removed but they still use the same system on the backend. It's possible that they are updated the form before re-publishing it so it's worth holding off on any re-writing of this scraper

[ksxykg4dyh-prog]

Any updates on this one please?