fix: add retry logic for API timeout errors#1
Open
beaglemoo wants to merge 7 commits into
Open
Conversation
- Ghost publisher generates HTML with regional price tables and UK top 20 cheapest for E10 and diesel, auto-updates sillymoo.dev/uk-fuel-prices - Grafana dashboard with stat panels, price table, time series trends, brand and region comparison charts with template variables - Updated config to support Ghost page_id for direct page updates
- Add extract_location() to parse town/city from station trading names - Rewrite Ghost page with client-side search/filter covering all ~6,300 stations - Embed station data as JSON, render results dynamically with JavaScript - Search matches on station name, brand, and extracted location - Sort by unleaded, diesel, super, or name with clickable controls - Paginated results (100 at a time) for performance - Keep regional highlights and Top 20 as static sections below search - Fix: pass page_id to Ghost API, use internal blog URL for admin access
Ghost's codeinjection_foot has a ~65KB limit which can't hold the ~655KB station JSON. Instead, SCP the JSON to Ghost's content/files directory and fetch it client-side. Also removes regional highlight sections (redundant with search), adds __main__.py for python -m support, and adds ghost.host config for SSH uploads.
Adds JSON-LD Dataset markup to the fuel prices page code injection header, telling Google this is a dataset based on GOV.UK government data with regular updates.
Show top 10 for Unleaded, Diesel, Super Unleaded, and Premium Diesel instead of just top 20 for Unleaded and Diesel.
- Station names link to Google Maps search (Top 10 tables + search results) - Brand filter buttons (Shell, BP, Esso, Tesco, etc.) - Price staleness indicator with coloured dots and age labels - Search result summary stats (avg, cheapest, range) - Enhanced Dataset schema with keywords, spatialCoverage, distribution - Dynamic FAQ schema with live cheapest price data - Keyword-rich static paragraph for Google crawlability - Mobile polish: iOS zoom fix, 44px tap targets, scrollable brand bar
The OAuth token request and data fetch had no retry handling for timeouts, causing complete run failures when the GOV.UK API was temporarily slow (e.g. during BST clock change). Adds up to 3 retries with 5/15/30s backoff delays for ReadTimeout, ConnectTimeout, and PoolTimeout errors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
httpx.ReadTimeoutduring the clock changeReadTimeout,ConnectTimeout, andPoolTimeoutin bothauth.pyandclient.pyTest plan