Skip to content
This repository was archived by the owner on Apr 30, 2026. It is now read-only.

Fix robotparser: use longest-match rule per robots.txt spec (#148951)#1

Open
AmSach wants to merge 4 commits into
mainfrom
fix/robotparser-longest-match
Open

Fix robotparser: use longest-match rule per robots.txt spec (#148951)#1
AmSach wants to merge 4 commits into
mainfrom
fix/robotparser-longest-match