[Privoxy-users] gitweb performance/errors

[Fabian Keil]

tmknight88--- via Privoxy-users <privoxy-users at lists.privoxy.org> wrote on 2026-01-25 at 12:06:35:

> Hi, I wanted to report (assuming you are unaware) that the gitweb interface
> of privoxy.git performance has degraded since approximately 23 Jan 2026.
> For example, the summary page can take many seconds to load and frequently
> times out with a generic nginx error message.

Thanks for the report.

I was already aware of the problem but haven't had time to look into solving it.

The robots.txt already disallows crawling of /gitweb and a bunch
of obvious "AI" bots receive static error pages, but since about
three days ago there are simply too many requests coming in from
clients that ignore the robots.txt and don't use obvious "AI" bot
User-Agent headers either.

Looks like we need more crawler counter measures.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.privoxy.org/pipermail/privoxy-users/attachments/20260125/fc70ec75/attachment.bin>