[Privoxy-devel] privoxy-regression-test --check-ssl flaky

[Fabian Keil]

Roland Rosenfeld via Privoxy-devel <privoxy-devel at lists.privoxy.org> wrote on 2026-06-29 at 16:16:13:

> in https://bugs.debian.org/1141069 I was informed, that
>  privoxy-regression-test --check-ssl
> runs very flaky since ~1 month.
> I run the test by hand several times now and can confirm the problem.

I was able to reproduce the issue as well.
 
> sometimes no-subject times out, then self-signed and no-subject return
> 503 instead of 403, next time expired, no-common-name and no-subject
> return 503 and so on.
> 
> No idea, what changed (this worked okay before, except that the timing
> looks of the beginning of the problem looks like the problems started
> only days after our 4.2.0 release, but during the release cycle there
> were no problems).  Maybe some anti-AI mechanism on check.ssl.com,
> which fails on several requests in a short time?
> 
> Any idea to fix this issue?  Are there (better) alternatives to
> badssl.com, that can be used for testing?  Or does it make sense to
> retry and/or skip a test, if it returns 503 or timeout instead of 403?
> 
> Running a local copy of badssl (the code is on
> https://github.com/chromium/badssl.com), sounds like overkill at least
> for the Debian CI-testing...

I'm not aware of any badssl.com alternatives.

I'm currently testing a --tolerate-flakiness option that only
counts status code 200 as error that affects the exit code and
results in output like:

fk at t520 ~ $privoxy-regression-test.pl --check-bad-ssl --tolerate-flakiness
2026-07-01 16:10:26: Requesting pages from badssl.com with various certificate problems. This will only work if Privoxy has been configured properly and can reach the Internet.
2026-07-01 16:10:26: Requesting https://expired.badssl.com/
2026-07-01 16:10:36: Fetch failure: 'curl: (28) Operation timed out after 10038 milliseconds with 0 bytes received'
2026-07-01 16:10:46: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
2026-07-01 16:10:46: Requesting https://wrong.host.badssl.com/
2026-07-01 16:10:56: Fetch failure: 'curl: (28) Operation timed out after 10002 milliseconds with 0 bytes received'
2026-07-01 16:11:06: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
2026-07-01 16:11:06: Requesting https://self-signed.badssl.com/
2026-07-01 16:11:16: Fetch failure: 'curl: (28) Operation timed out after 10001 milliseconds with 0 bytes received'
2026-07-01 16:11:26: Fetch failure: 'curl: (28) Operation timed out after 10027 milliseconds with 0 bytes received'
2026-07-01 16:11:26: Requesting https://untrusted-root.badssl.com/
2026-07-01 16:11:36: Fetch failure: 'curl: (28) Operation timed out after 10001 milliseconds with 0 bytes received'
2026-07-01 16:11:46: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
2026-07-01 16:11:46: Requesting https://no-common-name.badssl.com/
2026-07-01 16:11:48: Requesting https://no-subject.badssl.com/
2026-07-01 16:11:49: Requesting https://incomplete-chain.badssl.com/
2026-07-01 16:11:54: There were 4 requests that did not result in status code 403!
2026-07-01 16:11:54: Tolerated 4 failure(s) due to the --tolerate-flakiness option.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.privoxy.org/pipermail/privoxy-devel/attachments/20260701/598001b4/attachment.bin>