[Privoxy-devel] privoxy-regression-test --check-ssl flaky

[Fabian Keil]

Fabian Keil via Privoxy-devel <privoxy-devel at lists.privoxy.org> wrote on 2026-07-01 at 16:13:43:

> Roland Rosenfeld via Privoxy-devel <privoxy-devel at lists.privoxy.org> wrote on 2026-06-29 at 16:16:13:
> 
> > in https://bugs.debian.org/1141069 I was informed, that
> >  privoxy-regression-test --check-ssl
> > runs very flaky since ~1 month.
> > I run the test by hand several times now and can confirm the problem.
> 
> I was able to reproduce the issue as well.
>  
> > sometimes no-subject times out, then self-signed and no-subject return
> > 503 instead of 403, next time expired, no-common-name and no-subject
> > return 503 and so on.
> > 
> > No idea, what changed (this worked okay before, except that the timing
> > looks of the beginning of the problem looks like the problems started
> > only days after our 4.2.0 release, but during the release cycle there
> > were no problems).  Maybe some anti-AI mechanism on check.ssl.com,
> > which fails on several requests in a short time?
> > 
> > Any idea to fix this issue?  Are there (better) alternatives to
> > badssl.com, that can be used for testing?  Or does it make sense to
> > retry and/or skip a test, if it returns 503 or timeout instead of 403?
> > 
> > Running a local copy of badssl (the code is on
> > https://github.com/chromium/badssl.com), sounds like overkill at least
> > for the Debian CI-testing...
> 
> I'm not aware of any badssl.com alternatives.
> 
> I'm currently testing a --tolerate-flakiness option that only
> counts status code 200 as error that affects the exit code and
> results in output like:
> 
> fk at t520 ~ $privoxy-regression-test.pl --check-bad-ssl --tolerate-flakiness
> 2026-07-01 16:10:26: Requesting pages from badssl.com with various certificate problems. This will only work if Privoxy has been configured properly and can reach the Internet.
> 2026-07-01 16:10:26: Requesting https://expired.badssl.com/
> 2026-07-01 16:10:36: Fetch failure: 'curl: (28) Operation timed out after 10038 milliseconds with 0 bytes received'
> 2026-07-01 16:10:46: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
> 2026-07-01 16:10:46: Requesting https://wrong.host.badssl.com/
> 2026-07-01 16:10:56: Fetch failure: 'curl: (28) Operation timed out after 10002 milliseconds with 0 bytes received'
> 2026-07-01 16:11:06: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
> 2026-07-01 16:11:06: Requesting https://self-signed.badssl.com/
> 2026-07-01 16:11:16: Fetch failure: 'curl: (28) Operation timed out after 10001 milliseconds with 0 bytes received'
> 2026-07-01 16:11:26: Fetch failure: 'curl: (28) Operation timed out after 10027 milliseconds with 0 bytes received'
> 2026-07-01 16:11:26: Requesting https://untrusted-root.badssl.com/
> 2026-07-01 16:11:36: Fetch failure: 'curl: (28) Operation timed out after 10001 milliseconds with 0 bytes received'
> 2026-07-01 16:11:46: Fetch failure: 'curl: (28) Operation timed out after 10003 milliseconds with 0 bytes received'
> 2026-07-01 16:11:46: Requesting https://no-common-name.badssl.com/
> 2026-07-01 16:11:48: Requesting https://no-subject.badssl.com/
> 2026-07-01 16:11:49: Requesting https://incomplete-chain.badssl.com/
> 2026-07-01 16:11:54: There were 4 requests that did not result in status code 403!
> 2026-07-01 16:11:54: Tolerated 4 failure(s) due to the --tolerate-flakiness option.

The "--tolerate-flakiness" option is available in git master now.
 
Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 870 bytes
Desc: OpenPGP digital signature
URL: <https://lists.privoxy.org/pipermail/privoxy-devel/attachments/20260726/af201867/attachment.bin>