[Privoxy-devel] 0007-Create-a-fast-CGI-function

[Fabian Keil]

Lee <ler762 at protonmail.com> wrote on 2023-08-25 at 05:10:56:

> On Thursday, August 17th, 2023 at 7:37 AM, Fabian Keil wrote:
> 
> > Obviously I think you have a valid use case but I'm
> > not sure that we need a new "fast CGI function" when
> > we could optionally speed up the existing one instead
> > which seems cleaner to me.
> 
> It might be possible to speed up the existing function a bit, but unless
> you can remove all disk accesses from the current function it _will_ be
> slow. I just tried a single call of
> http://config.privoxy.org/show-url-info?url=adsense.monetize-me.com
> hopefully the attached show-url-info-log.txt comes through, but the
> short story is that show-url-info reads in
> 
> templates\show-url-info
> templates\mod-title
> templates\mod-unstable-warning
> templates\mod-local-help
> templates\mod-support-and-service
> 
> and it takes a bit more than 8 milliseconds to read all that.
> But that's the first time; later on, when everything is in cache??? it's
> down to about 2 ms to read all that in.  Which would explain the 14
> seconds for show-url-info vs. 8 seconds for show-url-final-info that I
> just got when processing
> "https://pgl.yoyo.org/as/serverlist.php?hostformat=junkbuster (and
> that's with my 'if (continue_chatting && !(csp->flags &
> CSP_FLAG_CRUNCHED) ' patch to not always be checking the config file
> timestamps)
> 
> Now think about how long it would take to process more than 400K lines
> instead of 3K lines in a host file.

I'm proposing that "we" add code that optionally lets Privoxy ignore
the templates and skip work that isn't needed for your use case.

I just would prefer that "we" keep the /show-url-info handler and
don't add another one that then does work the already existing
one could do instead.

> > What do you think about adding support for three additional
> > optional variables to the "/show-url-info" handler instead?
> > 
> > Naming is hard, but right of my head we could add support
> > for "final-results-only=1"
> 
> And that would use an in-memory template instead of reading it in from
> disk?

I think the "final-results-only=1" should only suppress the non-final
matches while "format=text/plain" should result in no reading of the
HTML templates.

"We" could use an in-memory text-based template or simple build the
output page on the fly without using any template which should be
even faster as it wouldn't require pcrs.

> > and additionally add something
> > like "format=text/plain" so Privoxy doesn't waste time on
> > producing (somewhat ugly) HTML output which IIRC your script
> > tries to ignore (but still has to download) anyway.
> 
> It would be interesting to see how much plain text vs. html text would
> speed up things.

Agreed.

> > Finally
> > we could add something like "include-multi-actions=0".
> 
> All that sounds fine.  I just want it to be fast (and correct :).

Great.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.privoxy.org/pipermail/privoxy-devel/attachments/20230827/9de8f206/attachment.bin>