[Privoxy-devel] 0007-Create-a-fast-CGI-function

[Fabian Keil]

Lee <ler762 at protonmail.com> wrote on 2023-09-05 at 20:28:40:

> On Monday, September 4th, 2023 at 12:24 PM, Fabian Keil wrote:
> 
> > Lee wrote on 2023-08-29 at 12:01:44:
> > 
> > > On Sunday, August 27th, 2023 at 11:37 AM, Fabian Keil wrote:
> > > 
> > > > I'm proposing that "we" add code
> > > > that optionally lets Privoxy ignore
> > > > the templates and skip work that isn't needed for your use case.
> > > 
> > > That was what I originally wanted to do.
> > > The patch I gave you is pretty much my limit. I wanted to return just
> > > Final results:
> > > + or - block
> > > but I couldn't figure out how to do only that :(
> > 
> > So you only want to know whether or not the block action applies?
> 
> Yup - that's it.  I want to give Privoxy a host name and find out if it would be blocked or not.

That could actually be useful for Privoxy-Regression-Test as well.

> > > > > > What do you think about adding support for three additional
> > > > > > optional variables to the "/show-url-info" handler instead?
> > > > > > 
> > > > > > Naming is hard, but right of my head we could add support
> > > > > > for "final-results-only=1"
> > > > > 
> > > > > And that would use an in-memory template instead of reading it in
> > > > > from disk?
> > > > 
> > > > I think the "final-results-only=1" should only suppress the non-final
> > > > matches while "format=text/plain" should result in no reading of the
> > > > HTML templates.
> > > 
> > > What happens when the user gives just "final-results-only=1"? The
> > > template gets read in and the result is returned as text/html?
> > 
> > That seems reasonable to me.
> 
> That also seems reasonable to me, but useless for my purposes.
> Reading a template in is **slow**.  (at least on my PC)

I suppose some filled-out templates could be cached as another
mechanism to speed things up.

> > > And if I get to add to my wish-list of things to do that are beyond my
> > > ability to do, it would be nice if the "final-results-only=1" output was
> > > created in a single output buffer. The way Privoxy works now, the
> > > headers go in one buffer and the body goes in another - resulting in the
> > > user program having to do two reads to get the results; one to get the
> > > HTTP 1.1 200 OK headers and another to get the <DOCTYPE html> text.
> > > It'd be faster if all that was sent/received in a single buffer.
> > 
> > 
> > I'm not so sure about this. Sending the headers before the body
> > allows the client to read and parse them while Privoxy is still
> > sending the body.
> 
> Which seems like a good idea for traffic coming from the Internet.
> Especially now with https-inspection allowing Privoxy to read in
> the entire web page and apply filters again.. it certainly seems
> reasonable to expect things to be faster if Privoxy can pass
> along the headers while it's still working on collecting and
> filtering the body.

If Privoxy is filtering the body it can't send the headers
prematurely as it may have to change the Content-Length header
if a body modification changed the length.

> I expect it to be noticeably slower when everything is happening
> on the same machine.  I originally had this in my awk program for
> talking to Privoxy:
> 
>   print "GET http://config.privoxy.org/show-url-final-info?url=" url " HTTP/1.1"  |& webserver
>   print "Host: config.privoxy.org"                |& webserver
>   print "Accept: text/html"                       |& webserver
>   print "Connection: Keep-Alive"                  |& webserver
>   print ""                                        |& webserver
> 
> That's slower than
> 
>   printf("GET http://config.privoxy.org/show-url-final-info?url=%s HTTP/1.1\r\n"\
>          "Host: config.privoxy.org\r\n"\
>          "Accept: text/html\r\n"\
>          "Connection: Keep-Alive\r\n\r\n", url ) |& webserver
> 
> because the multiple print statements tend to cause multiple writes
> to Privoxy.  The single printf causes a single write to Privoxy and
> is clearly faster.

That's what I would expect.

Is your awk program free software and publicly available?

> > > and .. do you really want to allow final-results-only=0 or
> > > include-multi-actions=1 ? Maybe just a single flag that means final
> > > results only, no multi-actions and content-type: text/plain would be
> > > better?
> > 
> > 
> > final-results-only=0 would be the default behaviour and result
> > in both the final results and the intermediate results to be
> > shown.
> > 
> > Maybe we need to think about this some more.
> 
> Sounds good.  All I want is a single flag that means
> 1. do what you're doing now
> or
> 2. be _*really*_ fast and just tell me if this host would be blocked or not.

I'll think about it some more.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.privoxy.org/pipermail/privoxy-devel/attachments/20230907/da8655ef/attachment.bin>