[Privoxy-devel] 0007-Create-a-fast-CGI-function

[Lee]

On Monday, September 4th, 2023 at 12:24 PM, Fabian Keil wrote:

> Lee wrote on 2023-08-29 at 12:01:44:
> 
> > On Sunday, August 27th, 2023 at 11:37 AM, Fabian Keil wrote:
> > 
> > > I'm proposing that "we" add code
> > > that optionally lets Privoxy ignore
> > > the templates and skip work that isn't needed for your use case.
> > 
> > That was what I originally wanted to do.
> > The patch I gave you is pretty much my limit. I wanted to return just
> > Final results:
> > + or - block
> > but I couldn't figure out how to do only that :(
> 
> So you only want to know whether or not the block action applies?

Yup - that's it.  I want to give Privoxy a host name and find out if it would be blocked or not.

> > > > > What do you think about adding support for three additional
> > > > > optional variables to the "/show-url-info" handler instead?
> > > > > 
> > > > > Naming is hard, but right of my head we could add support
> > > > > for "final-results-only=1"
> > > > 
> > > > And that would use an in-memory template instead of reading it in
> > > > from disk?
> > > 
> > > I think the "final-results-only=1" should only suppress the non-final
> > > matches while "format=text/plain" should result in no reading of the
> > > HTML templates.
> > 
> > What happens when the user gives just "final-results-only=1"? The
> > template gets read in and the result is returned as text/html?
> 
> That seems reasonable to me.

That also seems reasonable to me, but useless for my purposes.  Reading a template in is **slow**.  (at least on my PC)

> > Same deal
> > if only "format=text/plain" is specified.. all the intermediate results
> > are returned along with the final results as text/plain?
> 
> 
> I'm not sure if we need to support this.

Right.. I can't think of a use case for that.

> > And if I get to add to my wish-list of things to do that are beyond my
> > ability to do, it would be nice if the "final-results-only=1" output was
> > created in a single output buffer. The way Privoxy works now, the
> > headers go in one buffer and the body goes in another - resulting in the
> > user program having to do two reads to get the results; one to get the
> > HTTP 1.1 200 OK headers and another to get the <DOCTYPE html> text.
> > It'd be faster if all that was sent/received in a single buffer.
> 
> 
> I'm not so sure about this. Sending the headers before the body
> allows the client to read and parse them while Privoxy is still
> sending the body.

Which seems like a good idea for traffic coming from the Internet.  Especially now with https-inspection allowing Privoxy to read in the entire web page and apply filters again.. it certainly seems reasonable to expect things to be faster if Privoxy can pass along the headers while it's still working on collecting and filtering the body.

I expect it to be noticeably slower when everything is happening on the same machine.  I originally had this in my awk program for talking to Privoxy:

  print "GET http://config.privoxy.org/show-url-final-info?url=" url " HTTP/1.1"  |& webserver
  print "Host: config.privoxy.org"                |& webserver
  print "Accept: text/html"                       |& webserver
  print "Connection: Keep-Alive"                  |& webserver
  print ""                                        |& webserver

That's slower than

  printf("GET http://config.privoxy.org/show-url-final-info?url=%s HTTP/1.1\r\n"\
         "Host: config.privoxy.org\r\n"\
         "Accept: text/html\r\n"\
         "Connection: Keep-Alive\r\n\r\n", url ) |& webserver

because the multiple print statements tend to cause multiple writes to Privoxy.  The single printf causes a single write to Privoxy and is clearly faster.

> > Are you imagining a case where someone would want one or two of
> > final-results-only=1
> > format=text/plain
> > include-multi-actions=0
> > but not all three?
> 
> 
> I haven't thought about it that much.
> 
> > and .. do you really want to allow final-results-only=0 or
> > include-multi-actions=1 ? Maybe just a single flag that means final
> > results only, no multi-actions and content-type: text/plain would be
> > better?
> 
> 
> final-results-only=0 would be the default behaviour and result
> in both the final results and the intermediate results to be
> shown.
> 
> Maybe we need to think about this some more.

Sounds good.  All I want is a single flag that means
1. do what you're doing now
or
2. be _*really*_ fast and just tell me if this host would be blocked or not.

Thanks,
Lee