[Privoxy-devel] 0007-Create-a-fast-CGI-function

[Lee]

On Thursday, August 17th, 2023 at 7:37 AM, Fabian Keil wrote:

> Obviously I think you have a valid use case but I'm
> not sure that we need a new "fast CGI function" when
> we could optionally speed up the existing one instead
> which seems cleaner to me.

It might be possible to speed up the existing function a bit, but unless you can remove all disk accesses from the current function it _will_ be slow.
I just tried a single call of 
  http://config.privoxy.org/show-url-info?url=adsense.monetize-me.com
hopefully the attached show-url-info-log.txt comes through, but the short story is that show-url-info reads in

templates\show-url-info
templates\mod-title
templates\mod-unstable-warning
templates\mod-local-help
templates\mod-support-and-service

and it takes a bit more than 8 milliseconds to read all that.
But that's the first time; later on, when everything is in cache??? it's down to about 2 ms to read all that in.  Which would explain the 14 seconds for show-url-info vs. 8 seconds for show-url-final-info that I just got when processing
  "https://pgl.yoyo.org/as/serverlist.php?hostformat=junkbuster
(and that's with my 'if (continue_chatting && !(csp->flags & CSP_FLAG_CRUNCHED) ' patch to not always be checking the config file timestamps)

Now think about how long it would take to process more than 400K lines instead of 3K lines in a host file.

> What do you think about adding support for three additional
> optional variables to the "/show-url-info" handler instead?
> 
> Naming is hard, but right of my head we could add support
> for "final-results-only=1"

And that would use an in-memory template instead of reading it in from disk?

> and additionally add something
> like "format=text/plain" so Privoxy doesn't waste time on
> producing (somewhat ugly) HTML output which IIRC your script
> tries to ignore (but still has to download) anyway.

It would be interesting to see how much plain text vs. html text would speed up things.

> Finally
> we could add something like "include-multi-actions=0".

All that sounds fine.  I just want it to be fast (and correct :).

Regards,
Lee
-------------- next part --------------
12:03:20.1146995 AM	privoxy.exe	8628	IRP_MJ_CREATE	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	Desired Access: Generic Read, Disposition: Open
12:03:20.1148289 AM	privoxy.exe	8628	IRP_MJ_QUERY_EA	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	
12:03:20.1150828 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	Offset: 0, Length: 4,096, Priority: Normal
12:03:20.1151626 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	Offset: 0, Length: 4,096, I/O Flags: Non-cached, Paging I/O, Priority: Normal
12:03:20.1216891 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	Offset: 4,096, Length: 4,096
12:03:20.1221270 AM	privoxy.exe	8628	IRP_MJ_CREATE	C:\MyProgs\Privoxy.307\templates\mod-title		SUCCESS	Desired Access: Generic Read, Disposition: Open
12:03:20.1221719 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\mod-title		SUCCESS	Offset: 0, Length: 300, Priority: Normal
12:03:20.1222019 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\mod-title		END OF FILE	Offset: 300, Length: 4,096
12:03:20.1222168 AM	privoxy.exe	8628	IRP_MJ_CLEANUP	C:\MyProgs\Privoxy.307\templates\mod-title		SUCCESS	
12:03:20.1223095 AM	privoxy.exe	8628	IRP_MJ_CREATE	C:\MyProgs\Privoxy.307\templates\mod-unstable-warning	SUCCESS	Desired Access: Generic Read, Disposition: Open
12:03:20.1223390 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\mod-unstable-warning	SUCCESS	Offset: 0, Length: 375, Priority: Normal
12:03:20.1223632 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\mod-unstable-warning	END OF FILE	Offset: 375, Length: 4,096
12:03:20.1223754 AM	privoxy.exe	8628	IRP_MJ_CLEANUP	C:\MyProgs\Privoxy.307\templates\mod-unstable-warning	SUCCESS	
12:03:20.1224428 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\show-url-info		SUCCESS	Offset: 8,192, Length: 2,100
12:03:20.1225918 AM	privoxy.exe	8628	IRP_MJ_CREATE	C:\MyProgs\Privoxy.307\templates\mod-local-help		SUCCESS	Desired Access: Generic Read, Disposition: Open
12:03:20.1226197 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\mod-local-help		SUCCESS	Offset: 0, Length: 540, Priority: Normal
12:03:20.1226476 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\mod-local-help		END OF FILE	Offset: 540, Length: 4,096
12:03:20.1226599 AM	privoxy.exe	8628	IRP_MJ_CLEANUP	C:\MyProgs\Privoxy.307\templates\mod-local-help		SUCCESS	
12:03:20.1227465 AM	privoxy.exe	8628	IRP_MJ_CREATE	C:\MyProgs\Privoxy.307\templates\mod-support-and-service	SUCCESS	Desired Access: Generic Read, Disposition: Open
12:03:20.1227877 AM	privoxy.exe	8628	IRP_MJ_READ	C:\MyProgs\Privoxy.307\templates\mod-support-and-service	SUCCESS	Offset: 0, Length: 2,889, Priority: Normal
12:03:20.1228367 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\mod-support-and-service	END OF FILE	Offset: 2,889, Length: 4,096
12:03:20.1228489 AM	privoxy.exe	8628	IRP_MJ_CLEANUP	C:\MyProgs\Privoxy.307\templates\mod-support-and-service	SUCCESS	
12:03:20.1228774 AM	privoxy.exe	8628	FASTIO_READ	C:\MyProgs\Privoxy.307\templates\show-url-info	END OF FILE	Offset: 10,292, Length: 4,096
12:03:20.1228880 AM	privoxy.exe	8628	IRP_MJ_CLEANUP	C:\MyProgs\Privoxy.307\templates\show-url-info	SUCCESS	
12:03:20.1146995
================
        .0081885