[Privoxy-users] Privoxy on a Raspberry Pi

Lee ler762 at gmail.com
Thu Mar 16 13:31:39 UTC 2017 

On 3/16/17, Phyks <phyks at phyks.me> wrote:
> Hi,
>
> $ time curl http://www.clubic.com -s > /dev/null  # (without privoxy)
>
> is around 0.3s.
>
> $ time curl -s -H 'Accept-Encoding: gzip,deflate' http://www.clubic.com
> | gunzip > /dev/null  # (without privoxy, enforce GZip decompression)
>
> is around the same time.
>
>
> $ time curl -x localhost:8118 http://www.clubic.com -s > /dev/null  #
> (with privoxy)
>
> is around 0.5s after editing the user.action as indicated in previous
> email (around 2s previously). It sometimes spikes around 2s with
> Privoxy, but I guess this is just some side issue with I/O or something
> else.

It could be the normal variation you get on a live network - eg
$ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null

real    0m1.404s
user    0m0.000s
sys     0m0.062s

$ time curl  http://www.privoxy.org -s > /dev/null

real    0m0.452s
user    0m0.000s
sys     0m0.015s

$ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null

real    0m0.640s
user    0m0.000s
sys     0m0.015s

> I am not sure whether these timings looks good or not, and what is
> happening with the gzipped data taking so long in Privoxy.

Maybe the Raspberry Pi is just slow unzipping the data?  I don't see
the problem on Windows:

2017-03-16 09:21:43.187 00000e0c Header: New HTTP Request-Line: GET / HTTP/1.1
2017-03-16 09:21:43.187 00000e0c Request: www.clubic.com/
2017-03-16 09:21:43.187 00000e0c Connect: to www.clubic.com
2017-03-16 09:21:43.483 00000e0c Connect: Connected to
www.clubic.com[141.138.91.70]:80.
2017-03-16 09:21:43.483 00000e0c Connect: Created new connection to
www.clubic.com:80 on socket 424.
2017-03-16 09:21:43.499 00000e0c Connect: to www.clubic.com successful
2017-03-16 09:21:43.577 00000e0c Header: scan: HTTP/1.1 200 OK
2017-03-16 09:21:43.577 00000e0c Header: scan: Server: nginx
2017-03-16 09:21:43.577 00000e0c Header: scan: Date: Thu, 16 Mar 2017
13:21:55 GMT
2017-03-16 09:21:43.577 00000e0c Header: scan: Content-Type:
text/html; charset=UTF-8
2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Length: 23772
2017-03-16 09:21:43.592 00000e0c Header: scan: Connection: keep-alive
2017-03-16 09:21:43.592 00000e0c Header: scan: Cache-Control:
max-age=300, public
2017-03-16 09:21:43.592 00000e0c Header: scan: Vary: Accept-Encoding
2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Encoding: gzip
2017-03-16 09:21:43.592 00000e0c Header: scan: X-Your-IP: x.x.x.x
2017-03-16 09:21:43.592 00000e0c Header: Keeping the server header
'Connection: keep-alive' around.
2017-03-16 09:21:43.592 00000e0c Header: csp->expected_content_length=23772
2017-03-16 09:21:43.592 00000e0c Header: Adding: Proxy-Connection: keep-alive
2017-03-16 09:21:43.592 00000e0c Connect: socket_is_still_usable:
socket 248: no_data_waiting=1 status=1
2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
socket 248: no_data_waiting=1 status=1
2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
socket 248: no_data_waiting=1 status=1
2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
socket 248: no_data_waiting=1 status=1
2017-03-16 09:21:43.670 00000e0c Connect: Done reading from server.
Content length: 23772 as expected. Bytes most recently read: 3070.
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: begin
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 65536
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 131072
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 262144
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: decompressed
2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: finished
2017-03-16 09:21:43.670 00000e0c Header: Adjusted Content-Length to 168242
2017-03-16 09:21:43.670 00000e0c Header: Crunching: Content-Encoding: gzip
2017-03-16 09:21:43.686 00000e0c Connect: jcc.c end for (;;)
content_length=168242 byte_count=23772
2017-03-16 09:21:43.764 00000e0c Connect: socket_is_still_usable:
socket 424: no_data_waiting=1 status=1
2017-03-16 09:21:43.764 00000e0c Connect: Waiting for the next client
request on socket 248. Keeping the server socket 424 to www.clubic.com
open.
2017-03-16 09:21:43.779 00000e0c Connect: socket_is_still_usable:
socket 248: no_data_waiting=0 status=1
2017-03-16 09:21:43.873 00000e0c Connect: Client request 2 arrived in
time on socket 248.
2017-03-16 09:21:43.873 00000e0c Connect: Complete client request received.
2017-03-16 09:21:43.873 00000e0c Header: scan: GET
http://www.clubic.com/assets/css/c06c6f0.css HTTP/1.1


Lee


>
>
> Le 2017-03-16 00:50, Lee a écrit :
>> This looks strange:
>> 2017-03-15 21:59:51.262 b6ceb460 Connect: Done reading from server.
>> Content length: 23531 as expected. Bytes most recently read: 4319.
>> 2017-03-15 21:59:52.517 b6ceb460 Header: Adjusted Content-Length to
>> 168029
>> 2017-03-15 21:59:52.517 b6ceb460 Header: Crunching: Content-Encoding:
>> gzip
>>
>> privoxy received 23KB of gziped data and unzips it to 168KB -- which
>> takes about 1.25 seconds!??
>>
>> Can you try adding
>>
>> { -filter }
>> /
>>
>> to the end of your user.action and try it again?
>>
>> http://config.privoxy.org/user-manual/actions-file.html#FILTER
>>   Compressed content can't be filtered either, but if Privoxy is
>> compiled with zlib support
>>   and a supported compression algorithm is used (gzip or deflate),
>> Privoxy can first
>>   decompress the content and then filter it.
>>
>> Regards,
>> Lee
>>
>>
>> On 3/15/17, Phyks <phyks at phyks.me> wrote:
>>> Hi,
>>>
>>> Framadrop works for me, weird. Anyway, I put it on a pastebin here:
>>> http://paste.alacon.org/43566.
>>>
>>> I did not yet try to update Privoxy, I will look into it (this is
>>> still
>>> on 3.0.21).
>>>
>>> The pi is running its own Unbound DNS resolver. It should work fine,
>>> and
>>> typically if I make my laptop use it as well, I have DNS resolution
>>> times comparable with 8.8.8.8.
>>>
>>> Thanks,
>>> --
>>> Phyks
>>>
>>> Le 15/03/2017 à 22:27, Ian Silvester a écrit :
>>>> Hi Phyks,
>>>>
>>>> I could not retrieve the log excerpt stored at framadrop.org - could
>>>> you
>>>> check and reconfirm?
>>>>
>>>> That aside, could you also supply details of the Pi's DNS
>>>> configuration
>>>> - are the servers listed explicitly anywhere or is it using DHCP and
>>>> fall back to those supplied by the LAN's router?
>>>>
>>>> Cheers,
>>>>
>>>> Ian
>>>>
>>>>
>>>> My PGP public
>>>> key[http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc]
>>>>
>>>> On Wed, 15 Mar 2017, at 17:07, Phyks wrote:
>>>>> Hi,
>>>>>
>>>>> I tried to debug it further. Here are some tests with the current
>>>>> settings.
>>>>>
>>>>> Using Privoxy home page:
>>>>> * Without Privoxy, the main HTML takes around 80ms to load.
>>>>> * With Privoxy, it takes around 500ms to load, if I reload the page
>>>>> (even clearing the browser cache), it drops to about 80ms which is
>>>>> in
>>>>> agreement with the without Privoxy case.
>>>>>
>>>>> (Screenshots available at
>>>>> https://lutim.phyks.me/gallery#jqNGoAnQ/86EWpbzr.png,Q23nsaOT/1XMA8IBi.png,
>>>>> put online as I am unsure about the policy with attached images on
>>>>> this
>>>>> ML)
>>>>>
>>>>>
>>>>> If I do the same test with http://clubic.com,
>>>>> * Without Privoxy, the main HTML takes around 54ms to load.
>>>>> * With Privoxy, it takes around 1.5s and this is reproducible even
>>>>> if I
>>>>> reload immediately the page.
>>>>>
>>>>>
>>>>> Looking at htop in parallel, Privoxy is spawning multiple processes
>>>>> (about 10) and CPU load goes from 3% to around 100%, used by these
>>>>> processes.
>>>>>
>>>>> The Raspberry Pi and my laptop are on the same local network, and
>>>>> the
>>>>> Raspberry Pi is wired with 100M ethernet to the router, so the
>>>>> network
>>>>> itself should be out of troubles.
>>>>>
>>>>>
>>>>> Looking at the log as stated in the contact page
>>>>> (https://framadrop.org/r/6j81o2D1m0#XNEfEVjRtqAXaHHI3YMRH3jtKRfxKdm7EADyfL/yEHk=),
>>>>> it
>>>>> seems that Privoxy is taking 1.5s to actually load the HTML content
>>>>> from
>>>>> the remote server.
>>>>>
>>>>> However,
>>>>>
>>>>> ```
>>>>> $ time curl http://www.clubic.com -s > /dev/null
>>>>>
>>>>> real    0m0.291s
>>>>> user    0m0.120s
>>>>> sys 0m0.030s
>>>>> ```
>>>>> so that it seems to be a problem related to Privoxy itself (and not
>>>>> due
>>>>> to some network limitation on the Raspberry Pi or whatever like
>>>>> this).
>>>>>
>>>>>
>>>>> Thanks!
>>>>> --
>>>>> Phyks
>>>>>
>>>>> Le 14/03/2017 à 19:50, Fabian Keil a écrit :
>>>>>> Phyks <phyks at phyks.me> wrote:
>>>>>>
>>>>>>> I installed Privoxy on Raspbian (latest, Debian derived, based on
>>>>>>> Jessie) through the repositories (version 3.0.21-7+deb8u1
>>>>>>> installed).
>>>>>>> Everything is running on a regular Raspberry Pi 1.
>>>>>>
>>>>>> Note that the current Privoxy version is 3.0.26 which means
>>>>>> you are missing out on a bunch of changes including various
>>>>>> performance improvements:
>>>>>> http://ijbswa.cvs.sourceforge.net/viewvc/ijbswa/current/ChangeLog?view=markup
>>>>>>
>>>>>> Our current versioning scheme is a bit silly and we only bump
>>>>>> the last number.
>>>>>>
>>>>>>> When I browse to any page (default configuration, no extra actions
>>>>>>> /
>>>>>>> filter added yet), the CPU usage goes to 100% on the Raspberry Pi,
>>>>>>> and
>>>>>>> it takes a few seconds to actually proxy the page, increasing the
>>>>>>> load
>>>>>>> time from a few milliseconds to some seconds.
>>>>>>
>>>>>> That seems unreasonably slow to me, too.
>>>>>>
>>>>>>> I was wondering if anyone had experience with a similar setup and
>>>>>>> could
>>>>>>> give me some input about the performances of the system. Also, is
>>>>>>> there
>>>>>>> anything I could do to make it lighter?
>>>>>>
>>>>>> If you enable the debug directives recommended at:
>>>>>> https://www.privoxy.org/user-manual/contact.html
>>>>>> the log file should help to diagnose the issue.
>>>>>>
>>>>>> Please reproduce the problem with logging enabled and provide
>>>>>> a log excerpt.
>>>>>>
>>>>>> Please also double check that the CPU usage is actually
>>>>>> caused by Privoxy itself.
>>>>>>
>>>>>> Fabian
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Privoxy-users mailing list
>>>>> Privoxy-users at lists.privoxy.org
>>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>> _______________________________________________
>>>> Privoxy-users mailing list
>>>> Privoxy-users at lists.privoxy.org
>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>
>>> _______________________________________________
>>> Privoxy-users mailing list
>>> Privoxy-users at lists.privoxy.org
>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>
>
> --
> Phyks
> _______________________________________________
> Privoxy-users mailing list
> Privoxy-users at lists.privoxy.org
> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>

More information about the Privoxy-users mailing list