[Privoxy-users] Privoxy on a Raspberry Pi

Phyks phyks at phyks.me
Mon Mar 20 13:33:57 UTC 2017


Hi,

Sorry for the late reply, I have been quite busy recently. I confirm 
everything is working well provided that I drop `img-reorder` filter, 
which I guess is just too computation intensive for my Raspberry Pi. I 
did not notice it was enabled by default (with a few extra filters) in 
the Debian package.

I found http://projects.zubr.me/wiki/adblock2privoxy and wanted to 
import the rules in my Privoxy instance on the Raspberry Pi, but it also 
takes a long time to proceed the pages (and consumes a lot of CPU). I 
guess the Raspberry Pi (at least first model) is not powerful enough for 
these kinds of tasks :/ I should try to grab a newer model and see if it 
fits better.

Thanks again for the help in setting this up!


Le 2017-03-16 14:31, Lee a écrit :
> On 3/16/17, Phyks <phyks at phyks.me> wrote:
>> Hi,
>> 
>> $ time curl http://www.clubic.com -s > /dev/null  # (without privoxy)
>> 
>> is around 0.3s.
>> 
>> $ time curl -s -H 'Accept-Encoding: gzip,deflate' 
>> http://www.clubic.com
>> | gunzip > /dev/null  # (without privoxy, enforce GZip decompression)
>> 
>> is around the same time.
>> 
>> 
>> $ time curl -x localhost:8118 http://www.clubic.com -s > /dev/null  #
>> (with privoxy)
>> 
>> is around 0.5s after editing the user.action as indicated in previous
>> email (around 2s previously). It sometimes spikes around 2s with
>> Privoxy, but I guess this is just some side issue with I/O or 
>> something
>> else.
> 
> It could be the normal variation you get on a live network - eg
> $ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null
> 
> real    0m1.404s
> user    0m0.000s
> sys     0m0.062s
> 
> $ time curl  http://www.privoxy.org -s > /dev/null
> 
> real    0m0.452s
> user    0m0.000s
> sys     0m0.015s
> 
> $ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null
> 
> real    0m0.640s
> user    0m0.000s
> sys     0m0.015s
> 
>> I am not sure whether these timings looks good or not, and what is
>> happening with the gzipped data taking so long in Privoxy.
> 
> Maybe the Raspberry Pi is just slow unzipping the data?  I don't see
> the problem on Windows:
> 
> 2017-03-16 09:21:43.187 00000e0c Header: New HTTP Request-Line: GET / 
> HTTP/1.1
> 2017-03-16 09:21:43.187 00000e0c Request: www.clubic.com/
> 2017-03-16 09:21:43.187 00000e0c Connect: to www.clubic.com
> 2017-03-16 09:21:43.483 00000e0c Connect: Connected to
> www.clubic.com[141.138.91.70]:80.
> 2017-03-16 09:21:43.483 00000e0c Connect: Created new connection to
> www.clubic.com:80 on socket 424.
> 2017-03-16 09:21:43.499 00000e0c Connect: to www.clubic.com successful
> 2017-03-16 09:21:43.577 00000e0c Header: scan: HTTP/1.1 200 OK
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Server: nginx
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Date: Thu, 16 Mar 2017
> 13:21:55 GMT
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Content-Type:
> text/html; charset=UTF-8
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Length: 23772
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Connection: keep-alive
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Cache-Control:
> max-age=300, public
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Vary: Accept-Encoding
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Encoding: gzip
> 2017-03-16 09:21:43.592 00000e0c Header: scan: X-Your-IP: x.x.x.x
> 2017-03-16 09:21:43.592 00000e0c Header: Keeping the server header
> 'Connection: keep-alive' around.
> 2017-03-16 09:21:43.592 00000e0c Header: 
> csp->expected_content_length=23772
> 2017-03-16 09:21:43.592 00000e0c Header: Adding: Proxy-Connection: 
> keep-alive
> 2017-03-16 09:21:43.592 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: Done reading from server.
> Content length: 23772 as expected. Bytes most recently read: 3070.
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: begin
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 65536
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 
> 131072
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 
> 262144
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: decompressed
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: finished
> 2017-03-16 09:21:43.670 00000e0c Header: Adjusted Content-Length to 
> 168242
> 2017-03-16 09:21:43.670 00000e0c Header: Crunching: Content-Encoding: 
> gzip
> 2017-03-16 09:21:43.686 00000e0c Connect: jcc.c end for (;;)
> content_length=168242 byte_count=23772
> 2017-03-16 09:21:43.764 00000e0c Connect: socket_is_still_usable:
> socket 424: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.764 00000e0c Connect: Waiting for the next client
> request on socket 248. Keeping the server socket 424 to www.clubic.com
> open.
> 2017-03-16 09:21:43.779 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=0 status=1
> 2017-03-16 09:21:43.873 00000e0c Connect: Client request 2 arrived in
> time on socket 248.
> 2017-03-16 09:21:43.873 00000e0c Connect: Complete client request 
> received.
> 2017-03-16 09:21:43.873 00000e0c Header: scan: GET
> http://www.clubic.com/assets/css/c06c6f0.css HTTP/1.1
> 
> 
> Lee
> 
> 
>> 
>> 
>> Le 2017-03-16 00:50, Lee a écrit :
>>> This looks strange:
>>> 2017-03-15 21:59:51.262 b6ceb460 Connect: Done reading from server.
>>> Content length: 23531 as expected. Bytes most recently read: 4319.
>>> 2017-03-15 21:59:52.517 b6ceb460 Header: Adjusted Content-Length to
>>> 168029
>>> 2017-03-15 21:59:52.517 b6ceb460 Header: Crunching: Content-Encoding:
>>> gzip
>>> 
>>> privoxy received 23KB of gziped data and unzips it to 168KB -- which
>>> takes about 1.25 seconds!??
>>> 
>>> Can you try adding
>>> 
>>> { -filter }
>>> /
>>> 
>>> to the end of your user.action and try it again?
>>> 
>>> http://config.privoxy.org/user-manual/actions-file.html#FILTER
>>>   Compressed content can't be filtered either, but if Privoxy is
>>> compiled with zlib support
>>>   and a supported compression algorithm is used (gzip or deflate),
>>> Privoxy can first
>>>   decompress the content and then filter it.
>>> 
>>> Regards,
>>> Lee
>>> 
>>> 
>>> On 3/15/17, Phyks <phyks at phyks.me> wrote:
>>>> Hi,
>>>> 
>>>> Framadrop works for me, weird. Anyway, I put it on a pastebin here:
>>>> http://paste.alacon.org/43566.
>>>> 
>>>> I did not yet try to update Privoxy, I will look into it (this is
>>>> still
>>>> on 3.0.21).
>>>> 
>>>> The pi is running its own Unbound DNS resolver. It should work fine,
>>>> and
>>>> typically if I make my laptop use it as well, I have DNS resolution
>>>> times comparable with 8.8.8.8.
>>>> 
>>>> Thanks,
>>>> --
>>>> Phyks
>>>> 
>>>> Le 15/03/2017 à 22:27, Ian Silvester a écrit :
>>>>> Hi Phyks,
>>>>> 
>>>>> I could not retrieve the log excerpt stored at framadrop.org - 
>>>>> could
>>>>> you
>>>>> check and reconfirm?
>>>>> 
>>>>> That aside, could you also supply details of the Pi's DNS
>>>>> configuration
>>>>> - are the servers listed explicitly anywhere or is it using DHCP 
>>>>> and
>>>>> fall back to those supplied by the LAN's router?
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Ian
>>>>> 
>>>>> 
>>>>> My PGP public
>>>>> key[http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc]
>>>>> 
>>>>> On Wed, 15 Mar 2017, at 17:07, Phyks wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> I tried to debug it further. Here are some tests with the current
>>>>>> settings.
>>>>>> 
>>>>>> Using Privoxy home page:
>>>>>> * Without Privoxy, the main HTML takes around 80ms to load.
>>>>>> * With Privoxy, it takes around 500ms to load, if I reload the 
>>>>>> page
>>>>>> (even clearing the browser cache), it drops to about 80ms which is
>>>>>> in
>>>>>> agreement with the without Privoxy case.
>>>>>> 
>>>>>> (Screenshots available at
>>>>>> https://lutim.phyks.me/gallery#jqNGoAnQ/86EWpbzr.png,Q23nsaOT/1XMA8IBi.png,
>>>>>> put online as I am unsure about the policy with attached images on
>>>>>> this
>>>>>> ML)
>>>>>> 
>>>>>> 
>>>>>> If I do the same test with http://clubic.com,
>>>>>> * Without Privoxy, the main HTML takes around 54ms to load.
>>>>>> * With Privoxy, it takes around 1.5s and this is reproducible even
>>>>>> if I
>>>>>> reload immediately the page.
>>>>>> 
>>>>>> 
>>>>>> Looking at htop in parallel, Privoxy is spawning multiple 
>>>>>> processes
>>>>>> (about 10) and CPU load goes from 3% to around 100%, used by these
>>>>>> processes.
>>>>>> 
>>>>>> The Raspberry Pi and my laptop are on the same local network, and
>>>>>> the
>>>>>> Raspberry Pi is wired with 100M ethernet to the router, so the
>>>>>> network
>>>>>> itself should be out of troubles.
>>>>>> 
>>>>>> 
>>>>>> Looking at the log as stated in the contact page
>>>>>> (https://framadrop.org/r/6j81o2D1m0#XNEfEVjRtqAXaHHI3YMRH3jtKRfxKdm7EADyfL/yEHk=),
>>>>>> it
>>>>>> seems that Privoxy is taking 1.5s to actually load the HTML 
>>>>>> content
>>>>>> from
>>>>>> the remote server.
>>>>>> 
>>>>>> However,
>>>>>> 
>>>>>> ```
>>>>>> $ time curl http://www.clubic.com -s > /dev/null
>>>>>> 
>>>>>> real    0m0.291s
>>>>>> user    0m0.120s
>>>>>> sys 0m0.030s
>>>>>> ```
>>>>>> so that it seems to be a problem related to Privoxy itself (and 
>>>>>> not
>>>>>> due
>>>>>> to some network limitation on the Raspberry Pi or whatever like
>>>>>> this).
>>>>>> 
>>>>>> 
>>>>>> Thanks!
>>>>>> --
>>>>>> Phyks
>>>>>> 
>>>>>> Le 14/03/2017 à 19:50, Fabian Keil a écrit :
>>>>>>> Phyks <phyks at phyks.me> wrote:
>>>>>>> 
>>>>>>>> I installed Privoxy on Raspbian (latest, Debian derived, based 
>>>>>>>> on
>>>>>>>> Jessie) through the repositories (version 3.0.21-7+deb8u1
>>>>>>>> installed).
>>>>>>>> Everything is running on a regular Raspberry Pi 1.
>>>>>>> 
>>>>>>> Note that the current Privoxy version is 3.0.26 which means
>>>>>>> you are missing out on a bunch of changes including various
>>>>>>> performance improvements:
>>>>>>> http://ijbswa.cvs.sourceforge.net/viewvc/ijbswa/current/ChangeLog?view=markup
>>>>>>> 
>>>>>>> Our current versioning scheme is a bit silly and we only bump
>>>>>>> the last number.
>>>>>>> 
>>>>>>>> When I browse to any page (default configuration, no extra 
>>>>>>>> actions
>>>>>>>> /
>>>>>>>> filter added yet), the CPU usage goes to 100% on the Raspberry 
>>>>>>>> Pi,
>>>>>>>> and
>>>>>>>> it takes a few seconds to actually proxy the page, increasing 
>>>>>>>> the
>>>>>>>> load
>>>>>>>> time from a few milliseconds to some seconds.
>>>>>>> 
>>>>>>> That seems unreasonably slow to me, too.
>>>>>>> 
>>>>>>>> I was wondering if anyone had experience with a similar setup 
>>>>>>>> and
>>>>>>>> could
>>>>>>>> give me some input about the performances of the system. Also, 
>>>>>>>> is
>>>>>>>> there
>>>>>>>> anything I could do to make it lighter?
>>>>>>> 
>>>>>>> If you enable the debug directives recommended at:
>>>>>>> https://www.privoxy.org/user-manual/contact.html
>>>>>>> the log file should help to diagnose the issue.
>>>>>>> 
>>>>>>> Please reproduce the problem with logging enabled and provide
>>>>>>> a log excerpt.
>>>>>>> 
>>>>>>> Please also double check that the CPU usage is actually
>>>>>>> caused by Privoxy itself.
>>>>>>> 
>>>>>>> Fabian
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Privoxy-users mailing list
>>>>>> Privoxy-users at lists.privoxy.org
>>>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>> _______________________________________________
>>>>> Privoxy-users mailing list
>>>>> Privoxy-users at lists.privoxy.org
>>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>> 
>>>> _______________________________________________
>>>> Privoxy-users mailing list
>>>> Privoxy-users at lists.privoxy.org
>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>> 
>> 
>> --
>> Phyks
>> _______________________________________________
>> Privoxy-users mailing list
>> Privoxy-users at lists.privoxy.org
>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>> 

-- 
Phyks


More information about the Privoxy-users mailing list