[Privoxy-users] Privoxy on a Raspberry Pi
Phyks
phyks at phyks.me
Mon Mar 20 13:33:57 UTC 2017
Hi,
Sorry for the late reply, I have been quite busy recently. I confirm
everything is working well provided that I drop `img-reorder` filter,
which I guess is just too computation intensive for my Raspberry Pi. I
did not notice it was enabled by default (with a few extra filters) in
the Debian package.
I found http://projects.zubr.me/wiki/adblock2privoxy and wanted to
import the rules in my Privoxy instance on the Raspberry Pi, but it also
takes a long time to proceed the pages (and consumes a lot of CPU). I
guess the Raspberry Pi (at least first model) is not powerful enough for
these kinds of tasks :/ I should try to grab a newer model and see if it
fits better.
Thanks again for the help in setting this up!
Le 2017-03-16 14:31, Lee a écrit :
> On 3/16/17, Phyks <phyks at phyks.me> wrote:
>> Hi,
>>
>> $ time curl http://www.clubic.com -s > /dev/null # (without privoxy)
>>
>> is around 0.3s.
>>
>> $ time curl -s -H 'Accept-Encoding: gzip,deflate'
>> http://www.clubic.com
>> | gunzip > /dev/null # (without privoxy, enforce GZip decompression)
>>
>> is around the same time.
>>
>>
>> $ time curl -x localhost:8118 http://www.clubic.com -s > /dev/null #
>> (with privoxy)
>>
>> is around 0.5s after editing the user.action as indicated in previous
>> email (around 2s previously). It sometimes spikes around 2s with
>> Privoxy, but I guess this is just some side issue with I/O or
>> something
>> else.
>
> It could be the normal variation you get on a live network - eg
> $ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null
>
> real 0m1.404s
> user 0m0.000s
> sys 0m0.062s
>
> $ time curl http://www.privoxy.org -s > /dev/null
>
> real 0m0.452s
> user 0m0.000s
> sys 0m0.015s
>
> $ time curl -x localhost:8118 http://www.privoxy.org -s > /dev/null
>
> real 0m0.640s
> user 0m0.000s
> sys 0m0.015s
>
>> I am not sure whether these timings looks good or not, and what is
>> happening with the gzipped data taking so long in Privoxy.
>
> Maybe the Raspberry Pi is just slow unzipping the data? I don't see
> the problem on Windows:
>
> 2017-03-16 09:21:43.187 00000e0c Header: New HTTP Request-Line: GET /
> HTTP/1.1
> 2017-03-16 09:21:43.187 00000e0c Request: www.clubic.com/
> 2017-03-16 09:21:43.187 00000e0c Connect: to www.clubic.com
> 2017-03-16 09:21:43.483 00000e0c Connect: Connected to
> www.clubic.com[141.138.91.70]:80.
> 2017-03-16 09:21:43.483 00000e0c Connect: Created new connection to
> www.clubic.com:80 on socket 424.
> 2017-03-16 09:21:43.499 00000e0c Connect: to www.clubic.com successful
> 2017-03-16 09:21:43.577 00000e0c Header: scan: HTTP/1.1 200 OK
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Server: nginx
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Date: Thu, 16 Mar 2017
> 13:21:55 GMT
> 2017-03-16 09:21:43.577 00000e0c Header: scan: Content-Type:
> text/html; charset=UTF-8
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Length: 23772
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Connection: keep-alive
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Cache-Control:
> max-age=300, public
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Vary: Accept-Encoding
> 2017-03-16 09:21:43.592 00000e0c Header: scan: Content-Encoding: gzip
> 2017-03-16 09:21:43.592 00000e0c Header: scan: X-Your-IP: x.x.x.x
> 2017-03-16 09:21:43.592 00000e0c Header: Keeping the server header
> 'Connection: keep-alive' around.
> 2017-03-16 09:21:43.592 00000e0c Header:
> csp->expected_content_length=23772
> 2017-03-16 09:21:43.592 00000e0c Header: Adding: Proxy-Connection:
> keep-alive
> 2017-03-16 09:21:43.592 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.670 00000e0c Connect: Done reading from server.
> Content length: 23772 as expected. Bytes most recently read: 3070.
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: begin
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc 65536
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc
> 131072
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: realloc
> 262144
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: decompressed
> 2017-03-16 09:21:43.670 00000e0c Connect: decompress_iob: finished
> 2017-03-16 09:21:43.670 00000e0c Header: Adjusted Content-Length to
> 168242
> 2017-03-16 09:21:43.670 00000e0c Header: Crunching: Content-Encoding:
> gzip
> 2017-03-16 09:21:43.686 00000e0c Connect: jcc.c end for (;;)
> content_length=168242 byte_count=23772
> 2017-03-16 09:21:43.764 00000e0c Connect: socket_is_still_usable:
> socket 424: no_data_waiting=1 status=1
> 2017-03-16 09:21:43.764 00000e0c Connect: Waiting for the next client
> request on socket 248. Keeping the server socket 424 to www.clubic.com
> open.
> 2017-03-16 09:21:43.779 00000e0c Connect: socket_is_still_usable:
> socket 248: no_data_waiting=0 status=1
> 2017-03-16 09:21:43.873 00000e0c Connect: Client request 2 arrived in
> time on socket 248.
> 2017-03-16 09:21:43.873 00000e0c Connect: Complete client request
> received.
> 2017-03-16 09:21:43.873 00000e0c Header: scan: GET
> http://www.clubic.com/assets/css/c06c6f0.css HTTP/1.1
>
>
> Lee
>
>
>>
>>
>> Le 2017-03-16 00:50, Lee a écrit :
>>> This looks strange:
>>> 2017-03-15 21:59:51.262 b6ceb460 Connect: Done reading from server.
>>> Content length: 23531 as expected. Bytes most recently read: 4319.
>>> 2017-03-15 21:59:52.517 b6ceb460 Header: Adjusted Content-Length to
>>> 168029
>>> 2017-03-15 21:59:52.517 b6ceb460 Header: Crunching: Content-Encoding:
>>> gzip
>>>
>>> privoxy received 23KB of gziped data and unzips it to 168KB -- which
>>> takes about 1.25 seconds!??
>>>
>>> Can you try adding
>>>
>>> { -filter }
>>> /
>>>
>>> to the end of your user.action and try it again?
>>>
>>> http://config.privoxy.org/user-manual/actions-file.html#FILTER
>>> Compressed content can't be filtered either, but if Privoxy is
>>> compiled with zlib support
>>> and a supported compression algorithm is used (gzip or deflate),
>>> Privoxy can first
>>> decompress the content and then filter it.
>>>
>>> Regards,
>>> Lee
>>>
>>>
>>> On 3/15/17, Phyks <phyks at phyks.me> wrote:
>>>> Hi,
>>>>
>>>> Framadrop works for me, weird. Anyway, I put it on a pastebin here:
>>>> http://paste.alacon.org/43566.
>>>>
>>>> I did not yet try to update Privoxy, I will look into it (this is
>>>> still
>>>> on 3.0.21).
>>>>
>>>> The pi is running its own Unbound DNS resolver. It should work fine,
>>>> and
>>>> typically if I make my laptop use it as well, I have DNS resolution
>>>> times comparable with 8.8.8.8.
>>>>
>>>> Thanks,
>>>> --
>>>> Phyks
>>>>
>>>> Le 15/03/2017 à 22:27, Ian Silvester a écrit :
>>>>> Hi Phyks,
>>>>>
>>>>> I could not retrieve the log excerpt stored at framadrop.org -
>>>>> could
>>>>> you
>>>>> check and reconfirm?
>>>>>
>>>>> That aside, could you also supply details of the Pi's DNS
>>>>> configuration
>>>>> - are the servers listed explicitly anywhere or is it using DHCP
>>>>> and
>>>>> fall back to those supplied by the LAN's router?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Ian
>>>>>
>>>>>
>>>>> My PGP public
>>>>> key[http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc]
>>>>>
>>>>> On Wed, 15 Mar 2017, at 17:07, Phyks wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I tried to debug it further. Here are some tests with the current
>>>>>> settings.
>>>>>>
>>>>>> Using Privoxy home page:
>>>>>> * Without Privoxy, the main HTML takes around 80ms to load.
>>>>>> * With Privoxy, it takes around 500ms to load, if I reload the
>>>>>> page
>>>>>> (even clearing the browser cache), it drops to about 80ms which is
>>>>>> in
>>>>>> agreement with the without Privoxy case.
>>>>>>
>>>>>> (Screenshots available at
>>>>>> https://lutim.phyks.me/gallery#jqNGoAnQ/86EWpbzr.png,Q23nsaOT/1XMA8IBi.png,
>>>>>> put online as I am unsure about the policy with attached images on
>>>>>> this
>>>>>> ML)
>>>>>>
>>>>>>
>>>>>> If I do the same test with http://clubic.com,
>>>>>> * Without Privoxy, the main HTML takes around 54ms to load.
>>>>>> * With Privoxy, it takes around 1.5s and this is reproducible even
>>>>>> if I
>>>>>> reload immediately the page.
>>>>>>
>>>>>>
>>>>>> Looking at htop in parallel, Privoxy is spawning multiple
>>>>>> processes
>>>>>> (about 10) and CPU load goes from 3% to around 100%, used by these
>>>>>> processes.
>>>>>>
>>>>>> The Raspberry Pi and my laptop are on the same local network, and
>>>>>> the
>>>>>> Raspberry Pi is wired with 100M ethernet to the router, so the
>>>>>> network
>>>>>> itself should be out of troubles.
>>>>>>
>>>>>>
>>>>>> Looking at the log as stated in the contact page
>>>>>> (https://framadrop.org/r/6j81o2D1m0#XNEfEVjRtqAXaHHI3YMRH3jtKRfxKdm7EADyfL/yEHk=),
>>>>>> it
>>>>>> seems that Privoxy is taking 1.5s to actually load the HTML
>>>>>> content
>>>>>> from
>>>>>> the remote server.
>>>>>>
>>>>>> However,
>>>>>>
>>>>>> ```
>>>>>> $ time curl http://www.clubic.com -s > /dev/null
>>>>>>
>>>>>> real 0m0.291s
>>>>>> user 0m0.120s
>>>>>> sys 0m0.030s
>>>>>> ```
>>>>>> so that it seems to be a problem related to Privoxy itself (and
>>>>>> not
>>>>>> due
>>>>>> to some network limitation on the Raspberry Pi or whatever like
>>>>>> this).
>>>>>>
>>>>>>
>>>>>> Thanks!
>>>>>> --
>>>>>> Phyks
>>>>>>
>>>>>> Le 14/03/2017 à 19:50, Fabian Keil a écrit :
>>>>>>> Phyks <phyks at phyks.me> wrote:
>>>>>>>
>>>>>>>> I installed Privoxy on Raspbian (latest, Debian derived, based
>>>>>>>> on
>>>>>>>> Jessie) through the repositories (version 3.0.21-7+deb8u1
>>>>>>>> installed).
>>>>>>>> Everything is running on a regular Raspberry Pi 1.
>>>>>>>
>>>>>>> Note that the current Privoxy version is 3.0.26 which means
>>>>>>> you are missing out on a bunch of changes including various
>>>>>>> performance improvements:
>>>>>>> http://ijbswa.cvs.sourceforge.net/viewvc/ijbswa/current/ChangeLog?view=markup
>>>>>>>
>>>>>>> Our current versioning scheme is a bit silly and we only bump
>>>>>>> the last number.
>>>>>>>
>>>>>>>> When I browse to any page (default configuration, no extra
>>>>>>>> actions
>>>>>>>> /
>>>>>>>> filter added yet), the CPU usage goes to 100% on the Raspberry
>>>>>>>> Pi,
>>>>>>>> and
>>>>>>>> it takes a few seconds to actually proxy the page, increasing
>>>>>>>> the
>>>>>>>> load
>>>>>>>> time from a few milliseconds to some seconds.
>>>>>>>
>>>>>>> That seems unreasonably slow to me, too.
>>>>>>>
>>>>>>>> I was wondering if anyone had experience with a similar setup
>>>>>>>> and
>>>>>>>> could
>>>>>>>> give me some input about the performances of the system. Also,
>>>>>>>> is
>>>>>>>> there
>>>>>>>> anything I could do to make it lighter?
>>>>>>>
>>>>>>> If you enable the debug directives recommended at:
>>>>>>> https://www.privoxy.org/user-manual/contact.html
>>>>>>> the log file should help to diagnose the issue.
>>>>>>>
>>>>>>> Please reproduce the problem with logging enabled and provide
>>>>>>> a log excerpt.
>>>>>>>
>>>>>>> Please also double check that the CPU usage is actually
>>>>>>> caused by Privoxy itself.
>>>>>>>
>>>>>>> Fabian
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Privoxy-users mailing list
>>>>>> Privoxy-users at lists.privoxy.org
>>>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>> _______________________________________________
>>>>> Privoxy-users mailing list
>>>>> Privoxy-users at lists.privoxy.org
>>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>>
>>>> _______________________________________________
>>>> Privoxy-users mailing list
>>>> Privoxy-users at lists.privoxy.org
>>>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>>>
>>
>> --
>> Phyks
>> _______________________________________________
>> Privoxy-users mailing list
>> Privoxy-users at lists.privoxy.org
>> https://lists.privoxy.org/mailman/listinfo/privoxy-users
>>
--
Phyks
More information about the Privoxy-users
mailing list