[Privoxy-users] external-filter example

[U.Mutlu]

It's funny that nothing of default.action, user.action, and my.action
applies to the below URL [I'm using the default configs it has installed].
(my.action and my.filter is just my experimental stuff with the said "myTee"
external-filter (now moved from user.* to my.*)):

http://config.privoxy.org/show-url-info?url=www.startpage.com

Matches for http://www.startpage.com:
In file: match-all.action View
{+change-x-forwarded-for{block}
+client-header-tagger{css-requests}
+client-header-tagger{image-requests}
+client-header-tagger{range-requests}
+hide-from-header{block}
+set-image-blocker{pattern} }
/
In file: default.action View
(no matches in this file)
In file: user.action View
(no matches in this file)
In file: my.action View
(no matches in this file)



U.Mutlu wrote on 07/22/2020 11:06 PM:
> Lee wrote on 07/22/2020 09:29 PM:
>> On 7/22/20, U.Mutlu <um at mutluit.com> wrote:
>>> Update: sorry, had forgotten to build with "--enable-external-filters".
>>> After rebuilding and reinstalling it now starts ok,
>>> but the external-filter does not get invoked :-(
>>> How to fix that?
>>
>> Have you added
>> debug    64 # debug regular expression filters
>> to the config file to see if privoxy tries to run the filter?
>> https://www.privoxy.org/gitweb/?p=privoxy.git;a=blob;f=filters.c;h=b490353b2d93c5b1cc40ab80a22cc717372f6cda;hb=HEAD#l1834
>>
>
> Thx. Now tried with "debug 64" and it shows that it recognizes the said
> external-filter, but IMHO the action for it is not executed:
>
> ....
> 2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter "user-agent"
> ("Tags the request with the complete User-Agent header.")
> 2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Adding re_filter job
> 's@^User-Agent:.*@$0 at i' to filter user-agent succeeded.
> 2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter "referer"
> ("Tags the request with the complete Referer header.")
> 2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Adding re_filter job
> 's@^Referer:.*@$0 at i' to filter referer succeeded.
> 2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter
> "content-type" ("Tags the request with the content type declared by the server.")
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job
> 's@^Content-Type:\s*([^;]+).*@$1 at i' to filter content-type succeeded.
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter
> "privoxy-control" ("Creates tags with the content of X-Privoxy-Control headers.")
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job
> 's@^X-Privoxy-Control:\s*@@i' to filter privoxy-control succeeded.
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter
> "privoxy-control" ("Removes X-Privoxy-Control headers.")
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job
> 's@^X-Privoxy-Control:.*@@i' to filter privoxy-control succeeded.
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter
> "privoxy-control" ("Creates tags with the content of X-Privoxy-Control headers.")
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job
> 's@^X-Privoxy-Control:\s*@@i' to filter privoxy-control succeeded.
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter
> "privoxy-control" ("Removes X-Privoxy-Control headers.")
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job
> 's@^X-Privoxy-Control:.*@@i' to filter privoxy-control succeeded.
> 2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter "myTee" ("MY
> TEE TEST")
> ...
>
> There is no other log entry regarding this "myTee" filter.
> Ie. it seems the action for it does not get executed.
>
>> If the filter isn't being called, maybe you need
>> https://www.privoxy.org/user-manual/filter-file.html
>>    Enabled content filters are applied to any content whose "Content
>> Type" header is recognised as a sign of text-based content, with the
>> exception of text/plain. Use the force-text-mode action to also filter
>> other content.
>
> But doing so seems to be dangerous according to the doc
> https://www.privoxy.org/user-manual/actions-file.html#FORCE-TEXT-MODE
> It says:
> "Warning - Think twice before activating this action. Filtering binary data
> with regular expressions can cause file damage."
>
>
> I just want to get a copy of the whole HTML that privoxy gets from the server,
> either the unmodified raw data, or the filtered version after
> privoxy applies the other defined filters.
>
> I think the HTML is always a text file, even if it contains binary data
> like picture images etc. (they are somehow text encoded I think using
> base64 encoding or so). Is this assumption wrong?
>
> If I read the doc right then the external-filter handler has to write
> the received data either verbatim (ie. no filtering), or a filtered
> version of the data, back to stdout, so privoxy can continue doing its job.
>
> A working example for such an external-filter would be helpful.
> Unfortunately I couldn't find any on the web :-(