[Privoxy-users] external-filter example

[U.Mutlu]

Lee wrote on 07/22/2020 09:29 PM:
> On 7/22/20, U.Mutlu <um at mutluit.com> wrote:
>> Update: sorry, had forgotten to build with "--enable-external-filters".
>> After rebuilding and reinstalling it now starts ok,
>> but the external-filter does not get invoked :-(
>> How to fix that?
>
> Have you added
> debug    64 # debug regular expression filters
> to the config file to see if privoxy tries to run the filter?
> https://www.privoxy.org/gitweb/?p=privoxy.git;a=blob;f=filters.c;h=b490353b2d93c5b1cc40ab80a22cc717372f6cda;hb=HEAD#l1834

Thx. Now tried with "debug 64" and it shows that it recognizes the said 
external-filter, but IMHO the action for it is not executed:

....
2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter "user-agent" 
("Tags the request with the complete User-Agent header.")
2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Adding re_filter job 
's@^User-Agent:.*@$0 at i' to filter user-agent succeeded.
2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter "referer" 
("Tags the request with the complete Referer header.")
2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Adding re_filter job 
's@^Referer:.*@$0 at i' to filter referer succeeded.
2020-07-22 22:31:57.821 7f838df67700 Re-Filter: Reading in filter 
"content-type" ("Tags the request with the content type declared by the server.")
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job 
's@^Content-Type:\s*([^;]+).*@$1 at i' to filter content-type succeeded.
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter 
"privoxy-control" ("Creates tags with the content of X-Privoxy-Control headers.")
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job 
's@^X-Privoxy-Control:\s*@@i' to filter privoxy-control succeeded.
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter 
"privoxy-control" ("Removes X-Privoxy-Control headers.")
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job 
's@^X-Privoxy-Control:.*@@i' to filter privoxy-control succeeded.
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter 
"privoxy-control" ("Creates tags with the content of X-Privoxy-Control headers.")
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job 
's@^X-Privoxy-Control:\s*@@i' to filter privoxy-control succeeded.
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter 
"privoxy-control" ("Removes X-Privoxy-Control headers.")
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Adding re_filter job 
's@^X-Privoxy-Control:.*@@i' to filter privoxy-control succeeded.
2020-07-22 22:31:57.822 7f838df67700 Re-Filter: Reading in filter "myTee" ("MY 
TEE TEST")
...

There is no other log entry regarding this "myTee" filter.
Ie. it seems the action for it does not get executed.

> If the filter isn't being called, maybe you need
> https://www.privoxy.org/user-manual/filter-file.html
>    Enabled content filters are applied to any content whose "Content
> Type" header is recognised as a sign of text-based content, with the
> exception of text/plain. Use the force-text-mode action to also filter
> other content.

But doing so seems to be dangerous according to the doc 
https://www.privoxy.org/user-manual/actions-file.html#FORCE-TEXT-MODE
It says:
"Warning - Think twice before activating this action. Filtering binary data 
with regular expressions can cause file damage."


I just want to get a copy of the whole HTML that privoxy gets from the server,
either the unmodified raw data, or the filtered version after
privoxy applies the other defined filters.

I think the HTML is always a text file, even if it contains binary data
like picture images etc. (they are somehow text encoded I think using
base64 encoding or so). Is this assumption wrong?

If I read the doc right then the external-filter handler has to write
the received data either verbatim (ie. no filtering), or a filtered
version of the data, back to stdout, so privoxy can continue doing its job.

A working example for such an external-filter would be helpful.
Unfortunately I couldn't find any on the web :-(