This is AdBlock's old support site. It is available for archival purposes only. To create a new discussion, or to visit the up-to-date knowledge base, please visit help.getadblock.com

Use a broader array of null content types for redirection of blocked URLs

James Edward Lewis II's Avatar

James Edward Lewis II

10 Sep, 2013 11:47 PM

As I worked out for Adblock Plus, you could redirect many URLs to various types of "null content" (valid files of minimal size for their type, with the proper Internet media types set): https://adblockplus.org/forum/viewtopic.php?f=10&t=7422&p=82629#p82629

To your credit, you already do this for images and frames, as Adblock Plus once did in its experimental build, and it is your source code that dissuaded me from trying to send "null JPEG" and "null GIF" (etc.) files instead of the null PNG that yours sends, to avoid conflicts with Ghostery.

In case you don't want to open that thread, suffice it to say that I used the null SWF file from the nullserv project, along with my homespun null WAV, MID, WEBM, MKV, FLV, MP4, and AVI files, and redirected scripts, stylesheets, and generic OBJECTS to data: URIs indicating the MIME type and having no content (like "data:text/css," where the comma *is* required); additionally, I have losslessly optimized the icons and allowed the main document to be blocked too, and the build itself is hosted here (I'll update it on occasion and base it on the latest stable version of AdBlock...I'd base it on an experimental build, the way I do for ABP, if I could find one): https://jansal.googlecode.com/svn/trunk/adblock/adblockforchrome-experimental.crx

I've gone ahead and attached the losslessly optimized icons (except delete.gif, which I optimized as delete.png, necessitating a change in another file in the extension).

I've been a long-time Adblock Plus user, switching to it from AdBlock after ABP implemented WebRequest a couple years ago, but I just might go back to AdBlock on a regular basis.

  1. 1 Posted by James Edward Le... on 11 Sep, 2013 12:06 AM

    James Edward Lewis II's Avatar

    Here are two short relevant diffs for my experimental swap-out build...

    for filtering/filteroptions.js
    22c22
    < ElementTypes.DEFAULTTYPES = 1023;
    ---
    > ElementTypes.DEFAULTTYPES = 2047;//now blocking main document

    for pages/resourceblock.html
    73c73
    < background-image: url(../img/delete.gif);
    ---
    > background-image: url(../img/delete.png);

    The one for background.js will need to be attached.

  2. 2 Posted by James Edward Le... on 11 Sep, 2013 01:07 AM

    James Edward Lewis II's Avatar

    Below I have attached the other relevant diff, along with the two diffs posted above, optimized images from the bundled jquery build, and the optimized delete.png file.

  3. 3 Posted by Tomáš on 15 Sep, 2013 05:31 PM

    Tomáš's Avatar

    Hi James Edward Lewis II,

    After testing on my computer some ads goes through and weren't blocked by AdBlock when I used your patches. Could you test it if all ads are blocked?

    Tomáš

  4. 4 Posted by James Edward Le... on 15 Sep, 2013 11:16 PM

    James Edward Lewis II's Avatar

    What filter-lists do you use, and what URLs did you try? I will try to compare stable AdBlock with this build using some set of URLs too.

  5. 5 Posted by James Edward Le... on 16 Sep, 2013 01:08 AM

    James Edward Lewis II's Avatar

    Okay, I just tested it with krebsonsecurity.com and found that when my ABP custom build is enabled, even if disabled on "this domain," neither swap-out build even hides ads, and when I tried just the AdBlock custom build, it hid a couple ads but failed to block the trackers, while AdBlock stable blocked both trackers.

    I didn't notice this before because my own content-blocking setup is so multi-layered, and I went through the trouble of peeling off all the layers to isolate the effect of AdBlock.

    I even made a variant that didn't try out that fancy regex-matching, and I can't seem to get anything to redirect to a blank application/octet-stream data URI.

    I just noticed that the "block resource" page doesn't mention any filter as being matched; maybe there's a problem if I try to pack this extension with any key other than Gundlach's (and maybe that's why there is no experimental build of AdBlock to begin with).

  6. 6 Posted by James Edward Le... on 16 Sep, 2013 04:44 AM

    James Edward Lewis II's Avatar

    Well I'll try more null-content redirection when or if this extension moves to declarativeWebRequest, but for now, while neither of my custom builds seems to work, I'm switching to AdBlock, which unlike ABP still uses at least *some* null-content redirection.

  7. 7 Posted by James Edward Le... on 26 Sep, 2013 03:39 AM

    James Edward Lewis II's Avatar

    Okay, I just tried again now that I saw both AdBlock Stable and ABP Dev updated, and I'm testing the background-page with some breakpoints and it looks like nothing ever gets considered "blocked" (that is, the problem shows up before the place where I made my changes); I'm not sure the problem is with my array of null-content redirections, but maybe there's something about the code that makes it fail to work unless I pack the extension with the same private key as Gundlach & Co.

    I'm still investigating why _myfilters.blocking.matches(), as used in Line 243 of background.js, always seems to return "false" in my experimental build but not in the official one, even though I didn't change any of the code that parses filters and determines whether a URL should be blocked and I use the same set of subscriptions and custom filters.

    EDIT: I'm not sure why, but when I just installed my custom build as an unpacked extension, it worked beautifully; I can't say the same for ABP, where I couldn't even get webrequest.js to load.

  8. 8 Posted by James Edward Le... on 26 Sep, 2013 04:40 AM

    James Edward Lewis II's Avatar

    As I said above, if you install my unofficial build of AdBlock, it doesn't block anything, but it seems if you unpack the CRX file and "Load unpacked extension" it works like my unofficial builds of ABP did when I first made them (but my unofficial builds of ABP have a more serious problem and do not work even when loaded unpacked); to help you test this out, I have attached a ZIP file of my latest unofficial build of AdBlock, based on the latest Stable build (2.6.8).

    I should mention, while looking at the documentation for the still-experimental declarativeWebRequest, I was glad to see built-in methods for redirecting to transparent images or blank pages, the very two things that AdBlock Stable already does by means of returning a redirectUrl; it's nice to know that even the Blink developers see the utility of returning null content instead of just cancelling the request.

  9. 9 Posted by Tomáš on 28 Sep, 2013 05:28 PM

    Tomáš's Avatar

    I will try this :)

  10. 10 Posted by Tomáš on 28 Sep, 2013 05:54 PM

    Tomáš's Avatar

    So, I can confirm that it is working like charm :) I will discuss this with Michael about implementing this into AdBlock.

    Thanks,
    Tomáš

  11. 11 Posted by Tomáš on 02 Oct, 2013 12:41 PM

    Tomáš's Avatar

    What is the value in using null objects instead of blocking requests? What does it make AdBlock do better or differently? Just a question :)

  12. 12 Posted by James Edward Le... on 04 Oct, 2013 05:15 AM

    James Edward Lewis II's Avatar

    It may be different now (I think now you get something like "This webpage was blocked by an extension"), but my thoughts for an expanded set of null-content redirections were formed almost 2 years ago: https://adblockplus.org/forum/viewtopic.php?p=56457#p56457

    Back when WebRequest was being tested, users noticed some garish error pages whenever content was blocked outright, so ways were found to substitute in such things as 1x1-pixel transparent .pngs, blank HTML pages, and the like so Chrome would have at least something to render...and I imagine (with more work) that a minimal MP4/FLV/WebM/MPEG video could be swapped in, but it would require much more sophisticated logic to ensure it doesn't break the plugin content.

    The idea that I had was that some websites and pieces of plugin content would assume an ad-blocker was running if a certain URL failed to load, but if at least something were returned, preferably something with the same content-type as was requested, some anti-adblock checks would be thwarted; I was hoping to make some video-players, like 4OD and Hulu, load some minimal "null MP4" or something so they'll think they're showing you an ad, rather than displaying the message "Sorry, we were unable to load a message from our sponsors" or something similar.

    As for the prevention of "garish error pages," that has mostly been done by redirecting blocked frames to about:blank and blocked images to a 1x1 transparent PNG.

  13. Support Staff 13 Posted by AdBlock on 07 Oct, 2013 03:36 PM

    AdBlock's Avatar

    James,

    Thanks for being interested in improving AdBlock.

    If your change could in fact stop a site from blocking AdBlock, I would be
    happy to incorporate it. I expect, however, that once a site has invested
    enough in their ant-AdBlock solution to be checking for blocked content,
    they'd be happy to respond to this change by instead checking the content
    itself. (Hulu in fact already does this -- in my tests some months back,
    any replacement video under a few seconds long didn't fool Hulu.)
     Basically, if they've started the arms race, then this would be an
    incremental change to AdBlock that they could incrementally respond to, so
    it's not worth implementing. Any arms race changes worth making on our end
    should make things truly difficult for the websites who want to detect
    AdBlock.

    Let me know if you think I'm mistaken.
    Michael

  14. 14 Posted by James Edward Le... on 08 Oct, 2013 11:41 AM

    James Edward Lewis II's Avatar

    (Hulu in fact already does this -- in my tests some months back, any replacement video under a few seconds long didn't fool Hulu.)

    I just got both of my experimental builds working (first ABP by fixing an unbalanced brace, then AdBlock...not sure why port.js was giving me "Port error: Could not establish connection. Receiving end does not exist"), and the first thing I tried to do was fool Hulu and I saw it wasn't working with my minimal MP4 file; I suspect a several-second-long MP4 file would be so large that AdBlock would no longer have a reasonable size.

    I've noticed that AdMuncher actually does handle Hulu well, cutting the countdown to 3 seconds, but it probably does something that a browser extension cannot do well.

    I wonder, however, whether the ability for the Hulu player to readily detect that the ad-content is too small relies on the fact that it's plugin content, while there might still be some benefit to redirecting a blocked Javascript URL to data:text/javascript, or a blocked XML URL to data:application/xml, instead of just blocking the requests, like it might be easy for a page to figure out that a file didn't load but not so easy to tell that it's not long enough.

    As I may have said before, at least you've picked the low-hanging fruit of aesthetically pleasing redirections: images to a 1x1 blank PNG, and frames to about:blank. I ought to test out whether it's worth it, from an aesthetic standpoint, to replace blocked SWF files with a minimal SWF, or a blocked HTML5 video with a minimal MP4 or WebM, even if it makes no difference to anti-adblock efforts.

    On a slightly related note, do you have any plans to make some sort of experimental build using the declarativeWebRequest API, as ABP did with WebRequest a couple years ago?
    I've been thinking about trying to do this myself, but I'm not so confident in my ability to do so; I've never even developed an extension from scratch.

    Thank you for your consideration.
    James

  15. AdBlock closed this discussion on 13 Nov, 2013 11:20 AM.

  16. James Edward Lewis II re-opened this discussion on 15 Nov, 2013 04:18 PM

  17. 15 Posted by James Edward Le... on 15 Nov, 2013 04:18 PM

    James Edward Lewis II's Avatar

    Just as an FYI to everyone watching this discussion, as of January this experimental build, which I am still maintaining, will not be able to be installed on Stable or Beta versions of Chrome, because it is outside the Chrome Web Store: https://groups.google.com/a/chromium.org/d/msg/chromium-extensions/...

    This is the relevant announcement: http://blog.chromium.org/2013/11/protecting-windows-users-from-mali...

    I might try to turn this into a full-fledged fork and host it on the Web Store, but honestly Gundlach already covered most of the good cases and I'm not sure it's even worth the $5 fee to keep this effort going.

  18. James Edward Lewis II closed this discussion on 15 Nov, 2013 04:19 PM.

Discussions are closed to public comments.
If you need help with AdBlock please start a new discussion.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac