In-Browser filtering of raster pixels in Leaflet, Part 3

In my last posting, I mentioned described the process of creating a three-band TIFF and then slicing it into three-band PNG tiles for display in Leaflet. The tiles are not visually impressive, being almost wholly black, but they have the proper R, G, and B codes corresponding to the location's PHZ, PCP, and GEZ. (if you don't know what those are, go back and read that post from a few days ago)

So now the last step: transforming these tiles based on a set of arbitrary RGB codes determined by the user. Let's go through the working source code for the L.TileLayer.PixelFilter and I'll describe how it works.

First off, we need two RGBA codes so we can color the pixels as hit or miss, and we need the list of [r,g,b] trios that would be considered a match. The initialize method accepts these as constructor params, and the various set functions allow this to be set later as well.

(The decision of what list of RGB codes should be passed into setPixelCodes() is a matter of business logic: selectors, clicks, lists... outside the scope of this discussion. See the demo code for a trivial example.)

Next, we need to intercept the map tile when it's in the browser. Thus the
tileload event handler set up in the constructor, which runs the tile through
applyFiltersToTile() to do the real work.

applyFiltersToTile() is the interesting part, which only took me a couple of hours sitting down with some HTML5 Canvas tutorials. Let's dissect it one piece at a time:

• "copy the image data onto a canvas" The first paragraph creates a Canvas of the same width and height as the image, and copies the data into it. The IMG object itself won't let us access raw pixel data, but once it's in a Canvas we can.
• "target imagedata" We then use createImageData() to construct a new and empty byte sequence, which can later be assigned back into a Canvas using putImageData() This is effectively a straight list of bytes, and as we write to it, we need to always write 4 bytes at a time for every pixel: R, G, B, and A.
• "generate the list of integers" Before we start looping over pixels, a performance optimization. Arrays have the indexOf() method so we can determine whether a given value is in an array, but it only works on primitive values and not three-element arrays. The lodash library has findIndex() which would work, but it means a new dependency... and also the performance was not so great (256 x 256 = 65536 pixels per tile). So I cheat, and translate the list of desired
  pixelCodes into a simple list of integers so that indexOf() can work after all.
• "iterate over the pixels" Now we start looping over the pixels in our Canvas and doing the actual substitution. Again, we step over the bytes in fours (R, G, B, A) and we would assign them into the output imagedata in fours as well. Each pixel is translated into an integer, and if that integer appears in the pixelcodes list of integers, it's a hit. Since indexOf() is a native function it's pretty swift, and since our pixelcodes list tends to be very short (10-50 values) the usually-avoided loop-in-loop is actually quite fast.
• "push a R, a G, and a B" Whatever the results, we push a new R, G, B, A set of 4 bytes onto the output imagedata.
•  "write the image back" And here we go: write the imagedata sequence back into the Canvas to replace the old data, then reassign the img.src to this Canvas's base64 representation of the newly-created imagedata. Now the visible tile isn't based on a HTTP file at all, but from a base64-encoded image in memory.

The only gotcha I found, was that upon calling setPixelCodes() the tiles would flash to their proper color and then instantly all turn into the no-match color. It worked... then would un-work itself?

The tileload event wraps the IMG element's load event. This means that when I assigned the img.src to replace the tile's visible representation... this was itself a tileload event! It would call applyFiltersToTile() again, and this time the RGB codes didn't match anything on my filters so all pixels were no-match. Worse, the assignment of img.src was again a tileload event, so it was in an infinite loop of processing the tile.

Thus the already_pixel_swapped flag. After processing the IMG element, this flag is set and
applyFiltersToTile() will skip out on subsequent runs on that IMG element. If we need to change pixel codes via setPixelCodes() that calls layer.redraw() which empties out all these old IMG elements anyway, replacing them with fresh new ones that do not have already_pixel_swapped set.

So yeah, it was an educational day. Not only did we achieve what the client asked for (the more complex version, not the simplified case I presented last week) and as a neat reusable package, but the performance is near instant.