tag:blogger.com,1999:blog-59322958476986888082024-03-05T13:24:16.156-08:00Gregor The Map GuyGregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.comBlogger117125tag:blogger.com,1999:blog-5932295847698688808.post-55386977165922906132017-11-13T16:02:00.000-08:002017-11-13T16:02:02.365-08:00QGIS: Adding Bounding Box Ordinates as Shapefile AttributesI mentioned last week, that for the purpose of zooming to a hydrology region when it's selected, we added some attributes to a CSV of metadata, which we were already loading for other UI and statistical purposes.<br />
<br />
These attributes were called <span style="font-family: Courier New, Courier, monospace;">bbox_s, bbox_w, bbox_n, and bbox_e</span> and are simply floats in WGS84 lat-long. But I thought I'd share how I generated 4 fields representing the bounding box, for these regions (and then later for a few hundred individual watersheds) quickly and easily.<br />
<br />
To back up a bit:<br />
<br />
<ul>
<li>The CSV was generated from the DBF component of that hydrology regions shapefile. Excel can load up the DBF just fine, then you use <i>Save As</i> and export as a CSV. Easy.</li>
<li>The shapefile has attributes of statistics which were generated by other means. The stats were miles of river, number of gauges, that sort of deal where GIS was the answer.</li>
<li>The only missing attributes now are the bounding box fields, preferably 4 floats in WGS84 lat-long.</li>
</ul>
<div>
This was done with QGIS's Field Calculator, using built-in functionality. The Field Calculator supports geometry operations including <i>xmin</i> and <i>transform</i>. So it's simple:</div>
<div>
<br /></div>
<div>
Take note of what SRS you are using, particularly its EPSG code.</div>
<div>
<br /></div>
<div>
Create a new field called <i>bbox_s</i> of type float. <i>Don't forget to give it a Precision for after the decimal place!</i> If you leave Precision at 0, the fields will still calculate but will then be truncated to whole numbers after you save.</div>
<div>
<br /></div>
<div>
Populate it with this formula, being sure to replace the first SRS with your dataset's own SRS:</div>
<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">y_min(transform($geometry, 'EPSG:3310', 'EPSG:4326'))</span></blockquote>
<br />
<div>
And there you are! You now have an attribute with the southern ordinate of the bbox. Repeat this 3 more times:</div>
<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bbox_w<br />x_min(transform($geometry, 'EPSG:3310', 'EPSG:4326'))</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bbox_e<br />x_max(transform($geometry, 'EPSG:3310', 'EPSG:4326'))</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bbox_s<br />y_min(transform($geometry, 'EPSG:3310', 'EPSG:4326'))</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bbox_n<br />y_max(transform($geometry, 'EPSG:3310', 'EPSG:4326'))</span></blockquote>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
</div>
<div>
Save your edits, and you're nearly done. Load up the DBF in Excel, maybe tidy up some other fields and their formatting, and export as CSV.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
</div>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-26103173078876102522017-11-06T15:25:00.000-08:002017-11-06T15:25:04.975-08:00More Mapbox! Using querySourceFeatures() ... or notThe project where I've been exploring Mapbox GL API continues along, and everyone is quite pleased with how it's coming along. Here's another technical posting, this time about something we needed which <b>did not</b> work out.<br />
<br />
The need: One of the map layers is hydrology regions. When one is selected, it would be great to highlight it and also zoom the map to it.<br />
<br />
Highlighting is already done and working just fine. I went with a filtering technique to swap between a choropleth of all regions, and a highlight of the one region. That's great.<br />
<br />
Zooming though, implies that we have the bounding box coordinates handy for the selected region. Can we get these from the API, since it has the records? <b>Answer: No.</b><br />
<br />
<h3>
querySourceFeatures Does Not Query</h3>
<br />
Mapbox API does provide a method that sounds promising: <span style="font-family: "courier new" , "courier" , monospace;">querySourceFeatures()</span> But it doesn't do what you're thinking. <b>It queries only the vector tiles which are currently visible in the map viewport.</b> As such, it can only query features that are already on the map and <i>is not</i> a search/query mechanism for your dataset.<br />
<br />
Example code:<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">// specify the URL of the tileset when declaring the Source</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">// and ALSO its suffixed name when connecting it to a Layer</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">map.addSource('myplaces', {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "type": "vector",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "url": "mapbox://mapbox.abcde12345"</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">});</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">map.addLayer({</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "id": "places",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "source": "myplaces",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "source-layer": "MyPlaces-abc123",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "type": "fill",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "paint": {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "fill-outline-color": "rgba(0,0,0,0)",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "fill-color": "rgba(0,0,0,0)"</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">});</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">var results = map.querySourceFeatures('myplaces', {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> sourceLayer: 'places',</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> filter: [ '==', 'name', 'Montana' ],</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">});</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">console.log(results);</span></blockquote>
</blockquote>
<br />
Limitations and expectations are:<br />
<br />
<ul>
<li>The setup is a bit goofy, that you must program in <i>both</i> the URL of your Mapbox tileset (when creating the Source) and also the readable suffixed name of it (when adding the Layer).</li>
<li>The Source isn't actually useful unless there is also a Layer and the Layer is on the map. You can render with 0 alpha to make it invisible, but it must be on the map.</li>
<li>The query only searches within the vector tiles currently visible on the map. As you pan and zoom, you'll see that <span style="font-family: "courier new" , "courier" , monospace;">results</span> changes even if your query does not.</li>
</ul>
<div>
The use case demonstrated by the Mapbox demos, is basically limited to querying what's under your mouse. And sadly, this does seem to be all that it's good for. <span style="font-family: "courier new" , "courier" , monospace;">querySourceFeatures()</span> is basically the same thing as <span style="font-family: "courier new" , "courier" , monospace;">queryRenderedFeatures()</span>except that you can specify the "click filtering" without changing the filters used for rendering.</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
So, What Now?</h3>
<div>
<br /></div>
<div>
Mapbox offers a fairly new service called Datasets. When you upload your dataset, unlike Tilesets which are digested into vector tiles, this one is digested into a database and exposed as a GeoJSON structure. This can then be queried and downloaded, and there's mention of an editing and authoring environment as well.</div>
<div>
<br /></div>
<div>
Of course, this means uploading a second copy of these datasets just for querying purposes, which isn't so great for maintainability.</div>
<div>
<br /></div>
<div>
In our case, we were already loading a CSV of metadata used for constructing other UI elements and populating some statistical tables. As such, it was no problem to add to that CSV some new <i>bbox_s, bbox_e, bbox_n, bbox_e</i> fields to facilitate the zooming behavior. As such, I did not get an opportunity to work with Datasets this time.</div>
<div>
<br /></div>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-79516977786766311392017-10-30T13:08:00.000-07:002017-10-30T13:08:00.282-07:00Mapbox Vector Tiles + Leaflet: Bugs and WorkaroundsA few weeks ago, I posted about my first experiences with Mapbox's vector tile service and the Leaflet adapter for it. A lot of things went well and the end result was quite nice, and I covered a lot of how-to items for what worked well.<br />
<br />
But a couple of items were buggy, or needed workarounds.<br />
<br />
<h3>
Published Changes Not Saving</h3>
<div>
<br /></div>
I mentioned this previously, but continue to replicate it. I make changes to my Style in Mapbox Studio, hit Publish, then wait patiently for the changes to show up in the browser. Sometimes it works almost instantly, sometimes I have to wait 10 minutes, and sometimes it just doesn't happen.<br />
<br />
I hit the <span style="font-family: "courier new" , "courier" , monospace;">/styles/v1/</span> endpoint in my debugger, and see that they really are serving up the old colors, are not yet including my new layer, are still showing the old name for the layer, and so on.<br />
<br />
<i>Workaround: go into Mapbox Studio again, make some change and then undo it, and hit Publish again. Sometimes a third time is required.</i><br />
<br />
<h3>
Clicking and mouseover effects</h3>
<div>
<br /></div>
The <a href="https://www.mapbox.com/mapbox-gl-js/api/#map#queryrenderedfeatures" target="_blank"><span style="font-family: "courier new" , "courier" , monospace;">queryRenderedFeatures()</span></a> method accepts a Mapbox GL Point as the query coordinates, not a L.LatLng since it's not Leaflet. The programming to convert a <span style="font-family: "courier new" , "courier" , monospace;">L.Map click</span> event into a Mapbox GL Point and then perform a query, is as follows:<br />
<blockquote class="tr_bq">
<span style="font-family: courier new, courier, monospace; font-size: x-small;">MAP.on('click', function (e) {</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> const canvas = MBOVERLAY._glMap.getCanvasContainer();</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> const rect = canvas.getBoundingClientRect();</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> const glpoint = new mapboxgl.Point(e.originalEvent.clientX - rect.left - canvas.clientLeft, e.originalEvent.clientY - rect.top - canvas.clientTop);</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> const features = MBOVERLAY._glMap.queryRenderedFeatures(glpoint, { layers: INTERACTION_LAYERS });</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> // now go do something with the resulting "features" list</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;">});</span><br />
<div>
<br /></div>
</blockquote>
<h3>
Maps becoming desynchronized on a zoom change</h3>
<div>
<br /></div>
When the <span style="font-family: "courier new" , "courier" , monospace;">L.Map</span> changes zoom level, e.g. any time we call a <span style="font-family: "courier new" , "courier" , monospace;">fitBounds()</span> or <span style="font-family: "courier new" , "courier" , monospace;">setView()</span> the Mapbox GL layer would fall out of sync. The vector layer would repaint, but often not at the same center, so would be offset from the rest of the map. This would happen pretty reliably too, with only moments of testing required.<br />
<br />
The workaround here was a bit of a hack: any time the map becomes idle after changing, have it call setCenter() on its own center after a brief timeout. This seems to trigger a repaint of the Mapbox GL layer, bringing it into sync with the rest of the map.<br />
<blockquote class="tr_bq">
<span style="font-family: courier new, courier, monospace; font-size: x-small;">MAP.on('moveend', function () {</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> if (! MAP.overlay._glMap) return; // map's not ready yet e.g. startup</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> const center = MAP.getCenter();</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> setTimeout(function () {</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> MAP.overlay._glMap.setCenter(center);</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> }, 250);</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;">});</span></blockquote>
<div>
Again, an ugly hack but a lot less ugly than what it solves!</div>
<br />
<h3>
Race conditions on startup</h3>
<div>
<br /></div>
Outside of the map is a list of hydrology regions and clicking one will trigger some behaviors on the map. Some of these behaviors include changing some filters on the Mapbox GL layer. Problem is, the Mapbox GL layer can take several seconds to initialize. If someone clicks during this time, all sorts of funky hell breaks loose in the console because <span style="font-family: "courier new" , "courier" , monospace;">MBOVERLAY._glMap</span> is not yet defined, the Style has not yet loaded, and so on.<br />
<br />
The workaround is not surprising: GL Map's <span style="font-family: "courier new" , "courier" , monospace;">load</span> event, which fires after the map has first loaded and completed its first painting of the layer. Now you know that <span style="font-family: "courier new" , "courier" , monospace;">MBOVERLAY._glMap</span> is defined and ready!<br />
<blockquote class="tr_bq">
<span style="font-family: courier new, courier, monospace; font-size: x-small;">MAP.overlay = L.mapboxGL({</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> accessToken: OVERLAY_ACCESSTOKEN,</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> style: OVERLAY_MAPBOXURL,</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> pane: 'overlayPane',</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;">}).addTo(MAP); </span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: courier new, courier, monospace; font-size: x-small;">MBOVERLAY._glMap.on('load', function () {</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> // addSource() and addLayer()</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;"> // and additional UI event handlers for folks who won't wait before clicking</span><br />
<span style="font-family: courier new, courier, monospace; font-size: x-small;">});</span></blockquote>
<h3>
What Went Well?</h3>
<div>
<br /></div>
<div>
Most things worked well! Those were the focus of the previous articles. :)<br />
<br />
Hooray Mapbox, for a really excellent product which allows for filtering and re-styling in ways that would traditionally have required a WMS server.<br />
<br /></div>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-57855920575364717642017-10-24T15:23:00.000-07:002017-10-24T15:23:07.132-07:00Leaflet-Mapbox-GL HOWTO: Highlighting an areaMy series on Mapbox GL API combined with Leaflet using <a href="http://leaflet-mapbox-gl.js/">leaflet-mapbox-gl.js</a> continues.<br />
<br />
We have been working within a function called <span style="font-family: Courier New, Courier, monospace;">selectRegionByName()</span> which would switch between choropleth view and borders-only view by way of filters. Last week we added waterways and river gauges to it as well, again using filters.<br />
<br />
This week's icing on the cake: we want to highlight the selected region as well. The black borders are showing and the choropleth is suppressed, but can we give this area a good thick orange border to make it pop?<br />
<br />
<br />
<h2>
The Easy Way: Filtering a Dedicated Layer</h2>
<br />
No surprise here: the easiest mechanism would be to add a new layer to your Style, and filter it to match the region. If we stack the highlight layer above the black outline layer and give the stroke an opacity of 1, this would effectively hide the black border of this region in favor of the orange border.<br />
<br />
<ul>
<li>Go into Mapbox Studio and add a new layer called <i>regionhighlight</i>.<br />Use the hydrology regions as the Tileset.<br />Select a line type rendering, give it a good thick stroke and make it orange.</li>
<li>Set a filter on the layer to match `region == Nonexistent` so that it's not in fact matching anything at all.</li>
<li>In client-side code, follow what we did the last few postings to add a new filter for <i>regionhighlight</i> matching the selected region name.</li>
</ul>
<div>
The resulting additions to <span style="font-family: Courier New, Courier, monospace;">selectRegionByName()</span> would look a lot like this:</div>
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// highlight overlay on</span><span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">: filter to match this one region</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter("highlightregion", ["==", 'name', name]);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// highlight overlay off: </span><span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">filter to match no region ever</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter("highlightregion", ["==", 'name', "Nonexistent"]);</span></blockquote>
</blockquote>
<br />
<h2>
The Harder Way: Creating a New Layer Client-Side</h2>
<br />
Back when I implemented the highlighting, I hadn't quite wrapped my head around filtering and hadn't come up with the easy method above. Instead, I used a technique described in Mapbox's documentation which uses a Tileset as a data source and adds virtual layers to the client-side map.<br />
<br />
First off, define a Source connected to your Tileset. You will need the Tileset's Mapbox URL and the name of it as it appears in your Tilesets page.<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// define a new Source</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// url is the Mapbox URL of the regions Tileset</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.addSource('regions', {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "type": "vector",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "url": "mapbox://yourusername.abcdef1234",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">});</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// define a line layer using the above Source</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// source is the Source name you defined above</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// source-layer is the Mapbox-mangled name of that Tileset</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.addLayer({</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "id": "highlightregion",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "source": "regions",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "source-layer": "HydroRegions-abcd1234",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "type": "line",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "paint": {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "line-width": 4,</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "line-color": "orange",</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> },</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "filter": ["==", 'name', "Nonexistent"],</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">});</span></blockquote>
</blockquote>
Now, to activate that virtual layer, you would apply a filter:<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// highlight overlay on</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// filter to match this one region</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter("highlightregion", ["==", 'name', name]);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// highlight overlay off</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// filter to match no region ever</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter("highlightregion", ["==", 'name', "Nonexistent"]);</span></blockquote>
</blockquote>
This technique is a lot more complicated, and since you need to use filtering anyway in order to operate it, the first technique I described (a dedicated highlighting layer) gets the job done with less work. But if you are using someone else's Style and editing it is not an option, this technique would allow you to accomplish much the same effect.<br />
<br />
<br />
<h2>
Stroke vs Fill</h2>
<div>
One last note about highlighting: stroke versus fill. Mapbox layers are either polygons (a fill, with no stroke) or else lines (a boundary but no fill). If you want to highlight with both, you will need to define two layers, then set the filtering on both of them.</div>
<div>
<br /></div>
<div>
This goes for both doing highlights within the Mapbox Studio Style, or using the <span style="font-family: Courier New, Courier, monospace;">addLayer()</span> technique.</div>
<div>
<br /></div>
<div>
<br /></div>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-83972232222249508222017-10-19T14:42:00.000-07:002017-10-19T14:42:06.990-07:00Leaflet-Mapbox-GL HOWTO: Filtering features on the mapMy series on Mapbox GL API combined with Leaflet using <a href="http://leaflet-mapbox-gl.js/">leaflet-mapbox-gl.js</a> continues.<br />
<br />
Previously we created a function <span style="font-family: Courier New, Courier, monospace;">selectRegionByName()</span> which would switch between a choropleth view and a plain-outline view, and zoom the map to a given region. Let's expand on that:<br />
<br />
<ul>
<li>When we zoom in to a region, we toggle between choropleth and simple outline, and also zoom to the given area.</li>
<li>Now we <i>also</i> want to show the waterways and the gauges within that region, but not in other regions so as to minimize the visual noise.</li>
</ul>
<div>
<br /></div>
<h2>
Filtering Is Easy</h2>
<div>
Quite simply, this is done with <span style="font-family: Courier New, Courier, monospace;">setFilter()</span> which you've already seen. In our case, the gauge points and waterway lines had already been tagged with a <i>region</i> attribute so filtering is quite easy. You should be able to patch these into <span style="font-family: Courier New, Courier, monospace;">selectRegionByName()</span> with no difficulty.</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// region selected</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// filter gauges and waterways layers, to match this region</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('gauges', ["==", 'region', name]);</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('waterways', ["==", 'region', name]);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// null selected</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// filter gauges and waterways layers, to match no region at all</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('gauges', ["==", 'region', 'Nonexistent']);</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('waterways', ["==", 'region', 'Nonexistent']);</span></blockquote>
</div>
<div>
<br /></div>
<div>
There's really not a lot to say about it. When an area is selected, set the filters to match only features in that region. When no area is selected, set an impossible filter.</div>
<div>
<br /></div>
<h2>
New Filters Replace Old Filters</h2>
<div>
The real caveat here is that <b><i>these filters replace your own filtering in the Style</i></b>. If the gauge locations were already filtered in your Style, to show only those with "Status = Operational" then guess what? You just blew away that filter in favor of this new one: it's showing all gauges in the region regardless of their Status field.</div>
<div>
<br /></div>
<div>
This can mean some duplication of effort, adding the filter in your Style via Studio, then remembering to replicate that filtering here in the client side code in addition to the new region filter. An alternative, would be to use <span style="font-family: Courier New, Courier, monospace;">getFilter()</span> to fetch the current set of filters, then add/remove the ones relevant to the region.</div>
<div>
<br /></div>
<div>
getFilter() returns the list of the filters currently applied to the layer. <span style="font-family: Courier New, Courier, monospace;">MBOVERLAY._glMap.getFilter("gauges")</span> would return something like this, indicating a filter that matches ALL of the following filters, a filter clause by region and a filter clause by status.</div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[ "all", ['==', 'region', 'Northeastern Watershed'], ['==', 'Status', 'Operational'] ]</span></blockquote>
<div>
What we would want here, is to remove the <i>region</i> filter and leave all the rest, including that <i>all</i> at the start and any other clauses that aren't for <i>region</i>. And here we are:</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// add a region filter</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// to whatever other filters were in place</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">var filters = MBOVERLAY._glMap.getFilter("gauges");</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">filters.push([ '==', 'region', name ]);</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('gauges', newfilters);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// clear the region filter</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">// but re-apply any other filters previously in place</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">var filters = MBOVERLAY._glMap.getFilter("gauges");</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">filters = filters.filter( (f) => { return f[1] != 'region' });</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MBOVERLAY._glMap.setFilter('gauges', filters);</span></blockquote>
</div>
<div>
This technique is a bit more work to set up, compared to the first example which is copy-paste in 20 seconds. But if you are already using filtering on your layers, this technique would preserve the filters already defined in the Style while also allowing you to define them client-side.</div>
<div>
<br /></div>
<div>
Of course, it's all dependent on your use case. If your filtering needs are fairly complex, or you're creating a whole filtering UI anyway, maybe you'd do better to define all your filters client side and keep them all in one place. </div>
<div>
<br /></div>
<div>
<br /></div>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-64243293404927025972017-10-16T14:25:00.000-07:002017-10-16T14:25:04.444-07:00Leaflet-Mapbox-GL HOWTO: Toggle between choropleth and boundariesMy series on Mapbox GL API combined with Leaflet using <a href="http://leaflet-mapbox-gl.js/">leaflet-mapbox-gl.js</a> continues.<br />
<br />
One of the first interesting tasks beyond just seeing the vector tiles on screen, would be to select one of those hydrology regions, then focus the map on it. When no area is selected, a statewide choropleth should indicate some statistical information about each region.<br />
<ul>
<li>One of the datasets (I mean Tilesets) loaded into Mapbox is of the statewide hydrology regions. They each have a statistical field indicating their percentage completeness, and we will want a choropleth representation of it with some known breakpoints.</li>
<li>When an area is selected, we want to hide the choropleth and zoom in to that area, and switch over to a second rendition of those polygons: simple thick lines. Thus the choropleth color won't be distracting now that we're looking at a detailed view.</li>
<li>An area could be de-selected entirely, at which time we should switch back to choropleth, zoom back to statewide extent, and hide the lines.</li>
<li>The bounding box extents for each area, are present here in the client-side code: region name, west, south, north, east. So no special tricks are required there.</li>
</ul>
<br />
<h2>
Adding Region Polygons to the Style</h2>
<br />
<ul>
<li>First step of course, was to upload the Tileset into Mapbox. Done.</li>
<li>Second, style up the two versions: a plain thick black outline when zoomed in, and a choropleth when zoomed out.</li>
<li>Solid black is pretty simple. Just click the layer name, fill in a color. Done.</li>
<li>Choropleth is more tedious but not too tough: Select the layer, and for the color hit the expander and select <i>Enable Property Function and select a field</i>. In my case I want to filter by value and I have category breaks already defined: I select <i>#</i> as the data type and <i>Categorical</i> as the classification. I then used "Add Stop" and entered the max values for each break, one by one.</li>
</ul>
<div>
And there we have it, a choropleth based on the completion percentage <i>and</i> a plain black outline. Both are visible in Mapbox Studio this whole time, but on the client side we're going to change that.</div>
<br />
<br />
<h2>
Toggling by Region Name</h2>
<br />
I wrote a function which accepts a region name, to zoom to it and highlight it and all. It also accepts <span style="font-family: "courier new" , "courier" , monospace;">null</span> in order to zoom to the statewide view, selecting no area at all and adjusting the highlights accordingly.<br />
<br />
We will expand on this in the coming days. Here's our starting content:<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">function selectRegionByName (name) {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> if (name) {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // zoom the map to this region's already-known info</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> const rawdata = REGION_INFO[name];</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MAP.fitBounds([[ rawdata.BBOX_S, rawdata.BBOX_W ], [ rawdata.BBOX_N, rawdata.BBOX_E ]]); </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // suppress the choropleth by setting a filter that matches nothing</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // enable the black borders by not filtering</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MBOVERLAY._glMap.setFilter('waterregions_choro', ['==', 'name', '']);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MBOVERLAY._glMap.setFilter('waterregions_black', null]);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> else {</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // zoom the map to the fixed bounds of the whole state</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MAP.fitBounds(WHOLESTATE_BOUNDS);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // allow the choropleth to show, by not filtering it</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> // hide the black outlines by impossible filtering</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MBOVERLAY._glMap.setFilter('waterregions_choro', null);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> MBOVERLAY._glMap.setFilter('waterregions_black', [ '==', 'name', '']);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">}</span></blockquote>
</blockquote>
<div>
The trick to toggling the layers is easy: to hide a layer, apply to it a filter which matches nothing; to show a layer, clear the filters.</div>
<div>
<br /></div>
<div>
It's simple now, but we will expand on this in the next few postings, so that <span style="font-family: "courier new" , "courier" , monospace;">selectRegionByName()</span> does some more interesting behaviors.</div>
<div>
<br /></div>
<h2>
Other Approaches to Toggling</h2>
<br />
To make the layer invisible, we could have tried a few other mechanisms. Which one works best for you, will vary by your use case. Most often, it's a case of "how would this strategy conflict with the existing map style, and cause unintended side effects?"<br />
<br />
<h3>
Filtering</h3>
<br />
For us, this layer has no filters at all. We either want to show them all or show none of them. As such, doing it by filter had few questions, and no likelihood of conflicting with our styles or layer stacking.<br />
<br />
If you do filter your dataset to form a layer, the <span style="font-family: "courier new" , "courier" , monospace;">setFilter()</span> route may not be for you. That use of <span style="font-family: "courier new" , "courier" , monospace;">setFilter()</span> as shown above really does change the layer, so removes all of your filters in favor of your new filters!<br />
<div>
<br /></div>
<h3>
addLayer() and removeLayer()</h3>
<br />
Use <span style="font-family: "courier new" , "courier" , monospace;">MAP._glMap.getLayer()</span> to stow a reference to the choropleth layer and the black-outline layer, then call <span style="font-family: "courier new" , "courier" , monospace;">MAP._glMap.addLayer()</span> and <span style="font-family: "courier new" , "courier" , monospace;">MAP._glMap.removeLayer()</span> as needed. It works, though is a bit more clunky in my opinion.<br />
<br />
Mapbox GL JS API does not support a z-index on layers, so the layer order when they are added back into the map, may not be the same as it was previously. They support a <i>before</i> parameter to help with the stacking order, which may work for you as long as you're not renaming layers nor changing their sequence via Mapbox Studio.<br />
<br />
<h3>
Layer Opacity</h3>
<br />
Another approach would be to alter the layer's color style via <span style="font-family: "courier new" , "courier" , monospace;">setPaintProperty()</span> so as to give it 0 opacity.<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">MBOVERLAY.setPaintProperty('waterregions_choro', 'fill-opacity', 0);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">MBOVERLAY.setPaintProperty('waterregions_choro', 'line-opacity', 0);</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">MBOVERLAY.setPaintProperty('waterregions_black', 'line-opacity', 1);</span></blockquote>
This can work well, as long as you remember to address both the stroke and the fill for all appropriate layers, <i>and</i> as long as you know what opacity would be "the default" when making it visible again.<br />
<br />
<h3>
</h3>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-59754015161862360142017-10-12T13:03:00.002-07:002017-10-12T14:31:56.601-07:00Vector Tiles with Mapbox + LeafletA couple of weeks back, I finally got to really dig in on Mapbox's implementation of vector tiles, and using vector tiles in Leaflet.<br />
<br />
Myself, I almost never get to use Mapbox Studio as the cartography work usually goes to one of my coworkers who has a good eye for that stuff. We're also coming in to vector tiles a bit late, having gotten a green-light that supporting older browsers (IE 8 on XP) and operating systems (Android 4.1) was not a requirement and that a few hours of R&D would be worth the very slick interface behaviors promised by vector tiles. As such, this was a learning experience all around.<br />
<br />
The result was a map that is really quite slick, involving behaviors which traditionally would have required a server deployment and a WMS server, and with an outcome more slick than would have been achieved with those methods anyway. Best of all, it stayed well within Mapbox's generous free tier.<br />
<br />
<h3>
The Application</h3>
<div>
<ul>
<li>Rivers and waterways (line), about 50,000 records spanning the entire state, about 100 MB in size. It's not the whole state, just the waterways considered interesting.</li>
<li>Several thousand stream gauge locations (point).</li>
<li>Ten defined hydrology regions (polygons) covering the state. These are to be displayed as plain black outlines <i>as well as</i> a choropleth based on some statistic of the waterways.</li>
<li>In the UI, a list of those regions. Picking one will zoom to the region, and will also filter the gauges and waterways to that region. The gauges and waterways are tagged with their region, so that filtering should be easy.</li>
<li>Tooltips when we mouse over waterways and gauges, and more-detailed popups when they are clicked.</li>
<li>Unspecified other Leaflet controls to be added to the map as needed: geocoders, custom legend panel, GreenInfo credits, custom scalebar, probably more.</li>
</ul>
</div>
The datasets above are fairly hefty in size, so simple GeoJSON techniques would not be appropriate here. Failing that, filtering by region would classically be a server-side phenomenon e.g. a WMS service and less-smooth interactivity.<br />
<br />
These days, Mapbox is full bore on vector tiles. Conceptually, vector tiles brings the benefits of GeoJSON such as slick interactivity and having attribute data already in memory, with the on-demand loading of tiles, plus the server-side processing involved in generating those vector tiles efficiently by not loading every vertex of every feature.<br />
<br />
Let's see what Mapbox can do!<br />
<br />
<h3>
Mapbox</h3>
We love <a href="https://www.mapbox.com/" target="_blank">Mapbox</a>. They have a habit of changing things fairly often and sometimes breaking things, but a lot more often, they get things just right and really shine.<br />
<br />
I rarely get to dig in to Mapbox Studio (they usually give the cartography to someone else), so I had some learning to do:<br />
<ul>
<li>A dataset is called a Tileset</li>
<li>Uploading a Tileset can take a couple of hours<br />In one case, the spinner never went away but I came back the next morning to find it still spinning. I reloaded the page and it had loaded just fine</li>
<li>A collection of datasets plus filtering and classifications-and-colors, organized into layers, is called a Style<br />(I think of class-and-color as a style, and a collection of layers as a map, but okay...)</li>
<li>Using the Mapbox GL API, layers can be hidden and shown, can have their filtering changed at runtime, etc.</li>
</ul>
<br />
<h3>
Leaflet + Mapbox GL JS API</h3>
Mapbox of course recommends their own <a href="https://www.mapbox.com/mapbox-gl-js/api/" target="_blank">Mapbox GL JavaScript API</a>. While their Mapbox API really is just Leaflet with a bunch of additions and so is almost-completely compatible with third-party Leaflet extensions, Mapbox GL API is not like that.<br />
<br />
We have all sorts of custom needs which aren't defined yet: custom map controls for legends and geocoders, maybe even Leaflet Draw at some later iteration. We're always skittish about using a semi-proprietary API that won't let us use freely define custom controls and incorporate third-party offerings, since we're <i>all about</i> delivering highly custom solutions.<br />
<br />
Fortunately, the folks at Mapbox have <a href="https://github.com/mapbox/mapbox-gl-leaflet" target="_blank">a semi-unofficial Leaflet adapter to Mapbox GL</a>. It works pretty well, and almost everything went perfectly as planned: the layer goes onto the map, I can toggle layer visibility and apply filters, I can publish and see the updated Style, etc.<br />
<br />
<h3>
Spoiler Alert: It's Awesome</h3>
The application came along well, giving slick interactivity, good performance, filtering capabilities, easy cartography changes as needed, and also the flexibility to incorporate arbitrary Leaflet controls and behaviors.<br />
<br />
Along the way I learned a lot, and what I learned is the topic of the next few days' postings.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-68422926234515702652017-04-13T09:37:00.004-07:002017-04-13T09:37:56.317-07:00ArcGIS tile + dynamic layer with automatic switchoverThis was another interesting one to puzzle out, and turned out to be surprisingly elegant once I had wrapped my head around it.<br />
<br />
<ul>
<li>The client has an ArcGIS Server with a tile service. We set up a Leaflet map for them showing these tiles, a couple of years back.</li>
<li>They now have dynamic services. They are not fast enough for production use at far-out zoom levels, but they are available at deeper scales than the tiles.</li>
<li>Can we have the map detect that they've zoomed in far enough, and silently switch between the two? Tiles when far-out and dynamic when close-in?</li>
</ul>
The answer was a comparatively simple wrapper class: <br />
<a href="https://github.com/GreenInfo-Network/L.TileLayer.TiledAndESRIDynamicTandem">https://github.com/GreenInfo-Network/L.TileLayer.TiledAndESRIDynamicTandem</a><br />
<br />
This uses the <a href="https://github.com/Esri/esri-leaflet" target="_blank">ESRI-Leaflet adapter</a>, and creates within itself both a <i>L.TileLayer</i> and a <i>L.ESRI.DynamicMapLayer</i>. It then watches the map's <i>zoomend</i> event and does what it should do.<br />
<br />
Our needs here were pretty specific, so there are a few things it does and does not support. If needs change, or someone else is interested, maybe this could be developed further.<br />
<ul>
<li>It has a setOpacity() method which will apply to both sub-layers. During this I discovered and had to work around some behaviors of the <i>dynamicMapLayer</i> class, e.g. one cannot set its opacity if it is not in the map at that time (one can with <i>L.TileLayer</i>).</li>
<li>Options are passed to both layers as-given, e.g. the *attribution* is the same for both. In this case that's perfect.</li>
</ul>
Maybe this will fit a need of yours, and maybe it won't. But for me, it was a fun and interesting puzzle, and the clients couldn't be happier.<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-53353422801163510142016-12-19T12:49:00.000-08:002016-12-19T12:49:01.711-08:00Scroll-Animated Image-Based MoviesLast week I had an interesting request that was fun to figure out. The GIS folks had put together a series of images forming an animation, as one zooms in on the California coast. Inspired by parallax image effects, they wanted to know if the image animation could be tied to scrolling the page, so the animation plays a frame at a time as you scroll.<br />
<br />
We found a few existing toolkits that did this such as ScrollMagic, but found that the animation was not at all smooth, and that it didn't work so well for some of the cases the web designers specifically had in mind. A few of them didn't work on mobile at all.<br />
<br />
So I ended up creating my own, and came up with two prototypes you may find interesting.<br />
<br />
<h3>
Scroll-Animated Header</h3>
<br />
This first prototype, has a top-and-center banner image on the page as with a lot of sites. But the page content locks to the height of your screen and forces scrolling, and the banner animates as you scroll.<br />
<br />
<a data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/GreenInfo-Network/ScrollAnimatedHeader&source=gmail&ust=1482265596785000&usg=AFQjCNHD-kbNoDR_Camz9U4i_4EWUxoHlg" href="https://github.com/GreenInfo-Network/ScrollAnimatedHeader" rel="noreferrer" target="_blank">https://github.com/GreenInfo-<wbr></wbr>Network/ScrollAnimatedHeader</a><br />
<br />
The effect is pretty neat, right?<br />
<br />
This one was meant as a proof of concept and a demonstration of a technique, and not a redistributable library. Still, if you're interested you can View Source above or hit up the repo: <a data-saferedirecturl="https://www.google.com/url?hl=en&q=https://greeninfo-network.github.io/ScrollAnimatedHeader/&source=gmail&ust=1482265596785000&usg=AFQjCNHXBMZMCXbuVIIwDHadDOgID9KtVg" href="https://greeninfo-network.github.io/ScrollAnimatedHeader/" rel="noreferrer" target="_blank">https://greeninfo-network.<wbr></wbr>github.io/<wbr></wbr>ScrollAnimatedHeader/</a><br />
<br />
<br />
<br />
<br />
<h3>
jquery.Reanimator</h3>
<br />
Turns out that what the web folks really wanted, was for multiple, smaller image-animations to be sprinkled and embedded throughout the page content. The same way you'd normally add some <i>float:right</i> and <i>float:left</i> images throughout your text, they wanted image-animations.<br />
<br />
So here's a look at a new jQuery plugin I created: <b>jQuery.Reanimator</b>.<br />
<br />
<a href="https://greeninfo-network.github.io/jQuery.Reanimator/">https://greeninfo-network.github.io/jQuery.Reanimator/</a><br />
<br />
Unlike the previous one, this is specially designed to work with multiple such animations, and to not make assumptions about the page flow (e.g. no fixed header and no fixed scrolling zone). And it's implemented as a jQuery plugin, so it's almost copy-paste easy to apply to any one or any several such animations.<br />
<br />
jQuery.Reanimator sports some neat features as well.<br />
<ul>
<li>Animations can have <i>mousemove</i> and <i>click</i> effects, to play their animations. This provides alternate ways of triggering the animation should scrolling be insufficient.</li>
<li>Ability to add padding to the top and bottom of the scrollable area for an image.</li>
<li>These options can be set in the constructor, and also in <i>data-reanimator-XXX</i> attributes to override that global setting. Makes it easy to customize individual animations.</li>
<li>A "debug mode" to show the scrollable areas, etc. to give insight into testing and tuning the animation for your website.</li>
</ul>
For more info and to download it for your site, hit up the repo:<br />
<br />
<a href="https://github.com/GreenInfo-Network/jQuery.Reanimator">https://github.com/GreenInfo-Network/jQuery.Reanimator</a> <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-55908784562940957392016-11-23T16:02:00.002-08:002016-11-23T16:07:58.353-08:00Many things brewingHappy harvest festival!<br />
<br />
I realized the other day that it's been 5 months since my last posting. Things have been happening, but many of them are still in progress so I've kept quiet. But here's a sampling of some of what I've been working on.<br />
<br />
<h3>
AccordionLegend control for Leaflet</h3>
<h3>
</h3>
<h3>
</h3>
This was
actually my prior blog post, back in June. It's a Leaflet control that
displays a spiffy collapsing legend, with layers broken into accordion
sections, with legend swatches and opacity sliders and all.<br />
<br />
<a href="http://greeninfo-network.github.io/L.Control.AccordionLegend/">http://greeninfo-network.github.io/L.Control.AccordionLegend/</a> <br />
<br />
Just today, someone reported a bug with it and I fixed that bug, then went on to improve the demo, and add a few more features.<br />
<br />
<h3>
Early Warning System v2</h3>
<br />
The EWS scrapes project disclosures from several international funding banks such as the World Bank, International Funding Corporation, African Development Bank, Asian Development Bank, and others. The material is reviewed by the International Accountability Project, looking projects that could threaten human rights, e.g. involuntary relocation of a whole town, and connecting with local activists.<br />
<br />
<a href="http://rightsindevelopment.org/?page_id=2421">http://rightsindevelopment.org/?page_id=2421</a><br />
<br />
I did the original EWS system back in 2011, but they want to clean things up, expand on the system, add support for other folks editing it, etc. Internal goals included some code cleanup, a nicer model structure, better error handling, a nicer search UI, and more. And, this project is in Django, which I just love, so there's a bonus.<br />
<br />
The EWSv2 is coming along well, and preliminary feedback is quite positive. We should have the thing launched to the public in a month or three.<br />
<br />
<h3>
Tobacco retailer visualizations, next-generation system</h3>
<h3>
</h3>
A long-time client of GreenInfo Network, CounterTools has us make interactive web maps of tobacco retailers, inspections and undercover purchases of tobacco, and derivative stats such as "percentage of undercover sales which sold, by county" Our oldest maps with them go back to late 2011 or early 2012, with the latest of the current generation having been set up only a few weeks ago.<br />
<br />
I'm working on the next generation of both the data management and mapping components. The project involves 3 agencies for 3 sections: back-end, front-end, and GIS/mapping. The mapping fits in the middle, firmly in both front-end and back-end: data models and tables, file uploads and data processing, data searching and reporting, and front-end cartography.<br />
<br />
Unlike the current generation, this one will work with a living dataset, with new retailers and retailer inspections being added daily, so there's a whole new dimension of intelligence in the upload capabilities and error handling, in the calculations and rendering, and so on. Also unlike the current generation, the front-end is in AngularJS, and the back-end is in Django. (yay!)<br />
<br />
I've been learning a lot from how the other teams do their job: AngularJS and Sails for the front-end, Django REST Framework for writing new APIs, their particular tactics for layout out the sections of the Django app, and so on. It's been a learning curve there, but I'm appreciating the education.<br />
<br />
<h3>
Coder Commando</h3>
<br />
I've also been handling the usual flow of miscellaneous requests that aren't large projects, but "my coder quit and I launch in two weeks!" situations. This happens a lot more often than you would imagine; four projects in the last five months, come immediately to mind.<br />
<br />
I don't usually blog about these since they're not usually the lengthy and involved projects, but they do keep me busy. Some of my recent ones have involved some neat moving parts:<br />
<ul>
<li>A tool to draw a polygon on a Google Maps map, and derive statistics from census blockgroup data, then download a PDF.</li>
<li>Learning about Expression Engine, a CMS from the folks who made CodeIgniter, and writing a bunch of fixes in various pages, some of them maps and some not.</li>
<li>A map of farms and farmers' markets statewide, with a nice CMS on the back end where one manages five pages of details about their farm. Notably, the part that lets you upload photos of your farm, has a UI for zooming and cropping the photo, so your 200x100 thumbnail looks the way you want it.</li>
</ul>
<br />
<h3>
Happy Harvest</h3>
<h3>
</h3>
So yeah, I've been silent but not idle. I'm told of at least 2 new projects waiting for signatures, and some these projects in progress will likely start going live over the next few months. Once the lid is off, I can describe them in more detail.<br />
<br />
Have a nice holiday!<br />
<br />
<br />
<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-22493896849512006012016-06-14T11:48:00.001-07:002016-06-14T11:48:05.674-07:00New Leaflet control: L.Control.AccordionLegendSome time back I came up with a custom legend control for Leaflet. Unlike the basic L.Layers control, we wanted it to be really snazzy:<br />
<ul>
<li>The layer list broken into sections, not by basemap/overlay, but thematic similarity: Parks in a section, Admin Boundaries as a section, Health Statistics as a section, etc. And the sections should have accordion-style behavior: clicking a section title should expand and collapse, show only one section at a time so as not to overload the screen, and so on.</li>
<li>Each layer has an opacity slider.</li>
<li>Each layer has a legend.</li>
</ul>
So, here it is:<br />
https://github.com/GreenInfo-Network/L.Control.AccordionLegend<br />
http://greeninfo-network.github.io/L.Control.AccordionLegend/<br />
<br />
Enjoy!<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-52177315064362955352016-06-03T12:38:00.002-07:002016-06-03T12:38:34.897-07:00New Leaflet control: L.Control.CartographicScaleA request we get now and then, is to add a readout of the current map scale denominator, e.g. a readout that changes from "1:24K" to "1:12K" as you zoom in a notch. These are handy for cartographers who still think in terms of scale, instead of zoom levels.<br />
<br />
OpenLayers had the handy-dandy <a href="http://dev.openlayers.org/docs/files/OpenLayers/Control/Scale-js.html" target="_blank">OpenLayers.Control.Scale</a> but I couldn't find a similar such control for Leaflet. So I wrote it.<br />
<br />
<a href="http://greeninfo-network.github.io/L.Control.CartographicScale/" target="_blank">http://greeninfo-network.github.io/L.Control.CartographicScale/</a><br /><br />
<a href="https://github.com/GreenInfo-Network/L.Control.CartographicScale" target="_blank">https://github.com/GreenInfo-Network/L.Control.CartographicScale</a><br />
<br />
<br />
<br />
I should point out that the accuracy of this sort of thing, has always been dubious. Between the projection itself (a "square degree" in Lagos is not the same area as in Barrow, and Greenland is not that big in real life) and the realities of screen resolution (96dpi is for mid-90s CRTs) these scale readouts have always had very poor accuracy and always will. It's the mathematics of a world that isn't flat, you know?<br />
<br />
Still, the control is accurate enough for cartographers to get the sense of "2K or 20K?" and that's what was really needed here.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-57466937171043469062016-05-20T17:51:00.000-07:002016-05-20T17:51:07.386-07:00In-Browser filtering of raster pixels in Leaflet, Part 3In my last posting, I mentioned described the process of creating a three-band TIFF and then slicing it into three-band PNG tiles for display in Leaflet. The tiles are not visually impressive, being almost wholly black, but they have the proper R, G, and B codes corresponding to the location's PHZ, PCP, and GEZ. (if you don't know what those are, go back and read that post from a few days ago)<br />
<br />
So now the last step: transforming these tiles based on a set of arbitrary RGB codes determined by the user. Let's go through <a href="https://github.com/GreenInfo-Network/L.TileLayer.PixelFilter/blob/master/leaflet-tileLayerPixelFilter.js" target="_blank">the working source code for the L.TileLayer.PixelFilter</a> and I'll describe how it works.<br />
<br />
First off, we need two RGBA codes so we can color the pixels as hit or miss, and we need the list of [r,g,b] trios that would be considered a match. The <i><span class="pl-en">initialize</span></i><span class="pl-k"> method accepts these as constructor params, and the various <i>set</i> functions allow this to be set later as well.</span><br />
<br />
<span class="pl-k">(The decision of what list of RGB codes should be passed into <i>setPixelCodes()</i> is a matter of business logic: selectors, clicks, lists... outside the scope of this discussion. See the demo code for a trivial example.)</span><br />
<span class="pl-k"><br /></span>
<span class="pl-k">Next, we need to intercept the map tile when it's in the browser. Thus the </span><br /><span class="pl-s"><span class="pl-pds"></span><i>tileload</i><span class="pl-pds"> event handler set up in the constructor, which runs the tile through </span></span><br /><i><span class="pl-en">applyFiltersToTile</span></i><span class="pl-k"><i>()</i> to do the real work.</span><br />
<span class="pl-k"><br /></span>
<i><span class="pl-en">applyFiltersToTile</span></i><span class="pl-k"><i>()</i> is the interesting part, which only took me a couple of hours sitting down with some HTML5 Canvas tutorials. Let's dissect it one piece at a time:</span><br />
<ul>
<li><span class="pl-k"><span class="pl-k">"</span><span class="pl-k"><span class="pl-c">copy the image data onto a canvas" </span></span>The first paragraph </span><span class="pl-k"><span class="pl-c"></span>creates a Canvas of the same width and height as the image, and copies the data into it. The IMG object itself won't let us access raw pixel data, but once it's in a Canvas we can.</span></li>
<li><span class="pl-k"><span class="pl-k">"</span><span class="pl-k"><span class="pl-c">target imagedata" </span></span>We then </span><span class="pl-k"><span class="pl-c"></span>use <i>createImageData()</i> to construct a new and empty byte sequence, which can later be assigned back into a Canvas using </span><span class="pl-en"><i>putImageData()</i> This is effectively a straight list of bytes, and as we write to it, we need to always write 4 bytes at a time for every pixel: R, G, B, and A.</span></li>
<li><span class="pl-en"><span class="pl-en">"</span><span class="pl-en"><span class="pl-c">generate the list of integers" </span></span>Before we start looping over pixels, a performance optimization</span><span class="pl-en"><span class="pl-c"></span>. Arrays have the <i>indexOf()</i> method so we can determine whether a given value is in an array, but it only works on primitive values and not three-element arrays. The <a href="https://lodash.com/" target="_blank">lodash library</a> has <i>findIndex()</i> which would work, but it means a new dependency... and also the performance was not so great (256 x 256 = 65536 pixels per tile). So I cheat, and translate the list of desired </span><br /><span class="pl-smi"><i>pixelCodes</i> into a simple list of integers so that <i>indexOf()</i> can work after all.</span></li>
<li><span class="pl-smi"><span class="pl-smi">"</span><span class="pl-c">iterate over the pixels" </span>Now we start looping over the pixels in our Canvas and doing the actual substitution. Again, we step over the bytes in fours (R, G, B, A) and we would assign them into the <i>output</i> imagedata in fours as well. Each pixel is translated into an integer, and if that integer appears in the </span><span class="pl-smi">pixelcodes list of integers, it's a hit. Since <i>indexOf()</i> is a native function it's pretty swift, and since our </span><span class="pl-smi">pixelcodes list tends to be very short (10-50 values) the usually-avoided loop-in-loop is actually quite fast.</span></li>
<li><span class="pl-smi">"</span><span class="pl-c">push a R, a G, and a B" Whatever the results, we push a new R, G, B, A set of 4 bytes onto the <i>output</i> imagedata.</span></li>
<li><span class="pl-smi"> "</span><span class="pl-c">write the image back" And here we go: write the imagedata sequence back into the Canvas to replace the old data, then reassign the <i>img.src</i> to this Canvas's base64 representation of the newly-created imagedata. Now the visible tile isn't based on a HTTP file at all, but from a base64-encoded image in memory.</span></li>
</ul>
<span class="pl-c">The only gotcha I found, was that upon calling </span><span class="pl-c"><i><span class="pl-en">setPixelCodes</span></i><span class="pl-k"><i>()</i> </span>the tiles would flash to their proper color and then instantly all turn into the no-match color. It worked... then would un-work itself?</span><br />
<span class="pl-c"><br /></span>
<span class="pl-c">The <i>tileload</i> event wraps the IMG element's <i>load</i> event. This means that when I assigned the <i>img.src</i> to replace the tile's visible representation... this was itself a <i>tileload</i> event! It would call </span><i><span class="pl-en">applyFiltersToTile</span></i><span class="pl-k"><i>()</i> again, and this time the RGB codes didn't match anything on my filters so all pixels were no-match. Worse, the assignment of <i>img.src</i> was again a <i>tileload</i> event, so it was in an infinite loop of processing the tile.</span><br />
<span class="pl-k"><br /></span>
<span class="pl-k">Thus the </span><span class="pl-smi"><i>already_pixel_swapped</i> flag. After processing the IMG element, this flag is set and </span><br /><i><span class="pl-en">applyFiltersToTile</span></i><span class="pl-k"><i>()</i> will skip out on subsequent runs on that IMG element. If we need to change pixel codes via <i>setPixelCodes()</i> that calls <i>layer.redraw()</i> which empties out all these old IMG elements anyway, replacing them with fresh new ones that do not have</span> <span class="pl-smi"><i>already_pixel_swapped</i> set.</span><br />
<br />
<span class="pl-smi">So yeah, it was an educational day. Not only did we achieve what the client asked for (the more complex version, not the simplified case I presented last week) and as a neat reusable package, but the performance is near instant.</span><br />
<br />
<br />
<span class="pl-c"></span>Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-76024735843457803052016-05-17T17:23:00.000-07:002016-05-17T17:23:03.720-07:00In-Browser filtering of raster pixels in Leaflet, Part 2A few days back I posted a link to <a href="https://github.com/greeninfo/L.TileLayer.PixelFilter" target="_blank">L.TileLayer.PixelFilter</a> This is a Leaflet TileLayer extension which rewrites the tiles after they have loaded, comparing each pixel against a set of pixel-codes and replacing the pixel with either a "matched" or "not matched" color. It's pretty useful for generating dynamic masks and highlights, if your back-end data is a raster and not vector data.<br />
<br />
I had said that the client's request was to display plant hardiness zones, and that I was using the Unique Values colors as the RGB codes to match against zone codes. I lied. The reality was much more complicated than that, but would have been distracting from the end and the result. So here's the longer story.<br />
<br />
<h4>
The Rasters and the Requirement</h4>
<br />
The client has 3 rasters: Plant Hardiness Zones (PHZ), Precipitation classifications (PCP), Global Ecoregions (GEZ). Each of these is a worldwide dataset, and each has about 20 discrete integer values numbered 10 to 30ish.<br />
<br />
The user selects multiple combination of these three factors, e.g. these two:<br />
<ul>
<li>Zone 7 (PHZ=7)</li>
<li>Rainfall 20-30 inches per year (PCP=3)</li>
<li>Temperate oceanic forest (GEZ=31)</li>
</ul>
<br />
<ul>
<li>Zone 7 (PHZ=7)</li>
<li>Rainfall 20-30 inches per year (PCP=3)</li>
<li>Temperate continental forest (GEZ=32)</li>
</ul>
The user would then see any areas which match any of these "criteria trios" highlighted on the map. The idea is that a plant that's invasive in these areas, would also be invasive in the other areas highlighted on the map.<br />
<br />
Fortunately, the rasters are not intended to be visible and do not need to be visually pleasing. They need to be either colored (the pixel matches any of the trios) or else transparent (not a match).<br />
<br />
<h4>
Raster To Tiles To Leaflet</h4>
<br />
Three variables and a need for one raster... sounds like we could have a three-band raster! I used ArcMap's <i>Composite Bands</i> tool to merge the three rasters into a three-band raster. Voila, one TIFF with three-part pixels such as (7, 3, 31) and (7, 3, 32) I just have to keep straight that R is PHZ, G is PCP, and B is GEZ.<br />
<br />
Second, I needed to slice this up into map tiles. But it's very important that:<br />
<ul>
<li>The tiles be in a format that preserves R, G, and B exactly. Something GIF or JPEG would be entirely unsuitable since GIF picks a 256-color palette, and JPEG is lossy and fudges together colors. TIFF on the other hand is not viewable in browsers as a map tile. But PNG is just perfect: RGB by default, and viewable in browsers.</li>
<li>The tile-slicing process (I picked <i>gdal2tiles.py</i>) must also preserve exact RGB values. The default <i>average</i> resampler uses interpolation, so a pixel in between a PCP=3 and PCP=2 would get PCP=2.5 and that's no good! Use the <i>nearest neighbor</i> resampling algorithm, which guarantees to use an existing pixel value.</li>
</ul>
Slicing up the TIFF into web-read PNGs was pretty straightforward:<br />
<blockquote class="tr_bq">
gdal2tiles.py -z 0-7 -r near -t_srs epsg:3857 threeband.tif tiles</blockquote>
<br />
Point Leaflet at it, and I get some nearly-black tiles loading and displaying just as I expected. Well, after I remembered that <i>gdal2tiles.py</i> generates tiles in TMS numbering scheme, so I had to set <i>tms:true</i> in my L.TileLayer constructor.<br />
<br />
I downloaded a few of the tiles and opened them up in GIMP and sure enough, that shade of black is actually (7,3,31) The tiles are not meant to be visually attractive... but those preserved RGB codes are beautiful to me. <br />
<br />
<br />
<h4>
Client-Side Pixel Detection and Swapping</h4>
<br />
That's the topic of my next posting: now that I have RGB tiles, how can I intercept them, compare them against a set of RGB codes, and appropriately color/clear pixels...? Turns out it was easier and more elegant than I had imagined.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-40528135670165731652016-05-14T17:02:00.000-07:002016-05-14T17:02:07.778-07:00In-Browser filtering of raster pixels in Leaflet, Part 1Last week I was asked to do something I haven't done before:<br />
<br />
A client has a raster of values, this being World Plant Hardiness Zones. Each pixel is a code number, which ranges from 1 to 13. If you're into gardening, you've seen the US variation on this on the back of seed packets.<br />
<br />
The client wants the user to select one or more hardiness zones, and have the appropriate pixels highlighted on the map. For reasons I'll cover in a later blog post, this needs to be done using the raster's RGB codes and not through some of the simpler techniques such as GeoJSON polygons with varying opacity.<br />
<br />
I ended up writing this Leaflet TileLayer extension. It takes a list of RGB codes, and then rewrites the tile images in-canvas so as to have only two colors: those which matched and those which did not.<br />
<blockquote class="tr_bq">
<a href="http://greeninfo-network.github.io/L.TileLayer.PixelFilter/">http://greeninfo-network.github.io/L.TileLayer.PixelFilter/</a><br />
<a href="https://github.com/GreenInfo-Network/L.TileLayer.PixelFilter/">https://github.com/GreenInfo-Network/L.TileLayer.PixelFilter/</a></blockquote>
<br />
And there's our basic need filled: near-instant pixel-filtering into "selected" and "non-selected" categories, with appropriate color fill.<br />
<br />
If you're in a hurry, there's the link to the demo and to the library. But the technical process involved, and the application of it for the client turned out to be very interesting indeed, so they will form my next few postings.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-79656297227467626512016-04-29T14:05:00.000-07:002016-04-29T14:05:00.596-07:00Heroku and Wordpress: AfterthoughtsThe last two postings were about my successful and happy migration of my personal Wordpress site from my home PC to Heroku. It went beautifully, but a little testing and a few days revealed some improvements. One could call these Pitfalls and Gotchas about Heroku, lessons for my next app.<br />
<br />
<br />
<br />
<h4>
Database Usage / Wordpress Revisions</h4>
<br />
My MySQL dump file was 2.3 MB, fitting quite well under the 5 MB mark. Wrong. After being loaded, it bloated to about 4.1 MB and I got a warning email from Heroku that I was nearing my capacity.<br />
<br />
<i>First off, thank you Heroku and ClearDB.</i> Nice job on the notification before this became a problem.<br />
<br />
Second, what's using my disk space? I did some queries (thank you, Stackoverflow) and found that wp_posts had 450 rows... for my 35 pages. Ah yes: <i>Page Revisions</i>. Wordpress keeps copies of every time I've clicked Update, so there are hundreds of extra pages in there.<br />
<br />
I trawled the net a little bit for two pieces of advice that set this right:<br />
<ul>
<li>Disabling revisions (or limiting them to 1 or 2 revisions, if you prefer), with a <i>wp-config.php</i> modification. Easy.</li>
<li>A plugin to clean out old revisions. I then uninstalled it since disabling revisions means there won't be more to clean up.</li>
</ul>
Now my table was down to 150 posts, which is right for 30ish pages and 120ish images. My dump file was under half the previous size.<br />
<br />
To finish the job, I had to take a dump of the table and then reload it, so as to drop and recreate the table. (databases do that, keep the space for the future; it's normally a good idea, but not here and today) And now I'm down to 1.6 MB, one third of my free tier.<br />
<br />
<h4>
Sending Email</h4>
<br />
Heroku doesn't have a sendmail setup, not even sure they have a SMTP service. So Wordpress's email capabilities were not working.<br />
<br />
Your basic options here are:<br />
<ul>
<li>Sign up for SendGrid, since Heroku has an addon for that. They have a free tier of 12,000 messages per month, which is really more than my blog could ever generate.</li>
<li>Supply your own SMTP server.</li>
</ul>
<br />
I went for the latter since I use GMail and can definitely navigate the steps of a Project, the API keys, and <a href="https://wordpress.org/plugins/gmail-smtp/" target="_blank">the GMail SMTP plugin for Wordpress</a>. The plugin's own instructions were somewhat outdated, but that wasn't a problem. And voila, Wordpress sending email.<br />
<br />
For your own programming, you would want to use PHPMailer or some such so you can specify your SMTP settings. A lot of things presume that there's a local sendmail or local SMTP, but not here.<br />
<br />
<br />
<br />
So yeah, that was it. Not too awful in the "gotchas" department. Not at all! I am suitably impressed with Heroku at virtually every level: price, performance, ease of deployment once you wrap your head around it, and notifications.<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-23358572305734440342016-04-27T13:54:00.000-07:002016-04-27T13:54:01.117-07:00Heroku and Wordpress: The Migration ProcessMy post day-before-yesterday begun the story of my migration from a home-hosted website to Heroku. Here's part two: The Migration Process<br />
<br />
<h4>
Step: Move my Media Library content into S3</h4>
<br />First
off, the uploaded photos should not be under version control; so they
shouldn't be in the repository and in the ephemeral filesystem.<br />
<br />
This was a tedious process of a few
hours since I had about 130 images.<br />
<ul>
<li>I set up the S3 plugin I mentioned above, made a few tests and confirmed that it's sweet.</li>
<li>I SFTP'd into my Wordpress site's wp-content/uploads folder and grabbed everything.</li>
<li>Then deleted everything from my Media Library,</li>
<li>Then uploaded it all again and watched it load into S3 and leave my uploads folder empty.</li>
<li>Then went through every posting and replaced all of the images, which of course were now broken. Annoying, but fortunately I only had 35 pages with images and did it in about 2 hours.</li>
</ul>
<br />
<h4>
Step: Init repo</h4>
<br />
I'm a fan of <a href="http://www.gitlab.com/" target="_blank">Gitlab</a>. They offer <b><i>unlimited private repositories </i></b>for
free, which is really excellent. I created the repository, then
followed their simple instructions to load my Wordpress files into the
repo and basically turn my site into a clone.<br />
<br />
I also created a .gitignore file with these two entries for some folders I'll be creating in a little bit. The <i>private</i>
is where I'll do some other work I don't want in version control, e.g.
working files and database dumps that I want to keep close. The <i>vendor</i> would be generated by composer later on, trust me.<br />
<blockquote class="tr_bq">
/private/<br />
/vendor/ </blockquote>
<br />
<h4>
Step: Add Heroku as a Secondary Master</h4>
<br />
The trick in allowing <i>git push</i> to push to deploy to Heroku, is that you tell your repo clone to use your Heroku as a secondary master.<br />
<br />
<blockquote class="tr_bq">
heroku git:remote -a your-server-name</blockquote>
<br />
As of now, when pushing you will need to distinguish between <i>git push origin master</i> and <i>git push heroku master</i>. One pushes into your git repository (Gitlab, Github) and the other would redeploy to Heroku.<br />
<br />
<h4>
Step: Wordpress Updates</h4>
<br />
I
then noticed that some updates were available for some plugins and for
Wordpress itself. So I ran those, and after each one noted that <i>git status</i>
reported exactly what I would expect from each upgrade. So three
commits later I had Wordpress and plugins all updated, with commit notes for each update. (I
could have done this before the repo init, but why not do it under version control?)<br />
<br />
<h4>
Step: Add a Procfile and composer.json file</h4>
<br />
For Heroku compatibility, it's advisable to add a <i>Procfile</i> and a <i>composer.json</i> and <i>composer.lock</i>
file, to indicate to heroku what PHP version to prefer, that you prefer
Apache over Nginx, etc. You will want these in version control.<br />
<br />
Procfile<br />
<blockquote class="tr_bq">
web: vendor/bin/heroku-php-apache2</blockquote>
<br />
composer.json<br />
<blockquote class="tr_bq">
{<br /> "require" : {<br /> "php": "^5.6.0"<br /> },<br /> "require-dev": {<br /> "heroku/heroku-buildpack-php": "*"<br /> }<br />}</blockquote>
<br />
<br />
composer.lock is generated from the composer.json with a command:<br />
<blockquote class="tr_bq">
composer update --ignore-platform-reqs</blockquote>
<br />
<h4>
Step: Push to Heroku</h4>
<br />
All set? Then here goes:<br />
<blockquote class="tr_bq">
git push heroku master</blockquote>
And I visit my website. And it's a Wordpress error that it can't make the database connection. That's to be expected: my wp-config.php has the old credentials and I still need to upload the database content. But that's definitely Wordpress making the error, so a fine start.<br />
<br />
<h4>
Step: Database Configuration</h4>
<br />
First step was to scrub my database credentials from the <i>wp-config.php</i> file. Yeah, dummy move to forget to do that, but the credentials are wrong for Heroku so are useless, and my local MySQL is going away in an hour anyway. But yes... don't do what I did. ;)<br />
<br />
When I scrubbed the database credentials, I replaced them with $_ENV variables like this:<br />
<blockquote class="tr_bq">
define('DB_NAME', $_ENV['DATABASE_BASE']);<br />define('DB_USER', $_ENV['DATABASE_USER']);<br />define('DB_PASSWORD', $_ENV['DATABASE_PASS']);<br />define('DB_HOST', $_ENV['DATABASE_HOST']);</blockquote>
Add, commit, push. Site is still dead but it's ready for this next trick.<br />
<br />
<br />
Run heroku config and it dumps the app's configuration variables back to me. I tease apart the database URL string, into my set of 4 environment variables for the 4 $_ENV items above.<br />
<blockquote class="tr_bq">
heroku config:set DATABASE_BASE='heroku_XXX' DATABASE_USER='XXX' DATABASE_PASS='XXX' DATABASE_HOST='XXX'</blockquote>
My app/dyno reboots and... the site's up!<br />
<br />
<h4>
Step: Database Data</h4>
<br />
A simple mysqldump was all it took to take a backup of my database, then loading it was one more command.<br />
<br />
<blockquote class="tr_bq">
mysqldump -h localhost -u olduser -p olddbname > private/dbdump.sql<br />
mysql -h herokuhost -u herokuuser -p herokudbname < private/dbdump.sql</blockquote>
<br />
It doesn't get as lot easier than this.<br />
<br />
And now the site is up! My data, on a new database on a new Heroku server. Shiny.<br />
<br />
<br />
<h4>
Step: Custom Domain</h4>
<br />
My site is <a href="http://www.fightingfantasyfan.info/">www.fightingfantasyfan.info</a> and not <a href="http://fightingfantasyfan.herokuapp.com/">fightingfantasyfan.herokuapp.com</a> Heroku calls this a <i>custom domain</i>. There are two steps in setting up the Heroku app to work properly with a custom domain. <br />
<br />
<ul>
<li>I hit up Heroku's dashboard and the Settings for my site, and added two <i>domains</i> for it: www.fightingfantasyfan.info and fightingfantasyfan.info This allows it to respond to these alternate hostnames, should they point to this app.</li>
<li>I went to my domain registrar's domain control panel and set up a
CNAME record, so that www.fightingfantasyfan.info is equivalent to
fightingfantasyfan.herokuapp.com This took a little bit to propagate, but was done about the time I finished my cup of coffee.</li>
</ul>
<br />
<br />
<h3>
And that really was it. Well, almost. More on this tomorrow.</h3>
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-34586190339922769842016-04-25T13:31:00.000-07:002016-04-25T13:31:01.266-07:00Heroku and WordpressThis is a bit off-topic for <b>The Map Guy</b> since it has nothing to do with maps, directly. But it's interesting and it's about the cloud, and about software deployment. It's about... Hosting a low-volume personal Wordpress website on Heroku for free.<br />
<br />
<h3>
Introduction</h3>
<br />
I have a couple of personal websites for hobbies. I'm almost ashamed to admit that I'm using Wordpress for them, being the hardcore, bad-ass hacker that I am... but in the evenings and with nothing at stake, the convenience of Wordpress works for me.<br />
<br />
I
had been hosting my site on a Raspberry Pi at home, but the reliability
just wasn't there. The Pi would crash from time to time and require
rebooting. After the second time it did this while I was away on a trip,
I decided it was time for a change. Even better, this is an opportunity
to say Hello To The Cloud with a real-world application with no stakes
involved.<br />
<br />
Enter Heroku, my hero. <br />
<br />
<a href="http://www.heroku.com/" target="_blank">Heroku</a> is a service that spins up virtual machines (they call them "apps"), using as their filesystem your git repository. The basic idea is this:<br />
<ul>
<li>You have a git repository which contains your website: HTML, PHP, Python, Ruby, etc.</li>
<li>You create an "app" via their dashboard, also called a "dyno", and it has an URL. You then attach "Add-Ons" such as a MySQL database, a PostgreSQL database, SMTP via SendGrid, and other such services that you'll need. </li>
<li>Using Heroku's command-line "toolbelt" you add to your clone of the repository, the fact that this clone is "connected" to that app/VPS. So your clone has two masters: <i>git push origin master</i> for saving your code to version control as usual, and <i>git push heroku master</i> to deploy it to the site.</li>
<li>When you push to <i>heroku master</i>, Heroku resets your VPS, loading up its filesystem from your repository content. Assuming that your code works, the site works too.</li>
<li>No permanent storage ("ephemeral filesystem"). When the VPS is load-balanced to another system, or put to rest, or you push again, it's reset from your repo. <i>This is very important if your web app will be making filesystem modifications such as accepting uploads. More on this later.</i></li>
</ul>
So the effect is a build-from-components server, enabling the specific services you need.<br />
<br />
For small personal sites, free tiers for the dyno and for the add-on services may prove sufficient to host the Wordpress code and the database entirely for free. The limitations of this free tier, as relevant to my personal Wordpress site, are:<ul>
<li>The dyno (the VPS itself) will go to sleep if there's no activity, and the next incoming activity to wake it up does mean some delay. The dyno must be asleep at least 6 hours per day, so using Uptime Robot to keep it awake is a no-no. The next step up to get rid of the sleeping, is only $10 per month.</li>
<li>The ClearDB MySQL database is free only to 5 MB. This may not be enough for a long-running daily blog, but for a few postings per month maybe it's just what you need. The next step up is only $10 per month for 1 GB.</li>
</ul>
<br />
Since Heroku does support PHP, and a <i>mysqldump</i> of my database is only 2 MB in size, this sounds right on target for a Heroku freebie.<br />
<br />
<br />
<h3>
Ephemeral Filesystem</h3>
<br />
Let me reiterate that "permanent storage" comment above. Your Heroku filesystem is a git repository. Your dyno does have the ability to write files on disk, e.g. an uploads folder, e.g. the Media Library component of Wordpress. <i><b>But you won't like it.</b><b> When your dyno resets, the filesystem is reinitialized and your modifications would be lost.</b></i><br />
<br />
By "reset" is meant a reboot, it being load-balanced onto a new server, it going to sleep and waking back up, your next <i>git push</i>, or using <i>heroku config:set</i> to change environment variables. Bye bye, uploaded files.<br />
<br />
The up side of this is that a hack of your website, assuming it didn't damage the database, can be solved by rebooting. The down side is that you need to find someplace else for long-term storage of any web-supplied files not in your repository (or else make a practice of manually downloading the files and adding to the repo? sounds awful)<br />
<br />
In the case of Wordpress the easiest solution I found is the <a href="https://wordpress.org/plugins/amazon-s3-and-cloudfront/" target="_blank">WP Offload S3 Lite</a> plugin. When you upload to the Media Library, it goes to Amazon S3 instead, and media URLs are rewritten to point to their S3 version. Between Amazon's generous 5 GB free tier for a year and the real price being a paltry 3 pence per gigabyte per month, even an image-heavy website can get by on pocket change per month.<br />
<br />
If you are writing your own ware, you'd want to code for your cloud storage of choice such as Amazon S3, Google Drive API, Dropbox API, etc. where you supply a file and get back a URL. I imagine you'd need to generate API keys, program the OAuth-style exchange of them, handle errors etc. and that sounds awful. For my own case here, though, the folks at Delicious Brains had generously done that heavy lifting for me.<br />
<br />
<br />
<h3>
Happy Ending</h3>
<br />
Spoiler time: My applications are all running on Heroku and the performance is quite acceptable. And I learned a lot in the process. The rest of this story is how I got to this happy place.<br />
<br />
<h4>
So, now that I've covered the basics, tomorrow's story will be about the migration process! </h4>
Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-67092346970193626812016-02-15T12:41:00.002-08:002016-02-15T12:42:34.930-08:00Some Leaflet controls for modal dialogsA recurring need we have, is to open a dialog or a modal from within a Leaflet map. Dialogs and modals are a great way to add Feedback links, to show the legend without taking up screen space, to add a Help or About panel, etc.<br />
<br />
A perfectly ordinary way would be a positioned DIV anchored to the edge
of the screen or to the edge of the map DIV, so it looks like a "tab".
You've surely seen this before for Feedback links along the edge of the
screen. But this time we wanted a button in-map, something that looked like a real Leaflet button. It wasn't tough to do, cuz Leaflet rocks.<br />
<br />
Now... having done it, why not polish it up so that next time it's copy-paste simple? Here we go:<br />
<br />
<a href="https://github.com/gregallensworth/L.Control.BootstrapModal">https://github.com/gregallensworth/L.Control.BootstrapModal</a><br />
<a href="https://github.com/gregallensworth/L.Control.jQueryDialog">https://github.com/gregallensworth/L.Control.jQueryDialog</a><br />
<br />
I even took the step of making a jQuery UI version. The project last week didn't use this, but it was a small step to adapt the button code, and it will surely serve us in the future.<br />
<br />
Enjoy!<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-24903102696403505082016-01-08T18:22:00.000-08:002016-01-05T15:56:05.290-08:00IonicMapStarter, the Ionic successor to MobileMapStarterAt winter solstice 2012, I took a few days of vacation time and used it to create <b><a href="https://github.com/gregallensworth/MobileMapStarter" target="_blank">MobileMapStarter</a></b>. The intent was to boil down the essential and generally-reusable bits of a mobile mapping app, strip out the application-specific stuff, and have a starting place for our future map apps. This meant:<br />
<ul>
<li>A working mobile framework, with page-view management and all</li>
<li>Working around certain bugs and issues of Leaflet when used inside page-view systems</li>
<li>Working boilerplate code for geocoding, location tracking, etc. </li>
<li>The thing being designed with configuration separate from execution so it's simple to reconfigure</li>
<li>Ability to cache tiles for offline use</li>
</ul>
And it worked out famously. I presented MobileMapStarter at FOSS4G 2014 (welcome to Portland!) and was surprised at how popular it became.<br />
<br />
But, jQuery Mobile has problems and I have been looking for a newer framework to replace it. Some weeks back I really got to like Ionic, and winter solstice 2015 has brought you...<br />
<br />
<div style="text-align: center;">
<span style="font-size: large;"><b>IonicMapStarter </b></span></div>
<div style="text-align: center;">
<span style="font-size: large;"><b><a href="https://github.com/Greeninfo-Network/IonicMapStarter" target="_blank">https://github.com/Greeninfo-Network/IonicMapStarter</a></b></span></div>
<br />
<br />
It's the same concept as before: a mobile app scaffold, that's easy to reconfigure and adapt to form your own mobile app. But this time it's built with Ionic. I mentioned a few weeks back some improvements which this change brought to ParkInfo Mobile:<br />
<ul>
<li>Better performance all around, from the map to general page-loading and panning behavior</li>
<li>Cleaner structure from the ground up, and a reduction in code volume</li>
<li>Improved UI for the offline tile caching, as well as improved capabilities</li>
</ul>
We still aren't releasing ParkInfo Mobile until we finish some branding<br />
and functional tweaks, but you can start on your next-generation mobile app tonight.<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-81770645719150295702016-01-04T14:06:00.001-08:002016-01-04T14:06:23.305-08:00JSHint + AngularJS = Tight as a drumSo, in the development of the new version of ParkInfo Mobile, I wanted to run things through some code-quality checks as a matter of course. I'm good at what I do, but having a second pair of eyes (cybereyes!) look for missing semicolons etc. sure won't hurt.<br />
<br />
I went with <a href="http://jshint.com/install/" target="_blank">JSHint</a> and I really like the results.<br />
<ul>
<li>JSHint noticed a few stragglers such as semicolons, trivial stuff that could lead to larger goofs. Making use of JavaScript statement-chains crossing lines, means that a stray semicolon or a missing semicolon cause truly bizarre malfunctions. For this reason I prefer not to use the multi-line syntax, but with Angular it does happen a lot so this extra check is nice.</li>
<li>JSHint also complains about undeclared variables, e.g. L which is declared in leaflet.js and angular which is declared in angular.js These weren't errors at all but are easily permanently silenced... and could have been invaluable if the "undeclared global" were actually a typo.</li>
<li>JSHint also reports unused variables. In most cases this was a callback that receives an error object, and we don't use the error object. But in a few cases it was dependency injections which were no longer in use. So a second use of JSHint was to check for unused dependencies and for erroneous undeclared dependencies. Very nice.</li>
</ul>
So, I threw together a quick-n-dirty shell scrip to run everything through JSHint, and I run it when I get ready to push.<br />
<br />
Step 1, install JSHint via npm. You're used to this if you use Cordova:<br />
<br />
<blockquote class="tr_bq">
npm install -g jshint </blockquote>
Step 2, set up this script. Heck, add it to your source archive: <br />
<blockquote>
#!/bin/sh<br /><br />for js in www/index.js www/controllers/*.js ; do<br /> echo "********** CHECKING $js"<br /> echo ""<br /><br /> jshint $js<br /><br /> echo ""<br />done</blockquote>
It's dead simple, nothing special... but it does keep things a bit tighter, and makes for easy and automated checks for some common goofs.<br />
<br />
Now, the other tool commonly used here is JSLint. But so far I'm not impressed with it. Reporting unused variables and potential typographical errors is great, but JSLint reports truly stupid stuff such as having a space after the colon in an object literal, stuff which truly has no impact nor potential for impact. Maybe later, but not today.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-91772816591906901042015-12-23T23:15:00.002-08:002015-12-23T23:15:54.003-08:00A new ParkInfo Mobile using Ionic & AngularJSIt's been three years this week, since I wrote my first mobile app: ParkInfo Mobile. And this week, the new one is still under wraps for some branding and color choices, but is functional. ParkInfo Mobile 2.0.0 should be in the app stores in a week or two, replacing my Christmas 2012 edition.<br />
<br />
I'm not particularly sentimental, but it's a big step forward in a few ways. Let me share a few interesting changes in the new version, and lessons learned by writing it.<br />
<br />
<h2>
What's New</h2>
<br />
First and perhaps largest: <b>It's written in Ionic, and thus AngularJS.</b> This is a paradigm shift from jQuery Mobile, in which the DOM and global variables rule everything. The code is broken into a bunch of bite-sized segments: each panel is one HTML page and one paragraph of code, though some of them refer to the "global" functions for some commonly-used functions and needs.<br />
<br />
<b>It doesn't use an onboard JSON file, but HTTP hits to a web service.</b> The original ParkInfo Mobile had a JSON file included, which could be used for searching and estimating distance without using the data connection. But it kinda sucks. Including the polygons made the file about 100 MB and would crash any phone, so it didn't happen. Using only the centroids means that we can estimate distance and direction to the centroid; for a city park that's not too shabby, but for a 500,000-acre preserve the centroid could be miles away and be misleading. The new one uses a web service to query the database; it's swift, simple, and effective. Better, the web service can accept a hint as to your origin lat-long and return the <i>closest point</i> to each of your results, greatly improving the accuracy and usefulness of the results.<br />
<br />
<b>The performance is fabulous, and look and feel is smoother.</b> They said that AngularJS and Ionic have decent performance, and I know that jQuery Mobile kinda sucks... but dang, it's really <i>that much improved</i>. Navigation is smoother, more fluid. The default styling doesn't include so many drop shadows and rounded corners, so the whole thing feels more bare... if a bit cartoonish with those colorful buttons. (Like I said, the colors and branding stuff is still pending.)<br />
<br />
<b>The UI for offline tile caching is improved.</b> The underlying technology is much the same: Cordova's file API, with ngCordova wrappers. But the UI for caching is greatly improved.<br />
<ul>
<li>Caching by current map view is disabled, in favor of two new options: cache around a street address, or around your current location. This is a lot more useful, generally speaking.</li>
<li>"Passive caching" will cache your map tiles to device storage while you browse the map. If a specific address or your GPS location don't fit what you need, just let the cache populate as you pan and zoom.</li>
<li>While downloading tiles, the UI doesn't block you with a modal, but starts up a progress bar and lets you go on your way. Downloading 1000 tiles can take a few minutes, but you're not waiting around.</li>
</ul>
<br />
<h2>
Lessons, Hacks and Limitations</h2>
<h3>
</h3>
Some of the things that went less smoothly: <br />
<br />
<b>When you need two controllers to know each others' state, AngularJS is a pain in the neck.</b> The cache panel needs a list of map layers, but it's the map controller which has the Leaflet object and the L.TileLayers. Ultimately I really did need to separate out the layer listing into a <i>Constant</i> where it could be seen by both. Angular's all about not believing in globals, but dogma had to take a seat while pragmatism prevailed.<br />
<br />
<b>Ionic's sidemenu is cute, but not flexible.</b> It took me only 2 hours to come up with a thing that our clients will demand that it cannot do: put an icon in the top-right corner, which varies on every page (different icon & link, or none). I tried everything from position:fixed to negative top margins, even tried a nested view in the titlebar, and it's a no -- that titlebar isn't part of the dynamic part of the view. So I went the way I knew I would eventually, started with a blank template where each page has its own titlebar so we have maximum flexibility. I knew it would happen since our capricious designers are the ones calling the shots, but it irked me that those "copy-paste my code, and you're a mobile developer" articles basically have someone copy-paste the <i>sidemenu</i> starter, knowing that it paints you into a corner less than 2 hours later.<br />
<br />
<b>I've still not wrapped my head around Services and writing my own promise chains.</b> The web service calls would ideally be a Service instead of a function defined in $rootScope. But a service which accepts callbacks as parameters, which makes a $http call and hands off to those callbacks just eluded me. After a day, I needed to just get past it so I gave up and went with a $rootScope function that accepts a pair of callbacks, without a promise involved. Similarly, I couldn't figure out a promise chain to wrap a dynamically-generated list of potentially thousands of Cordova File API calls and perform them sequentially, so I went with my old "i+1" method. Then again, other reading indicates that a chain of a few thousand promises maybe isn't such a hot idea anyway...<br />
<br />
<b>Debugging is a real nuisance</b> when there aren't any globals, and you can't see into scopes private data. Keeping everything in tiny secret boxes, doesn't do well toward being able to type <i>MAP.getCenter()</i> into the Firebug Console to find out something so simple -- and that makes all debugging just that much slower. I need to put in some effort, into figuring out techniques for debugging AngularJS apps aside from <i>console.log()</i> inside these isolated black boxes.<br />
<br />
<b><a href="http://tombatossals.github.io/angular-leaflet-directive/#!/" target="_blank">Angular-leaflet-directive</a> is pretty limited, and I've had to do a bunch of work to it myself.</b> I'll likely fork it and start adding my own patches into it in a more concerted manner. But some of the adjustments I've discovered and invented include:<br />
<ul>
<li>You must use in in a ion-view with <i>overflow-scroll="true"</i> or else the map will cease to receive click events if you navigate away from it. It's not just the map, it's something about the underlying DIV. Those "copy-paste your first mobile app!" postings don't mention stuff like that, eh?</li>
<li>It doesn't support popups, so I added a <i>popup</i> bit to it.</li>
<li>It doesn't propagate <i>bounds</i> changes back to the scope, only <i>center</i>. Fixed that. </li>
</ul>
<h3>
</h3>
<h2>
The Future of MobileMapStarter</h2>
<br />
My second mobile app was actually stripped down from my first: ParkInfo Mobile's map/cache/page functionality was stripped down to form <a href="https://github.com/gregallensworth/MobileMapStarter" target="_blank">MobileMapStarter</a>.<br />
<br />
Last year after I presented MMS at FOSS4G it got some popularity, but I've been less than 100% satisfied with it. It's jQuery Mobile so it's slow, the cache UI isn't so hot, etc.<br />
<br />
But here we have a new framework with a map, panels, and caching. If I strip this down a bit, it's a fresh new start for "MobileMapStarter, Ionic Edition". Schedule permitting, maybe I can have that done in a couple of weeks.<br />
<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-45845567355225131052015-11-30T20:52:00.002-08:002015-11-30T20:52:18.500-08:00Ionic and AngularJSIn my personal time, I have one mobile app that I manage: BattleTech Calculator. It's a calculator for an old 1980s board-and-dice game, makes a lot of the tedious math move more quickly so we can focus on pretending to blow up our giant robots.<br />
<br />
I went with Framework7 for this one, on grounds that it looked like a winner. It's HTML/CSS with some DOM manipulation capability similar to jQuery. It has basic routing, doesn't look half bad. And BTC being a non-work app, I figured there's no harm if Framework7 turned out not to work out well.<br />
<br />
<h3>
Trouble With Framework7</h3>
<br />
Ultimately, it did not work out well.<br />
<b><br /></b>
<b>First was the freezing/crashing issues.</b> I thought it was just my phone because the other two folks didn't have any problem... until they reported that they did. Just switching between tabs in the usual fashion, it would freeze solid and you'd have to terminate the app and start over.<br />
<br />
<b>Second was a reported visual rendering bug.</b> I could never replicate it myself, but I got several reports in the Store after one update, that when they switch tabs the old tab's content would be visible on top of the new tab's content. Again, I couldn't make it happen but the reports were consistent enough that, combined with the freezing, I decided to try something new.<br />
<br />
<b>And that's not including</b> some documentation inaccuracies, the $$ and $ bug, and a few other quirks and shortcomings.<br />
<br />
Ionic Framework is based on AngularJS, and on the web side of things that's a totally new thing for me. As I described in prior blog posts searching for a HTML framework, Angular is designed around concepts like modules and components, instead of the designer-driven daily-spec-change chaos which typifies a lot of our web-map applications. But in this case, I did have a solid idea of what I wanted, and even a working prototype in my old version.<br />
<br />
<i>It took some getting used to... but Ionic/Angular kinda rocks.</i><br />
<br />
<h3>
DOM manipulation and AngularJS</h3>
<br />
DOM manipulation is out the window. Because of data-binding on the form widgets, the <i>ng-show</i> and <i>ng-hide</i> directives allow other stuff to show and hide as the form is manipulated, allow for recalculations to happen any time the form is changed, etc. <i>The result is more intelligent form widgets and less code and code repetition.</i><br />
<br />
<b>Old</b>, a separate piece of logic to show only the applicable table:<br />
<blockquote class="tr_bq">
var which = $(this).val();<br />$('#tab-meleehitlocation span.meleetype').hide();<br />$('#tab-meleehitlocation span.meleetype-'+which).show();</blockquote>
<br />
<b>New</b>, the table is intelligent as to its visibility condition:<br />
<blockquote class="tr_bq">
<<span class="pl-ent">div</span> <span class="pl-e">ng-show</span>=<span class="pl-s"><span class="pl-pds">"</span>viewdata.meleetype == 'punch'<span class="pl-pds">"</span></span>> Lorem Ipsum </div></blockquote>
<br />
<b>Old</b>, remove the highlight from all rows, add to this one:<br />
<blockquote class="tr_bq">
$(<span class="pl-s"><span class="pl-pds">'</span>#tab-meleehitlocation span.meleetype-punch div.row<span class="pl-pds">'</span></span>).<span class="pl-en">removeClass</span>(<span class="pl-s"><span class="pl-pds">'</span>hit<span class="pl-pds">'</span></span>).filter('<span class="pl-s">[data-roll="<span class="pl-pds">'</span></span><span class="pl-k">+</span>rolled<span class="pl-k">+</span><span class="pl-s"><span class="pl-pds">'</span>"]<span class="pl-pds">'</span></span>).<span class="pl-en">addClass</span>(<span class="pl-s"><span class="pl-pds">'</span>hit<span class="pl-pds">'</span></span>);</blockquote>
<br />
<b>New</b>, the row knows when it should be highlighted: <br />
<blockquote class="tr_bq">
<<span class="pl-ent">div</span> <span class="pl-e">class</span>=<span class="pl-s"><span class="pl-pds">"</span>row<span class="pl-pds">"</span></span> <span class="pl-e">ng-class</span>=<span class="pl-s"><span class="pl-pds">"</span>{ highlighted:viewdata.outcome == 2}<span class="pl-pds">"</span></span>></blockquote>
This data-binding and implicit state awareness is a key point of what makes AngularJS what it is, but felt very foreign and difficult to wrap my head around at first. But once it sunk in... I'm impressed. I think that AngularJS's "state propagation intelligence" could achieve any sort of unpredictable DOM manipulation we could need, in a more intelligible fashion than our old jQuery implementations.<br />
<br />
<br />
<h3>
Ionic Framework and Ionic View</h3>
<br />
Ionic Platform itself is a set of tools based on Cordova/Phonegap, but adding their special sauce of CSS utility classes, button colors, a set of icons, etc. And yet, I was allowed to ignore and override to suit a few specific needs, e.g. a SVG dice icon, in lieu of FontAwesome which did work just fine.<br />
<br />
So as far as the HTML/CSS flexibility, Ionic really did work well here.<br />
<br />
And then there's <b>Ionic View</b>, Ionic's test-deployment platform. It's similar in theory to the Phonegap Developer App:<br />
<ul>
<li>Your testers download the Ionic View app and sign in to the Ionic View service, and see a list of app which they have been invited to test;</li>
<li>You type <i>ionic upload</i> to upload your <i></i>content to the Ionic View servers;</li>
<li>Testers re-sync your app, and re-launch and test. No code signing, OSX builds, etc.</li>
</ul>
This significantly accelerated my testing cycle, even as one single person on a weekend project. The uploads are as fast as Xcode compiling it and far faster than dropping the APKs onto Google Drive and waiting for them to sync. And since it's over data (meaning, while I'm at home, over wifi) I can test it as I walk around, refill my coffee, etc. without ADB pitching a fit and forgetting to reconnect the device, etc.<br />
<br />
So yeah, Ionic's non-Angular nature... also a winner.<br />
<br />
<br />
<h3>
What It Means For BattleTech Calculator</h3>
The new version of BTC isn't finished yet; I have one whole panel to go. But let's look at bits of logic and overall code volume, and see what Ionic has helped me to improve.<br />
<br />
<a href="https://github.com/gregallensworth/BattleTechCalculator/tree/master/www">https://github.com/gregallensworth/BattleTechCalculator/tree/master/www</a><br />
<a href="https://github.com/gregallensworth/BattleTechCalculator-Old/tree/master/www">https://github.com/gregallensworth/BattleTechCalculator-Old/tree/master/www</a><br />
<br />
<ul>
<li>The very large HTML is broken into partial views. This keeps me visually focused, which is great if you have "situational ADD" aka being interrupted all the time.</li>
<li>HTML volume is currently 609 lines compared to 1125 lines. Add another 125 for the last page, and this is still a 40% reduction.</li>
<li>JavaScript code volume is 295 lines (not counting that unfinished panel, maybe 150 more) compared to 522. This would be a mere 20% reduction... but still 100 fewer lines of code. This is largely due to the removal of DOM-fussing since AngularJS takes care of that automagically.</li>
<li>Almost every line of code is simply more readable: default settings are declared in the controller instead of as <i>checked</i> attributes, code is more logical since it doesn't read from fields and do the DOm shuffling.</li>
<li>The code is even easier on the eyes, since $('#') is absent.</li>
<li>The app runs very smoothly for the most part, under Ionic View, on my old Galaxy S3. Smoother than with Framework7.</li>
<li>The app hasn't crashed once yet, in a few days of development and testing. This counts for a lot!</li>
</ul>
<br />
<h3>
What it Means For Work</h3>
<br />
Virtually all of our work is websites, and the designers use mobile-unfriendly layouts with floating panels and the like. So Ionic won't be doing a lot there; it can scrape by for websites, but support for IE isn't so hot. But <i>for the few mobile apps we do each year, we have a winner.</i><br />
<br />
For websites, I am starting to get into the feel of AngularJS. <i>If we can combine this with something visual like Bootstrap, it could be very neat.</i> The big challenge there, of course, is that our clients and designers don't have Bootstrap on the brain so we end up rewriting large parts of the visuals anyway. But for a general system of DOM management and data management, AngularJS is looking pretty sweet, and I'll be working with it some more on personal projects.<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-5639700332687677802015-10-13T15:35:00.001-07:002015-11-30T20:53:40.514-08:00Census blocks and attributes: An easier wayHas it really been almost 5 months since my last posting? I've been doing plenty, but not a lot of it is so inspired <b>but also of wide enough interest</b> that I've felt it relevant to share.<br />
<br />
But if you work with census data, this may save your day:<br />
<a href="https://github.com/greeninfo/CensusShapefileMaker">https://github.com/greeninfo/CensusShapefileMaker</a><br />
<br />
It does a day's worth of census downloading, joining, trimming, and calculation... in 10 to 15 minutes (most of that being the FTP download).<br />
<br />
<h3>
</h3>
<h3>
History</h3>
<h3>
</h3>
I created this for a client. Her specific use case, is to have census blocks for a whole state (though sometimes one county), joined to a few attribute tables so we have statistics such as number of people, number of people who are black/white/etc, number of people who are under 18, and so on. We'd then derive some statistics such as "number of people under 18" which itself means adding up 8 attributes then dropping those 8 since we really only wanted their sum. And of course, we only want a dozen or so columns but those attribute tables are huge, so she'd drop hundreds of columns with unintuitive names such as p61ia She would do this process in ArcMap and it would take an entire day or sometimes two, and was highly error-prone.<br />
<br />
So, I sat down with <a href="http://mcdc.missouri.edu/applications/uexplore.shtml" target="_blank">MCDC's Dexter</a> and <a href="ftp://ftp2.census.gov/" target="_blank">the USCB FTP</a> and figured things out. <br />
<h3>
</h3>
<h3>
A few tricks and paths</h3>
<ul>
<li>Dexter allows one to specify only specific attribute fields to be downloaded, and allows downloads of both decennial census and annual ACS. This is great, since it makes the CSV files so much simpler to parse: they have the fields you asked for.</li>
<li>I did try having <b>ogr2ogr</b> perform the join between the CSV and the shapefile -- after a couple of hours it was obviously not getting anywhere. So the technique is to open the shapefile via Python <b>ogr</b> and loop over records, assigning their attributes. For performance, it loads the whole attribute table into a <i>dict</i> which sounds bad, but in reality works fine since modern laptops have 8 GB of RAM.</li>
<li>Good ol' <b>ogr2ogr</b> is still used for stripping out attributes, e.g. the stock shapefiles have extra attributes we didn't care for. This could have been done in Python <b>ogr</b> as well, but was already done in <b>ogr2ogr</b>.</li>
<li>We wanted a YOUTH field indicating number of people under age 18, but the <b>Age By Sex</b> table has that broken over 8 fields. So at the merge phase, that <b>youth</b> sum is what it loads from the CSV in lieu of the 8 individual fields.</li>
<li>Median Household Income (MHHINC) comes from ACS and not from decennial, and comes at the tract level instead of the block level. Fortunately, the hierarchical naming convention for the GEOID makes it dead simple to figure out which tract corresponds to a given block. During the merge phase, the first 11 digits of the tract are used to look up the corresponding tract -- easy.</li>
<li>As an afterthought, the thing can also strip down to one specific county (identified by its FIPS code, in the hierarchical GEOID) if you didn't want the whole state. This is done at the very end, since frankly the processing is so fast that the seconds you'd save by pruning it first aren't worth the programming trouble.</li>
</ul>
So yeah, if you work with census data a lot you may find this one useful. Enjoy!<br />
<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0tag:blogger.com,1999:blog-5932295847698688808.post-58823120037218232712015-05-27T15:16:00.001-07:002015-05-27T15:16:40.214-07:00Marker "spotlight" for Leaflet and for Google Maps APIA client really liked <a href="http://modestmaps.com/examples/spotlight.html" target="_blank">this spotlight effect</a> demonstrated for ModestMaps. It calls out the selected markers, without hiding the other markers or forcing the user to perform a new search. And hypothetically this would be a great feature for the legend, to mouse over the legend and have the matching markers "light up"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvnEU02sGLoukzAHbZN-1lFJvFXFfHD5z0nv6REnexNEYxKr4crVSnhNZUHCkjbTK9zy1-PmWiOYRhg1A80-7cEBPEL6YbXKfE21pIi_d6CyTg-Vtx9s-wECGWfCkCLZp74LS6JWNg/s1600/spotlight-screenshot.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvnEU02sGLoukzAHbZN-1lFJvFXFfHD5z0nv6REnexNEYxKr4crVSnhNZUHCkjbTK9zy1-PmWiOYRhg1A80-7cEBPEL6YbXKfE21pIi_d6CyTg-Vtx9s-wECGWfCkCLZp74LS6JWNg/s400/spotlight-screenshot.jpg" width="400" /></a></div>
<br />
But it didn't exist for Google Maps API (which the client uses), nor for Leaflet (which we use on 95% of our other projects). So I spent an afternoon coding this up:<br />
<a href="https://github.com/greeninfo/MapSpotlight">https://github.com/greeninfo/MapSpotlight</a><br />
<br />
All in all it was pretty easy, and even fun. I got to read about the HTML5 Canvas which I rarely get to work with outside of some charting libraries, and got to learn about Google Maps API's <b>OverlayView</b> capability, and the analogous <b>iLayer</b> interface for Leaflet.<br />
<br />
So yeah, go grab a copy and highlight some markers. :)<br />
<br />Gregor the Map Guyhttp://www.blogger.com/profile/14880809629054113501noreply@blogger.com0