Monday, January 27, 2014

ArcGIS API, Highcharts, and kickbutt elevation profiles - 4. Charting elevation data with Highcharts

This posting is #4 (and likely the final) in a series. Using the ArcGIS JavaScript API I made a really nice demo of dynamic elevation charts with cool mouse effects and all, and this series has been a tutorial/walkthrough from first principles up to geoprocessing. Now we're up to the fun stuff: charting the elevation using Highcharts, and adding cool interactive mouse effects. You really should read the previous posts, at also read up on the ArcGIS JavaScript API


Massaging The Elevation Data


The previous post left us with a "path" of our elevation data. This is a list of [x,y,z] tuples, each one being a map coordinate and elevation. Next step is to massage that list somewhat, to make a nice chart.

The X axis of an elevation profile for a trail, would be the distance that we have traveled by the time we reach that segment. The first segment of the path would have 0 mileage (X=0), and the 14th segment's mileage would be the sum of the lengths of the first 13 segments (X=2.5 for example, that being in miles).

The Y axis would be the elevation. We received the elevation in meters and will convert to feet (if you prefer meters, skip the conversion, right?).

At each point we also want a tooltip (Highcharts is cool that way) and that tooltip could tell us the elevation delta from our starting location, e.,g. "Elevation: 1234 ft, +14 ft relative to start"
// input: a Path given to us from the elevation service geoprocessor: a list of tuples, each one being X/Y/meters
// return: a list of points for charting and in USA units: { lon, lat, elevft, text, miles }
// this function does massaging to the input data, such as converting the elevation from meters to feet (we're in the USA)
// and adding up lengths to figure the distance traveled at the end of each segment; the length in miles is effectively the X axis of the chart
// added bonus: it also calculates the elevation delta from your start, which looks very good in tooltips
function makeElevationProfileFromPath(path) {
    // capture the elevation of the first node, so we can calculate elevation diffs for each point
    // e.g. "+23 ft from starting point"
    var start_elevft = Math.round(path[0][2] * 3.28084); // from meters to feet

    // create a list of points from the points given
    // keep a cumulative sum of distance traveled (meters) over segments so far; used to calculate "miles" as the X axis location for this node
    var points  = [];
    var total_m = 0;
    for (var i=0, l=path.length; i<l; i++) {
        var lon    = path[i][0];
        var lat    = path[i][1];
        var elevft = Math.round( path[i][2] * 3.28084 ); // from meters to feet

        // increment the total meters traveled when the hiker arrives at this node: that is, the distance from this node to the previous node
        // then express that as miles for the current node  (at this node, you have come X miles)
        if (i) {
            var plon = path[i-1][0];
            var plat = path[i-1][1];
            var slen = Math.sqrt( ((lon-plon)*(lon-plon)) + ((lat-plat)*(lat-plat)) );
            total_m += slen;
        }
        var miles  = 0.000621371 * total_m;

        // tooltip content: elevation is +- X feet relative to the starting node
        var delev  = Math.abs(start_elevft-elevft);
            delev = (elevft>=start_elevft ? '+' : '-') + delev;
        var text   = "Elevation: " + elevft + " ft" + "<br/>" + delev + " ft compared to start";

        // ready, stick it on
        points.push({ lon:lon, lat:lat, elevft:elevft, miles:miles, text:text });
    }

    // done!
    return points;
}

Basically, it accepts the list of 3-tuples and returns back a flat list of objects, each one forming a point on the chart. Each point has elevft which is the elevation in feet and thus the Y value, has miles which is the distance traveled and thus the X value, and has text which will be used as a spiffy tooltip. The points also have lat and lon so a chart point can be correlated to a location on the map, which will be handy when we add mouseover events to the chart. (yeah, lon and lat are misnomers; it's in the map's spatialReference)

References:
http://sampleserver6.arcgisonline.com/arcgis/rest/services/Elevation/WorldElevations/MapServer/exts/ElevationsSOE_NET/ElevationLayers/0
https://developers.arcgis.com/en/javascript/jsapi/geoprocessor-amd.html

Charting it

When the Elevation geoprocessor returns, we simply grabbed the path and called it good. Well, replace that with something that massages the path using makeElevationProfileFromPath() and then charts it using renderElevationProfileChart()
    elevsvc.submitJob(params, function (reply) {
        // success: grab the 1 path we were given back, convert it into chart-friendly points, then chart them
        var path;
        try {
            path = reply.geometries[0].paths[0];
        } catch(e) { alert("Elevation service didn't return an elevation profile."); }

        // two steps here: convert the path to points, then hand the points off for charting to the DIV with id="elevationgraph"
        // general principle of separating into steps, so we can debug them or mess with them separately
        var points = makeElevationProfileFromPath(path);
        renderElevationProfileChart(points,'elevationgraph');
    }, function (status) {
        //console.log('status ping');
    }, function (error) {
        alert(error);
    });

The renderElevationProfileChart() function is mostly just Highcharts' own innate awesomeness. We preprocess the data one more step to grab the minimum, because an elevation chart is relatively flat when you live at 8,500 feet and are trying to show a difference of +200 feet. And we rename the fields to properly be x and y and name as Highcharts expects.

// given a set of chart-friendly points as returned from makeElevationProfileFromPath() plot it via Highcharts
// this is straightforward Highcharts charting, with the only interesting magic being the series.mouseOver effect
// as you mouse over the chart, the lat & lon are noted from the moused-over chart point, and HIGHLIGHT_MARKER moves on the map
function renderElevationProfileChart(points,containerid) {
    // massage it into the "x" and "y" structs expected by Highcharts: lon & lat are extraneous (used for mouseover), X and Y are the axis position and values, ...
    // also keep track of the lowest elevation found, acts as our 0 on the chart
    var lowest = 1000000;
    var data   = [];
    for (var i=0, l=points.length; i<l; i++) {
        data.push({ x:points[i].miles, y:points[i].elevft, name:points[i].text, lon:points[i].lon, lat:points[i].lat });
        if (points[i].elevft < lowest) lowest = points[i].elevft;
    }

    // render the given set of points from makeElevationProfileFromPath() into a Highcharts graph
    // the idea is that we want to reuse code between various types of linear features that may have elevation, so we don't hardcode element IDs into the lower-level functions, you see...
    var chart = new Highcharts.Chart({
        chart: {
            type: 'area',
            renderTo: containerid
        },
        title: {
            text: 'Elevation Profile'
        },
        xAxis: {
            title: {
                text: 'Distance (mi)'
            }
        },
        yAxis: {
            title: {
                text: 'Elevation (ft)'
            },
            min:lowest,
            allowDecimals:false
        },
        legend: {
            enabled:false
        },
        tooltip: {
            crosshairs: [true,true],
            formatter: function () {
                return this.point.name;
            }
        },
        plotOptions: {
            area: {
                marker: {
                    enabled: false,
                    symbol: 'circle',
                    radius: 2,
                    states: {
                        hover: {
                            enabled: true
                        }
                    }
                }
            },
            series: {
                point: {
                    events: {
                        mouseOver: function() {
                            var point = new esri.geometry.Point(this.lon,this.lat,MAP.spatialReference);
                            if (HIGHLIGHT_MARKER.graphics.length) {
                                HIGHLIGHT_MARKER.graphics[0].setGeometry(point);
                            } else {
                                var outline = new esri.symbol.SimpleLineSymbol(esri.symbol.SimpleLineSymbol.STYLE_SOLID, new dojo.Color([0,0,0]), 1);
                                var symbol  = new esri.symbol.SimpleMarkerSymbol(esri.symbol.SimpleMarkerSymbol.STYLE_CIRCLE,10,outline,new dojo.Color(HIGHLIGHT_COLOR2) );
                                HIGHLIGHT_MARKER.add(new esri.Graphic(point,symbol));
                            }
                        }
                    }
                },
                events: {
                    mouseOut: function() {
                        HIGHLIGHT_MARKER.clear();
                    }
                }
            }
        },
        series: [{ name: 'Elevation', data: data }]
    });
}
Again, most of this is just how awesome Highcharts is. A lot of this is just configuring the axes and the tooltips (see the name field for the tooltip?).  The cool mouse interactivity is based on three things:
  • If your X axis were evenly spaced, your usual "data" would be a simple list of Y values, and Highcharts would assume that each datum's X was X+1  But each point can be an object, as in this case, where x and y are explicitly given as attributes. This is how you achieve a chart with unevenly spaced X values.
  • These point-objects can have any attributes, as long as at least x and y are present. We add the name attribute, which is then used in the tooltip callback, and the lat and lon attributes which are used in a series.point.mouseOver callback.
  • Wow, Highcharts has a really slick series.point.mouseOver callback. Just thought I'd point that out again. This callback specifically either adds or else updates a marker in HIGHLIGHT_MARKER, which is a GraphicsLayer. (this wasn't in earlier examples, go see the live demo). Tip: The code could be slightly more efficient if the symbol were defined ahead of time, so we don't construct the same symbol repeatedly.
And there you have it. Awesome elevation profiles from ESRI's elevation service, and it was easy!
Thanks for reading. If you enjoyed the series, post a comment and share with your friends.


Question: I renamed the fields in renderElevationProfileChart() to be x and y and name. Why didn't I just name them that in makeElevationProfileFromPath() ? Because I assume that the client specs will change, that some day they'll want those points to be used for other to-be-determined analyses, so I prefer easy-to-read names such as elevft instead of y, especially two years down the road when x and y aren't entirely self-explanatory (cuz they're certainly not map coordinates). Just a style tip for you, cuz two years from now you'll look at your own code and wonder what in the world you were thinking... It happens to all of us. ;)

References:
http://www.highcharts.com/demo/
http://api.highcharts.com/highcharts



Wednesday, January 22, 2014

ArcGIS API, Highcharts, and kickbutt elevation profiles - 3. Geoprocessing, Proxies, and the Elevation service

This posting is #3 in a series, using the ArcGIS JavaScript API to start with a very basic Hello World map, through some education on events, Graphics, and so on, and leading up to very excellent interactive elevation profile charts. If you haven't read the previous posts, at least read up on the ArcGIS JavaScript API

Proxy Servers

A quick note before we get started. You probably need to use a proxy program, to get around the browsers' same-origin restriction.

 Browsers refuse to accept data from a server (e.g. your geoprocessor) unless the URL is the same as your website. The ArcGIS JS API has workarounds in place so you don't need a proxy for very small requests (e.g. a Query with only 1 where clause and a very simple geometry) but a proxy is required for anything larger (e.g. when you create a long clause and/or more complex geometries).

You really should set up a proxy service:
  • Step 1: Install a proxy program. You can write your own (I have a previous blog post on that topic) or download one of ESRI's on their page about proxies.
  • Step 2: Define the proxyUrl like this:   esri.config.defaults.io.proxyUrl = "proxy.php";
Once you do this, requests that are too large will automagically be routed through your proxy. And if you set up your proxy correctly, the service calls will still work.

Geoprocessing Services

In the last posting, I showed how to use a Query to request info from an ArcGIS server. There are other services available via the API such as geocoders and directions services. I'll suffice with a simple example, as these services are well documented and you should be able to adapt the previous examples to ESRI's documentation and be off to a good start.
// a hypothetical GraphicsLayer, which may have polygons drawn onto it
// if you read the previous post, you know all about GraphicsLayer and Graphics  ;)
function findPolygonAcres() {
    polygons = DRAWN_AREAS.graphics;
    if (! polygons.length) return alert("No polygons showing.");
    unionAndArea(polygons);
}

// step 1: find the union of all the submitted polygons, so we don't double-count acres if two polygons overlap
// the service accepts a list of Geometry objects, returns a single Geometry which is the union of the ones you gave it
// we're passing in Graphics, and need to extract their Geometries
function unionAndArea(graphics) {
    var geoms = [];
    for (var i=0, l=graphics.length; i<l; i++) geoms.push( graphics[i].geometry );

    var geomsvc = new esri.tasks.GeometryService("http://sampleserver3.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer");
    geomsvc.union(geoms, function(singlegeom) {
        getPolygonArea(singlegeom);
    });
}

// step 2: given a single Geometry, ask the GeometryService to calculate the area
// note that the service will accept a list of polygons, but we did want to find the union
// so we don't double count AND I wanted to be flashy about cascading a geoprocessing result to another geoprocessor
function getPolygonArea(polygon) {
    var params             = new esri.tasks.AreasAndLengthsParameters();
    params.lengthUnit      = esri.tasks.GeometryService.UNIT_FOOT;
    params.areaUnit        = esri.tasks.GeometryService.UNIT_ACRES;
    params.calculationType = 'geodesic';
    params.polygons        = [ polygon ];

    // since we did a union, we know there's only 1 result, and can simply take item 0 from the returned list
    var geomsvc = new esri.tasks.GeometryService("http://sampleserver3.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer");
    geomsvc.areasAndLengths(params,function (areas_and_lengths) {
        var acres = areas_and_lengths.areas[0];
        alert('acres + ' acres');
    },function (error) {
        alert(error.details);
    });
}
Every GeometryService method has its own parameters and returns, and you'll want to spend some time with the documentation for whatever services you'll need in your application. Don't memorize the docs, as much as learn what services exist and learn the basic "input and callback" design pattern.

References:
https://developers.arcgis.com/en/javascript/jsapi/geometryservice-amd.html
https://developers.arcgis.com/en/javascript/jsapi/geometryservice-amd.html#union
https://developers.arcgis.com/en/javascript/jsapi/geometryservice-amd.html#areasandlengths


The Geoprocessor Task

The Geoprocessor task is to access some arbitrary geoprocessing endpoint. Typically these will be some custom geoprocessing service, to perform some calculation that ESRI's own services don't provide. Every custom geoprocessor will be different, by nature. For example, a geoprocessor could accept as parameters a point location and a dollar value, and return a list of Graphics which are houses within 10 miles within the price range. Or a geoprocessor could accept a list of polygon geometries, take the union internally and compare against the Census/ACS data, and return a structure of demographic attributes within that polygon (not necessarily Graphics at all).

Geoprocessors do have a .execute() method which hypothetically will submit the parameters and get back results in one call. But in reality I've always had to use the .submitJob() design pattern. It's slightly more complex, but for the services I've used it's more reliable than .execute().
// the generic design pattern for Geoprocessor, using submitJob
// this hypothetical geoprocessing service accepts a single Point geometry and a dollar amount
// and returns a list of all houses fitting the price range (a list of Graphics which could go onto the map)
var params = {};
params.dollars = document.getElementById('pricerange').value;
params.location = MAP.extent.getCenter();

var housefinder = new esri.tasks.Geoprocessor(SERVICE_URL);
housefinder.submitJob(params, function (results) {
    // param 2: success callback, with the parameter being the results structure
    // the structure depends on the service, and every service is different
    // this one says how many results came up, the largest square footage found, and a list of Graphics
    alert('Found ' + results.numresults + ' houses.' + "\n" 'Largest was ' + results.maxsqft + ' sq ft');
    for (var i=0, l=results.houses.length; i<l; i++) {
        GRAPHICS.add( results.houses[i] );
        document.getElementById('listing').innerHTML += results.houses[i].attributes.title;
    }
}, function (status) {
    // param 3: status callback; every few seconds the API will ping the service again to ask the status of the job
    // your service may be super spiffy and have useful messages such as "23 out of 55 processed" or "62% done"
    // and maybe your application would like to display that status in a popup
    console.log(status.message);
}, function (error) {
    // param 4: error callback
    alert(error.message);
});
 
References:
https://developers.arcgis.com/en/javascript/jsapi/geoprocessor-amd.html
http://sampleserver6.arcgisonline.com/arcgis/rest/services/Elevation/WorldElevations/MapServer/exts/ElevationsSOE_NET/ElevationLayers/0

The Elevation Geoprocessor




In our case, we're interested in ESRI's elevation geoprocessing service. It took a little fussing to figure out, but the inputs and outputs are:
  • Only one input param: geometries a list of esri.Geometry.Polyline objects
  • Output: an object with a .geometries attribute, one for each of your input Polylines; each geometry has a .paths attribute, a list of which corresponds to the paths in each Polyline; each path is a list of 3-tuples, each one being [ x, y, elevation ]  X and Y are in whatever coordinates you gave (usually the map's spatialReference) and Elevation is in meters.
In other words: You pass in a list of Polylines, it returns a list of lines and paths, but with Z information.
Question: When we query the trails, why not just have the trails server hand back the Z as part of the Query? We're already using .returnGeometry=true right? Answer: The Query made to the server will omit the &returnZ=true parameter if you include it. The ArcGIS JS API doesn't handle 3D data, all the way down to the esri.Geometry.Point, so even if you could modify the request over the wire, you'd get back data that the rest of the API can't handle. No, elevation data is a truly separate thing.
In our case we'll pass in exactly 1 line (the trail that's highlighted) and get back 1 geometry with 1 path, and there we go. And here it is. This builds on my previous code bites, where we created the MAP and the HIGHLIGHT_TRAIL GraphicsLayer.
// on a map click, make a query for the trail and then for its elevation profile...
dojo.connect(MAP, "onClick", function (event) {
    handleMapClick(event);
});

// handle a map click, by firing a Query
function handleMapClick(event) {
    // if the trails layer isn't in range, skip this
    if (! OVERLAY_TRAILS.visibleAtMapScale ) return;

    // compose the query: just the name field, and in this 50 meter "radius" from our click
    var query = new esri.tasks.Query();
    query.returnGeometry = true;
    query.outFields      = [ "NAME" ];
    query.geometry       = new esri.geometry.Extent({
        "xmin": event.mapPoint.x - 50,
        "ymin": event.mapPoint.y - 50,
        "xmax": event.mapPoint.x + 50,
        "ymax": event.mapPoint.y + 50,
        "spatialReference": event.mapPoint.spatialReference
    });

    var task = new esri.tasks.QueryTask(ARCGIS_URL + '/' + LAYERID_TRAILS );
    task.execute(query, function (featureSet) {
        handleMapClickResults(featureSet);
    });
}

// handle the Query result
function handleMapClickResults(features) {
    // start by clearing previous results
    HIGHLIGHT_TRAIL.clear();

    // grab the first hit; nothing found? bail
    if (! features.features.length) return;
    var feature = features.features[0];

    // highlight using the given vector geometry...
    var symbol = new esri.symbol.SimpleLineSymbol(esri.symbol.SimpleLineSymbol.STYLE_SOLID, new dojo.Color(HIGHLIGHT_COLOR), 5);
    HIGHLIGHT_TRAIL.add(feature);
    feature.setSymbol(symbol);

    // now make the geoprocessing call to fetch the elevation info
    // there's only 1 param: a list of geometries; in our case the list is 1 item, that being the feature we got as a result
    var elevsvc = new esri.tasks.Geoprocessor("http://sampleserver6.arcgisonline.com/arcgis/rest/services/Elevation/WorldElevations/MapServer/exts/ElevationsSOE_NET/ElevationLayers/0/GetElevations");
    var params = { geometries:[ feature.geometry ] };
    elevsvc.submitJob(params, function (reply) {
        // success: grab the 1 path we were given back, convert it into chart-friendly points, then chart them
        var path;
        try {
            path = reply.geometries[0].paths[0];
        } catch(e) { alert("Elevation service didn't return an elevation profile."); }

        // we now have a valid path, and want to massage it into chart-friendly format
        // more on that next time!
        console.log(path);
    }, function (status) {
    }, function (error) {
        alert(error);
    });
}
We're almost there! Clicking the map triggers a Query to find a trail under the click. The Query callback draws the returned Graphic onto the map (highlighting the trail) and then submits a geoprocessing request to the Elevation service. On a successful return, we have a single "path" which is a list of [x,y,z] tuples... and that's our elevation profile.

In my next posting, we'll massage the returned tuples into a nice chart-friendly structure, including miles traveled and elevation at each point, then chart is using my favorite chart system, Highcharts.

References:
https://developers.arcgis.com/en/javascript/jsapi/geoprocessor-amd.html
http://sampleserver6.arcgisonline.com/arcgis/rest/services/Elevation/WorldElevations/MapServer/exts/ElevationsSOE_NET/ElevationLayers/0

Saturday, January 18, 2014

ArcGIS API, Highcharts, and kickbutt elevation profiles - 2. Click events, Query tasks, and Graphics layers

This posting expands upon my previous posting, in which I showed you how to make a very basic Hello World map using ArcGIS JavaScript API. This is part 2 of 4, leading up to very excellent interactive elevation profile charts. If you haven't read the previous post, at least read up on the ArcGIS JavaScript API

Adding Events


ArcGIS JS API is based on Dojo, and in Dojo you "connect" events to handlers using .connect() method. For example, to add a click handler to the map would go like this:
// the click event is a standard DOM event, but has some extra attributes too
// event.mapPoint is a esri.geometry.Point instance
// https://developers.arcgis.com/en/javascript/jsapi/point-amd.html
dojo.connect(MAP, "onClick", function (event) {
    console.log( event.mapPoint );
});
The documentation lists the various types of events available to each of the classes, e.g. a Graphicslayer can accept a click event, and that event has a .graphic attribute indicating the specific Graphic that was clicked.

Basic Geometries


In OpenLayers and ArcGIS API, a geometry is something separate from a marker or a map feature: it's a set of coordinates in space, which can be read or set separately from the vector feature ("Graphic", see below). If you're coming from Leaflet, imagine the separation of a LatLng from a Marker, and keep in mind that this separation applies to Polygon and Polyline features.

In ArcGIS JS API, a Geometry also has a spatialReference  This is different from OpenLayers in which all features are presumed to have the same SRS as the map. As such, the constructor has an additional argument, this being the spatialReference ID.
// tip: 4326 is WGS94 latlon, 102100 is web mercator
var srs = new SpatialReference({ wkid: 4326 });
var point = new esri.geometry.Point(-106.83, 39.19, srs );
var rect = new esri.geometry.Extent(-106.88, 39.16, -106.79, 39.22, srs );

// tip: the map has a .extent attribute, and this has a .getCenter() method
// the Extent is an Extent instance like the one above, and getCenter() returns a Point geometry
console.log( MAP.extent );
console.log( MAP.getCenter() );
Reference: https://developers.arcgis.com/en/javascript/jsapi/geometry-amd.html

Adding Graphics


A Graphic is a vector feature, which has a geometry/location, attributes, and a symbolization. In this sense it's quite similar to OpenLayers.Feature.Vector. Leaflet's Path subclasses (Circle, Marker) are somewhat different as I described above, is that Leaflet tends to tightly integrate the geometry and the feature (again, think Marker vs LatLng).

In OpenLayers, a Graphic is not added directly to the Map but to a vector layer, and that layer is added to the map. ArcGIS JS API is similar: you add a esri.layers.GraphicsLayer to the map, then add a esri.Graphic to the layer. (Leaflet is unusual that a Path instance is treated as a Layer all on its own; think about a LayerGroup, and you're thinking on the same track as these "vector layers")
var DOT_COLOR  = [255,  0,  0];
var DOT_BORDER = [  0,  0,  0];


var DOTS = new esri.layers.GraphicsLayer();
MAP.addLayer(DOTS);

var point   = MAP.extent.getCenter();
var outline = new esri.symbol.SimpleLineSymbol(esri.symbol.SimpleLineSymbol.STYLE_SOLID, new dojo.Color(DOT_BORDER), 1);
var dotsym = new esri.symbol.SimpleMarkerSymbol(esri.symbol.SimpleMarkerSymbol.STYLE_CIRCLE,10,outline,new dojo.Color(DOT_COLOR) );
DOTS.add(new esri.Graphic(point,dotsym));
As you can see, this defines a "dotsym" symbol, and defines a point (the center of the map), then connects the two into a Graphic, then adds that Graphic to a previously-empty GraphicsLayer.

Exercise: Events + Graphics = Dots!


A quick demonstration to tie those two concepts together. When you click the map, the event's .mapPoint attribute supplies the geometry/location of the click. Combine this with a GraphicsLayer and you can litter your map with dots just by clicking.

HTML

<!DOCTYPE HTML>
<html>
<head>
    <!-- ArcGIS JS API and related CSS -->
    <link rel="stylesheet" href="http://js.arcgis.com/3.8/js/dojo/dijit/themes/claro/claro.css">
    <link rel="stylesheet" type="text/css" href="http://js.arcgis.com/3.8/js/esri/css/esri.css">
    <script src="//js.arcgis.com/3.8/"></script>

    <style type="text/css">
    #map {
        width:5in;
        height:5in;
        border:1px solid black;
    }
    </style>
</head>
<body class="claro">

    <div id="map"></div>

</div>

</body>
</html>
JavaScript
var MAP; // the esri.Map object
var OVERLAY_TRAILS; // an ArcGIS Dynamic Service Layer showing the trails
var HIGHLIGHT_MARKER; // Graphics layer showing a marker over the selected highlighted trail, e.g. on cursor movement over the elevation chart

var START_W = -106.88042;
var START_E = -106.79802;
var START_S =   39.16306;
var START_N =   39.22692;

var ARCGIS_URL = "http://205.170.51.182/arcgis/rest/services/PitkinBase/Trails/MapServer";
var LAYERID_TRAILS = 0;

var HIGHLIGHT_COLOR2 = [255,  0,  0]; // never mind the weird name, it'll make sense in a few more postings

require([
    "esri/map",
    "dojo/domReady!"
], function() {
    // the basic map
    MAP = new esri.Map("map", {
        extent: new esri.geometry.Extent({xmin:START_W,ymin:START_S,xmax:START_E,ymax:START_N,spatialReference:{wkid:4326}}),
        basemap: "streets"
    });

    // add the trails overlay to the map
    OVERLAY_TRAILS = new esri.layers.ArcGISDynamicMapServiceLayer(ARCGIS_URL);
    OVERLAY_TRAILS.setVisibleLayers([ LAYERID_TRAILS ]);
    MAP.addLayer(OVERLAY_TRAILS);

    // add an empty Graphics layer
    HIGHLIGHT_MARKER = new esri.layers.GraphicsLayer();
    MAP.addLayer(HIGHLIGHT_MARKER);

    // on a map click add a dot to the Graphics layer
    dojo.connect(MAP, "onClick", function (event) {
        // the clear() method removes all graphics from this Graphicslayer, so there'd only be 1 dot at a time
        //HIGHLIGHT_MARKER.clear();

        var point   = event.mapPoint;
        var outline = new esri.symbol.SimpleLineSymbol(esri.symbol.SimpleLineSymbol.STYLE_SOLID, new dojo.Color([0,0,0]), 1);
        var dot     = new esri.symbol.SimpleMarkerSymbol(esri.symbol.SimpleMarkerSymbol.STYLE_CIRCLE,10,outline,new dojo.Color(HIGHLIGHT_COLOR2) );
        var graphic = new esri.Graphic(point,dot);
        HIGHLIGHT_MARKER.add(graphic);
    });
}); // end of setup and map init

And there you go, a basic exercise in events, symbols, and Graphics.

References:
https://developers.arcgis.com/en/javascript/jsapi/map-amd.html#event-click
https://developers.arcgis.com/en/javascript/jsapi/point-amd.html
https://developers.arcgis.com/en/javascript/jsapi/graphicslayer-amd.html
https://developers.arcgis.com/en/javascript/jsapi/graphic-amd.html
https://developers.arcgis.com/en/javascript/jsapi/symbol-amd.html


Query Task


The Query task is used to make a query and find features on an ArcGIS REST endpoint. The most common use case (perhaps) is of identifying what is underneath the mouse click, but this can be done separately of any map, just to make a query.
// query one specific layer,  getting the NAME field for any trail where closed=0
// the success callback simply says how many results were found
var queryTask = new esri.tasks.QueryTask(ARCGIS_URL + LAYERID_TRAILS  );
var query     = new esri.tasks.Query();
query.returnGeometry = false;
query.outFields      = ['Name'];
query.where          = "Closed=0";

dojo.connect(queryTask, "onComplete", function(featureSet) {
    var howmany = featureSet.features.length;
    alert('found ' + howmany + ' results'):
});
queryTask.execute(query);
A featureSet result has a .features attribute, and this is a list of Graphic objects. Yes, the returned features aren't just geometries, but geometry-and-attributes already wrapped into a Graphic object, and almost ready to draw onto the map. The only thing missing is to define a symbol for the features (I mean, the Graphics) and you're ready to rock. More on that below.

Notes:
  • The returnGeometry flag to the Query determines whether the geometry is sent. If you only need attribute data (e.g. a list of trails) then set this to false and your queries will be much faster but your returned Graphics have a null geometry. If you don't intend to draw the returned vector features onto the map, definitely leave out the geometry.
  • There's a bug in 10.1 where queries will fail, if you submit the same query multiple times in the same session (reloading the page makes it work again). I like to add a random factor to my .where clause, as I'll show below.
  • You can do spatial queries such as "is contained within _____" by adding a .geometry attribute to your Query. This will also be illustrated below.

This example is a little more complex. It makes a Query, then plots the resulting Graphics onto the map, as well an pestering you with alerts as to what it found.
var MAP;
var OVERLAY_TRAILS;
var HIGHLIGHT_TRAIL;

var START_W = -106.88042;
var START_E = -106.79802;
var START_S =   39.16306;
var START_N =   39.22692;

var ARCGIS_URL = "http://205.170.51.182/arcgis/rest/services/PitkinBase/Trails/MapServer";
var LAYERID_TRAILS = 0;

esri.config.defaults.io.proxyUrl = "proxy.php";

require([
    "esri/map",
    "dojo/domReady!"
], function() {
    // the basic map
    MAP = new esri.Map("map", {
        extent: new esri.geometry.Extent({xmin:START_W,ymin:START_S,xmax:START_E,ymax:START_N,spatialReference:{wkid:4326}}),
        basemap: "streets"
    });

    // add the trails overlay to the map
    OVERLAY_TRAILS = new esri.layers.ArcGISDynamicMapServiceLayer(ARCGIS_URL);
    OVERLAY_TRAILS.setVisibleLayers([ LAYERID_TRAILS ]);
    MAP.addLayer(OVERLAY_TRAILS);

    // we'll want to highlight the trail, and to draw a marker linked to the chart; define those 2 graphics layers here
    // why not 1 layer with both graphics? easier to untangle this way, compraed to iterating over the features and finding which is the marker or line
    HIGHLIGHT_TRAIL  = new esri.layers.GraphicsLayer({ opacity:0.50 });
    MAP.addLayer(HIGHLIGHT_TRAIL);

    // on a map click, make a query for the trail and then for its elevation profile...
    dojo.connect(MAP, "onClick", function (event) {
        handleMapClick(event);
    });

    // now run a query for all non-closed trails,
    // rendering them to the map and annoying the user with alerts for every result
    var queryTask = new esri.tasks.QueryTask(ARCGIS_URL + LAYERID_TRAILS  );
    var query     = new esri.tasks.Query();
    query.returnGeometry = false;
    query.outFields      = ['Name'];
    query.where          = "Closed=0";

    var symbol = new esri.symbol.SimpleLineSymbol(esri.symbol.SimpleLineSymbol.STYLE_SOLID, new dojo.Color([255,255,  0]), 5);
    feature.setSymbol(symbol);

    // the success handler: iterate over the features (Graphics)
    // add them to the map, and also illustrate the use of the Graphics' .attributes property
    dojo.connect(queryTask, "onComplete", function(featureSet) {
        if (! featureSet.features.length) return alert('Nothing matched your search.');
        for (var i=0, l=featureSet.features.length; i<l; i++) {
            var feature = featureSet.features[i];
            alert(feature.attributes.Name);
            HIGHLIGHT_TRAIL.add(feature);
        }
    });
    queryTask.execute(query);
});
This example illustrates quite a lot of little details:
  • Basic map initialization, plus a GraphicsLayer
  • Performing a Query for records matching some criteria
  • Plotting the returned Graphics to the map and accessing their .attributes attribute


With a little imagination, you can see how to make use of Queries even if there's not a map involved. A returnGeometry=false Query for all Closed=0 trails could be used to populate a selection list or a list of all trails, fetched dynamically from the server. That's pretty cool, in that you don't need to maintain a "HTML copy" of which trails should be listed; fetch 'em from the server.

A quick mention of the Identify tool: ArcGIS JS API also has a IdentifyTask which functions quite similarly to the Query tool. However, it is based on the concept that you have clicked the map (much as a WMS GetFeatureInfo) and have a click position, the map's width and height, etc. For this reason, I often use a Query instead of a Identify, even if I'm clicking the map. The next section will show this in detail.

References:

http://gis.stackexchange.com/questions/52612/arcgis-server-10-1-inconsistent-querying-errors
http://forums.arcgis.com/threads/72894-Etags-and-Intermittent-QueryTask-Server-Errors


Bring It Together: Click To Query (not to Identify)


Our goal is to make kickbutt elevation profiles. In our case, the user would click the map and that would trigger a Query; the Query result would then be drawn onto the map, its attributes would be displayed into a table or HTML DIV, and so on.

I mentioned earlier the Identify tool, which is specifically designed to accept a map click and look up info at that location. But I prefer to use the Query tool, because it allows for more nuanced filtering (e.g. where Closed=0) and for more consistent program code: when the client wants to query a trail from a listing and not by a map click at all, your other Query would be very similar, and thus easier to debug or to keep in sync if you need to make changes to both "query modes".

And, if you use a Query you're still not missing out on location: a Query can accept a .geometry parameter to filter by location, and that location can even be fabricated without using a map click (e.g. if I were at this lat & lon, and wanted a circle of 5 miles, ...). The Identify tool, by contrast, uses the map's width and height and the screen location of the click, so you're really stuck with clicking, and the click must be in the viewport, and you miss out on killer stuff like "within a 5 mile circle of...".

Here's a simple click-to-Query demonstration. A click event has the Geometry.Point extracted and hands off to a generic "find what's at this location" handler. The neat part is that (unlike Identify) findWhatsHere() could be handed any arbitrary point.
// add to the MAP a click event handler
// it extracts the point that was clicked (a Geometry.Point, not a screen pixel)
// and hands off to a "query maker" for that location
// the beauty, is that findWhatsHere() could be fed any point location
dojo.connect(MAP, "onClick", function (event) {
    findWhatsHere(event.mapPoint);
});

// given an arbitrary point in the map's spatialReference (web mercator, from a click)
// buffer it out by 50 meters and make a Query for whatever's inside that square
function findWhatsHere(event) {
    // compose the query: just the name field, a 50 meter "square radius" from our click
    // how do we know it's meters? cuz the map's default spatialReference is web mercator which is meters
    var query = new esri.tasks.Query();
    query.returnGeometry = true;
    query.outFields      = [ "NAME" ];    query.where          = "Closed=0";    query.geometry       = new esri.geometry.Extent({
        "xmin": event.mapPoint.x - 50,
        "ymin": event.mapPoint.y - 50,
        "xmax": event.mapPoint.x + 50,
        "ymax": event.mapPoint.y + 50,
        "spatialReference": event.mapPoint.spatialReference
    });

    var task = new esri.tasks.QueryTask(ARCGIS_URL + '/' + LAYERID_TRAILS );
    task.execute(query, function (featureSet) {
        handleSearchResults(featureSet);
    });
}
With a little imagination you can make some improvements on this, and that's your homework should you choose to accept it:
  • findWhatsHere() could accept a radius parameter, and use that instead of the hardcoded 50. In the onClick handler you'd hardcode a 50, and the other hypothetical portion of code which would call findWhatsHere() could look up the radius from a select box, a saved personal personal preference, etc.
  • Or you could simply use map.extent and use the map's current viewport. If I were to do that, I wouldn't tie it to a Map's onClick event, but maybe to its pan-end and move-end events. Then, whenever the map is panned or zoomed, a new Query could be run. Then you'd get new results as you pan the map, kinda like Google Maps updates your results as you pan the map.
  • The query uses a rectangle centered on the location, and not truly a circle. As such it would return results that are close to the circle but technically outside it. If you're particular, instead you'd create a esri.geometry.Circle
References:
https://developers.arcgis.com/en/javascript/jsapi/query-amd.html
https://developers.arcgis.com/en/javascript/jsapi/circle-amd.html



Proxy Servers

A quick note before we move on.

 Browsers refuse to accept data from a server (e.g. your geoprocessor) unless the URL is the same as your website. The ArcGIS JS API has workarounds in place so you don't need a proxy for very small requests (e.g. a Query with only 1 where clause and a very simple geometry) but a proxy is required for anything larger (e.g. when you create a long clause and/or more complex geometries).

You really should set up a proxy service:
  • Step 1: Install a proxy program. You can write your own(I have a previous blog post on that topic) or download one of ESRI's on their page about proxies.
  • Step 2: Define the proxyUrl like this:   esri.config.defaults.io.proxyUrl = "proxy.php";
Once you do this, requests that are too large will automagically be routed through your proxy. And if you set up your proxy correctly, the service calls will still work.


So, this concludes #2 in the series. I hope you've enjoyed it.
Next up will be more advanced topics such as geoprocessing, and the elevation service.


Monday, January 13, 2014

ArcGIS API, Highcharts, and kickbutt elevation profiles - 1. Intro to ArcGIS JS API

My previous post was a teaser for this one: part 1 in a 4-part series on creating interactive elevation profile charts, using Highcharts and the ArcGIS JavaScript API. This series will refer often to this live demo, so you may want to have it open as you read this.

I was originally going to just post a few bites of code, and skip over the basics of the API. But I realized that my demo app isn't overly complicated, and can double as a tutorial on the ArcGIS JS API. So here you go: a little more complicated than Hello World, but a little more functional than "just use an ArcGIS Online application template"


Getting Started


If you're not familiar with it, you'll want to read up on the ArcGIS JavaScript API documentation. It's a surprisingly functional API, with access to a lot of cool services such as ESRI's geocoder, ESRI's elevation service, arbitrary ArcGIS geoprocessing endpoints, some good basemaps, etc.

Your basic map would come in these two parts, an HTML file and a JavaScript file. The code below covers version 3.8 of the API, which is current as of January 2014. 3.8 makes some minor changes from 3.7, most notably to dojo.require -- it now accepts the whole list of dependencies, and runs a callback after they're loaded.

The HTML

<!DOCTYPE HTML>
<html>
<head>
    <!-- ArcGIS JS API and related CSS -->
    <link rel="stylesheet" href="http://js.arcgis.com/3.8/js/dojo/dijit/themes/claro/claro.css">
    <link rel="stylesheet" type="text/css" href="http://js.arcgis.com/3.8/js/esri/css/esri.css">
    <script src="//js.arcgis.com/3.8/"></script>

    <!-- some minor styling, and our JavaScript code -->
    <script type="text/javascript" src="index.js"></script>
    <style type="text/css">
    #map {
        width:5in;
        height:5in;
        border:1px solid black;

        margin:0 auto 0 auto;
    }
    </style>
</head>
<body class="claro">

    <div id="map"></div>

</body>
</html>

The JavaScript (index.js)
var MAP;

var START_W = -106.88042;
var START_E = -106.79802;
var START_S =   39.16306;
var START_N =   39.22692;

var ARCGIS_URL = "http://205.170.51.182/arcgis/rest/services/PitkinBase/Trails/MapServer";
var LAYERID_TRAILS = 0;

require([
    "esri/map",
    "dojo/domReady!"
], function() {
    // the basic map, with a global reference of course
    MAP = new esri.Map("map", {
        extent: new esri.geometry.Extent({xmin:START_W,ymin:START_S,xmax:START_E,ymax:START_N,spatialReference:{wkid:4326}}),
        basemap: "streets"
    });

    // add the trails overlay to the map
    OVERLAY_TRAILS = new esri.layers.ArcGISDynamicMapServiceLayer(ARCGIS_URL);
    OVERLAY_TRAILS.setVisibleLayers([ LAYERID_TRAILS ]);
    MAP.addLayer(OVERLAY_TRAILS);
});


This is pretty minimal: load up Dojo and have it load its dependencies, and when that's done the callback will create a new  esri.Map bound to the DIV with id="map", with a specific starting zoom area (the extent).

It does go one extra step, though, and add a layer above the basemap. In this case, it's an ArcGIS REST service which will display hiking trails in the same area to which the map is zoomed. Just these two pieces, and you have your very first ArcGIS API map.


What? That's it?


I hate Hello World type applications, which claim to walk you through the API but which just hand you a working map and nothing else to go on. But you know what, in this case that really is it... for now. The next posting covers click events and the Identify task.

Wednesday, January 8, 2014

ArcGIS API, Highcharts, and kickbutt elevation profiles - Teaser

A recent website for a Bay Area parks agency, has some very good-looking elevation profiles. You pick a trail from the list, and an elevation profile appears: this great line chart going up and down to show the elevation as you would walk its length, AND as you mouse over the chart it draws a marker on the map to really give an idea of where you're talking about. Here's a clip from my version of it:



And a live demo courtesy of Github Pages:
http://gregallensworth.github.io/ArcGISElevationProfileCharting/

This article will span a few postings, and will describe step by step how I developed this. But for a spoiler, here's the big punchline: it doesn't use ArcGIS Online application templates, because we weren't able to get the extreme flexibility we needed for our demanding clients. Instead I did it with these parts:

- a basic map written in ArcGIS JS API
- a Geoprocessing call to ESRI's elevation service
- jQuery and the Highcharts charting library


The next few postings will be from the ground up: creating a basic map using the ArcGIS JS API, adding click event handlers and Identify tasks, and ultimately calling the elevation service and rendering a very dynamic chart.