Geoprocessing: Difference between revisions
Brian Wilson (talk | contribs) mNo edit summary |
Brian Wilson (talk | contribs) |
||
(32 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
In this case "Geoprocessing" means doing specialized spatial processing on data as required in my web maps work. | |||
This page used to cover geoprocessing in the ESRI world, and it was mostly about tips for Python+ArcPy. | |||
I rewrote it to document WPS (Web Processing Service) first in [[GeoServer]] and later on in ArcGIS Enterprise. | |||
I have separate pages on building and using GDAL and PostGIS, see the GIS category. | |||
I built a Docker Compose [[GeoServer]] for testing, and I have added plugins to it for WPS. | |||
I also have been developing an OpenLayers Javascript client. | |||
== Books == | |||
''Expert GeoServer'' by Ben Mearns, is a very short book (134 pages) covering primarily WPS and Tile Caching (two unrelated topics, both interesting). | |||
== Penn State class == | |||
I am walking through this full detailed lesson plan: [https://www.e-education.psu.edu/geog585/ Open Web Mapping] | |||
So far, very good! | |||
== Boundless tutorial == | |||
'''IGNORE''' should you run across this [https://suite.opengeo.org/docs/latest/processing/contour/setup.html Contour Map setup] from Boundless Geospatial. In the Boundless docs it's difficult for a beginnto tell what is a tool they provide (such as Composer) and what things are actually part of GeoServer. Theoretically it's all open source but in practice what you find on Github tends to be nonfunctional. | |||
* Composer supports creating maps using GeoServer, it has a style editor | |||
* WPS Builder is like Esri Model Builder | |||
== Browser-based geoprocessing == | |||
# | A couple options here are [https://github.com/bjornharrtell/jsts#readme jsts] and [http://turfjs.org/ turf]. (and Turf wraps JSTS) | ||
</ | |||
JSTS is a Javascript port of the Java Topology Suite (JTS). | |||
The Turf documentation wins though. And it's used in GeoMoose 3. (JSTS is a dependency so you get both if you use Turf.) | |||
=== Turf === | |||
Go to [http://turfjs.org/getting-started Getting Started] | |||
'''NB All coordinates in Turf have to be in WGS84.''' This is a minor pain. | |||
Also the default units in "buffer" is '''"kilometers"''' not "meters", what a surprise! :-) | |||
Sample (using Parcel packager) | |||
import buffer from '@turf/buffer' | |||
import { toWgs84, toMercator } from '@turf/projection' | |||
import jsonWM from './assets/mygeojson.json' // this data is in EPSG:3857, Web Mercator | |||
let json84 = toWgs84(jsonWM); | |||
let buffered = buffer(json84, 10, {"units":"meters"}); | |||
let bufferedWM = toMercator(buffered); | |||
https://blog.webkid.io/how-to-handle-geospatial-data-with-nodejs-and-turf/ | |||
== Server-based geoprocessing == | |||
=== CQL queries === | |||
You can create CQL filters and pass them in as part of the REST queries you send to GeoServer. | |||
You can also create new layers with CQL filters applied, for example you could create a new taxlots_astoria | |||
layer from taxlots with a CQL query of "CITY=Astoria", or in your browser app you could add "cql_filter=CITY=Astoria" to the URL. | |||
=== WPS servers === | |||
WPS Servers include GeoServer. | |||
Others are | |||
http://zoo-project.org/ and | |||
https://pywps.org/ | |||
Both of these are OSGEO projects, too. | |||
GeoServer WPS comes with a slew of predefined JTS tools. | |||
I found a good tutorial here: https://geoserver.geo-solutions.it/edu/en/wps/wps.html | |||
List available services on my server: | |||
https://geoserver.wildsong.biz/geoserver/ows?service=wps&version=1.0.0&request=GetCapabilities | |||
Describe the buffer service, in detail but also alas in XML (hard for a human to read): | |||
https://geoserver.wildsong.biz/geoserver/ows?service=wps&version=1.0.0&request=DescribeProcess&Identifier=JTS:buffer | |||
Code a request in request.xml and send it. | |||
curl -H "Content-type: xml" -X POST [email protected] https://geoserver.wildsong.biz/geoserver/wps -o response.json | |||
Now I have a JSON file and I can drag it onto my "example3" map to see it. | |||
== Use case: Find my neighbors == | |||
# User selects a taxlot in the Javscript client. | |||
# This causes adjacent taxlots to be selected. | |||
# Information about all selected taxlots appears in a table. | |||
First step, let's find the taxlot id of all our recently saved building footprints, I will pick one for the taxlot account number. | |||
1. Create a view that selects new buildings.<br /> | |||
2. Select all the taxlots that intersect those buildings and list their taxlot id numbers. | |||
CREATE OR REPLACE VIEW clatsop_wm.buildings_new AS SELECT * FROM clatsop_wm.buildings WHERE checked=3; | |||
SELECT tl.tax_id, tl.account_id FROM clatsop_wm.taxlots AS tl, clatsop_wm.buildings_new AS bldg WHERE ( | |||
ST_Intersects(tl.geom, bldg.geom) | |||
) ORDER BY tl.tax_id; | |||
Okay so let's pick one taxlot with account_id 25127, and find others near it. | |||
Using a distance of 30 meters picks up the taxlots across the street too. | |||
SELECT tl.taxlot, tl.account_id FROM clatsop_wm.taxlots AS tl WHERE | |||
ST_DWithin( | |||
(SELECT geom FROM clatsop_wm.taxlots WHERE account_id=25127), | |||
tl.geom, | |||
30 | |||
); | |||
[[File:neighbors.png]] | |||
[[Category:GIS]] |
Latest revision as of 19:11, 20 September 2019
In this case "Geoprocessing" means doing specialized spatial processing on data as required in my web maps work.
This page used to cover geoprocessing in the ESRI world, and it was mostly about tips for Python+ArcPy. I rewrote it to document WPS (Web Processing Service) first in GeoServer and later on in ArcGIS Enterprise.
I have separate pages on building and using GDAL and PostGIS, see the GIS category.
I built a Docker Compose GeoServer for testing, and I have added plugins to it for WPS. I also have been developing an OpenLayers Javascript client.
Books
Expert GeoServer by Ben Mearns, is a very short book (134 pages) covering primarily WPS and Tile Caching (two unrelated topics, both interesting).
Penn State class
I am walking through this full detailed lesson plan: Open Web Mapping So far, very good!
Boundless tutorial
IGNORE should you run across this Contour Map setup from Boundless Geospatial. In the Boundless docs it's difficult for a beginnto tell what is a tool they provide (such as Composer) and what things are actually part of GeoServer. Theoretically it's all open source but in practice what you find on Github tends to be nonfunctional.
- Composer supports creating maps using GeoServer, it has a style editor
- WPS Builder is like Esri Model Builder
Browser-based geoprocessing
A couple options here are jsts and turf. (and Turf wraps JSTS)
JSTS is a Javascript port of the Java Topology Suite (JTS).
The Turf documentation wins though. And it's used in GeoMoose 3. (JSTS is a dependency so you get both if you use Turf.)
Turf
Go to Getting Started
NB All coordinates in Turf have to be in WGS84. This is a minor pain.
Also the default units in "buffer" is "kilometers" not "meters", what a surprise! :-)
Sample (using Parcel packager)
import buffer from '@turf/buffer' import { toWgs84, toMercator } from '@turf/projection' import jsonWM from './assets/mygeojson.json' // this data is in EPSG:3857, Web Mercator let json84 = toWgs84(jsonWM); let buffered = buffer(json84, 10, {"units":"meters"}); let bufferedWM = toMercator(buffered);
https://blog.webkid.io/how-to-handle-geospatial-data-with-nodejs-and-turf/
Server-based geoprocessing
CQL queries
You can create CQL filters and pass them in as part of the REST queries you send to GeoServer. You can also create new layers with CQL filters applied, for example you could create a new taxlots_astoria layer from taxlots with a CQL query of "CITY=Astoria", or in your browser app you could add "cql_filter=CITY=Astoria" to the URL.
WPS servers
WPS Servers include GeoServer. Others are http://zoo-project.org/ and https://pywps.org/ Both of these are OSGEO projects, too.
GeoServer WPS comes with a slew of predefined JTS tools.
I found a good tutorial here: https://geoserver.geo-solutions.it/edu/en/wps/wps.html
List available services on my server: https://geoserver.wildsong.biz/geoserver/ows?service=wps&version=1.0.0&request=GetCapabilities
Describe the buffer service, in detail but also alas in XML (hard for a human to read): https://geoserver.wildsong.biz/geoserver/ows?service=wps&version=1.0.0&request=DescribeProcess&Identifier=JTS:buffer
Code a request in request.xml and send it.
curl -H "Content-type: xml" -X POST [email protected] https://geoserver.wildsong.biz/geoserver/wps -o response.json
Now I have a JSON file and I can drag it onto my "example3" map to see it.
Use case: Find my neighbors
- User selects a taxlot in the Javscript client.
- This causes adjacent taxlots to be selected.
- Information about all selected taxlots appears in a table.
First step, let's find the taxlot id of all our recently saved building footprints, I will pick one for the taxlot account number.
1. Create a view that selects new buildings.
2. Select all the taxlots that intersect those buildings and list their taxlot id numbers.
CREATE OR REPLACE VIEW clatsop_wm.buildings_new AS SELECT * FROM clatsop_wm.buildings WHERE checked=3; SELECT tl.tax_id, tl.account_id FROM clatsop_wm.taxlots AS tl, clatsop_wm.buildings_new AS bldg WHERE ( ST_Intersects(tl.geom, bldg.geom) ) ORDER BY tl.tax_id;
Okay so let's pick one taxlot with account_id 25127, and find others near it. Using a distance of 30 meters picks up the taxlots across the street too.
SELECT tl.taxlot, tl.account_id FROM clatsop_wm.taxlots AS tl WHERE ST_DWithin( (SELECT geom FROM clatsop_wm.taxlots WHERE account_id=25127), tl.geom, 30 );