Docker for Geoprocessing: Difference between revisions
Brian Wilson (talk | contribs) mNo edit summary |
Brian Wilson (talk | contribs) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 6: | Line 6: | ||
On my home computer [[Murre]] which runs Windows 10, I needed to install | On my home computer [[Murre]] which runs Windows 10, I needed to install | ||
* [https:// | * [https://code.visualstudio.com/ Visual Studio Code] for Windows with the extensions listed below | ||
* [https://docker.com/ Docker Desktop for Windows | * [https://www.docker.com/products/docker-desktop Docker Desktop], for Windows | ||
* [http://git-scm.com/ git for windows] because Visual Studio Code uses it | * [http://git-scm.com/ git for windows] because Visual Studio Code uses it | ||
* [http://cygwin.com/ cygwin] so I'd have bash, maybe you don't care about bash and use Powershell instead? | * [http://cygwin.com/ cygwin] so I'd have bash, maybe you don't care about bash and use Powershell instead? | ||
| Line 18: | Line 18: | ||
* Remote Development pack (includes Remote-Containers and Remote-SSH extensions) | * Remote Development pack (includes Remote-Containers and Remote-SSH extensions) | ||
== | === Python in VSCode === | ||
At home I have to install a virtual environment, in a Terminal, I do '''python3 -m venv .venv''' | |||
and then '''pip install -r requirements.txt''' | |||
Then I can run the debugger on local projects. | |||
=== Python in Docker container in VSCode === | |||
Start with a project that works like '''git clone https://github.com/Microsoft/vscode-remote-try-python''' | |||
When you open the project folder in VScode it should now offer to restart in Docker. Click "Restart" and wait while it downloads images and starts. | |||
Then open a Terminal and you should see a prompt similar to this: '''vscode@c52318f7383c:/workspaces/vscode-remote-try-python$''' | |||
Now you can set a breakpoint in app.py for example and run with F5. "print" statements will output to the TERMINAL window. | |||
==== Restart container ==== | |||
You can just restart VSCode, that's hardly any slower, but if you want, | |||
if you change your Dockerfile and you do this it will trigger a rebuild of the container, too. | |||
* F1 | |||
* Docker Container: Restart | |||
* Select the container with a familiar name (your project name will be embedded in it) | |||
* OKAY | |||
If there are errors, you can do "Reopen Locally", fix the errors, and then do "Restart in Container". | |||
== Building contours == | |||
The big picture is to create an elevation surface and then contours for Clatsop County from LiDAR data provided by DOGAMI. | The big picture is to create an elevation surface and then contours for Clatsop County from LiDAR data provided by DOGAMI. | ||
== Downloads == | |||
I created https://github.com/Wildsong/geoprocessing Wildsong/lidar_downloader] to download all the data for the county | |||
and got it working. This does no geoprocessing, it just grabs files from a remote server using a CSV full of URLs so it's not in a Docker (yet). | |||
Now I have data at home to work with. | |||
== Data == | |||
I unpacked the zips here: /d/GIS/OR/DOGAMI/ClatsopCounty | |||
== VRT mosaic == | |||
I am going to build VRT files using GDAL, so now I need the Docker. | |||
I've created a Github project called [https://github.com/Wildsong/geoprocessing Wildsong/geoprocessing]. | I've created a Github project called [https://github.com/Wildsong/geoprocessing Wildsong/geoprocessing]. | ||
The mosaic builder is called "surface". | |||
=== Source data in ArcInfo GRID === | |||
2020-03-26 For some reason, I have not been able to open the GRID files in ESRI products, but they open in QGIS so I am ignoring that problem. | |||
I think this is the GDAL driver to use [https://gdal.org/drivers/raster/aig.html?highlight=grid AIG driver] | |||
and there is additional information here [https://gdal.org/drivers/raster/arcinfo_grid_format.html Grid format] | |||
Latest revision as of 18:43, 26 March 2020
I am trying to get Geoprocessing jobs to run inside Docker so that they are more portable and easier to reuse across different machines.
COVID19 and WFH
COVID19 hit so I had to get set up to work at home. I had already been using git so my code was up there.
On my home computer Murre which runs Windows 10, I needed to install
- Visual Studio Code for Windows with the extensions listed below
- Docker Desktop, for Windows
- git for windows because Visual Studio Code uses it
- cygwin so I'd have bash, maybe you don't care about bash and use Powershell instead?
What else? I'll add as I think of it.
VSCode extensions that I currently have installed
- Debugger for Chrome
- Docker
- Python
- Remote Development pack (includes Remote-Containers and Remote-SSH extensions)
Python in VSCode
At home I have to install a virtual environment, in a Terminal, I do python3 -m venv .venv and then pip install -r requirements.txt
Then I can run the debugger on local projects.
Python in Docker container in VSCode
Start with a project that works like git clone https://github.com/Microsoft/vscode-remote-try-python When you open the project folder in VScode it should now offer to restart in Docker. Click "Restart" and wait while it downloads images and starts. Then open a Terminal and you should see a prompt similar to this: vscode@c52318f7383c:/workspaces/vscode-remote-try-python$
Now you can set a breakpoint in app.py for example and run with F5. "print" statements will output to the TERMINAL window.
Restart container
You can just restart VSCode, that's hardly any slower, but if you want, if you change your Dockerfile and you do this it will trigger a rebuild of the container, too.
- F1
- Docker Container: Restart
- Select the container with a familiar name (your project name will be embedded in it)
- OKAY
If there are errors, you can do "Reopen Locally", fix the errors, and then do "Restart in Container".
Building contours
The big picture is to create an elevation surface and then contours for Clatsop County from LiDAR data provided by DOGAMI.
Downloads
I created https://github.com/Wildsong/geoprocessing Wildsong/lidar_downloader] to download all the data for the county and got it working. This does no geoprocessing, it just grabs files from a remote server using a CSV full of URLs so it's not in a Docker (yet). Now I have data at home to work with.
Data
I unpacked the zips here: /d/GIS/OR/DOGAMI/ClatsopCounty
VRT mosaic
I am going to build VRT files using GDAL, so now I need the Docker. I've created a Github project called Wildsong/geoprocessing.
The mosaic builder is called "surface".
Source data in ArcInfo GRID
2020-03-26 For some reason, I have not been able to open the GRID files in ESRI products, but they open in QGIS so I am ignoring that problem. I think this is the GDAL driver to use AIG driver and there is additional information here Grid format
