Running GDAL scripts in ESRI Model Builder: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
mNo edit summary
Brian Wilson (talk | contribs)
mNo edit summary
Line 3: Line 3:


I am using the versions of Python, GDAL, and Shapely.
I am using the versions of Python, GDAL, and Shapely.
== Installing and using OSGEO4W ==
For 10.1 you should be able to use this, because arcpy is already bound to python 27. For 10.0???
* Install OSGeo4W (which includes Python 2.7.4) Get it from http://osgeo4w.osgeo.org/
* Re-run the installer and do an advanced install, selecting the file geodatabase plugin for Python.
* Update your environment to include these variables
OSGEOHOME        C:\OSGed4W
GDALHOME          %OSGEOHOME%
GDAL_DATA        %GDALHOME%\share\gdal
GDAL_DRIVER_PATH  %GDALHOME%\bin\gdalplugins
PYTHONHOME        %OSGEOHOME%\apps\Python27
PATH              %OSGEOHOME\bin;%PATH%;%PYTHONHOME\Scripts
* Create a file called Desktop10.0.pth and place it in your %PYTHONHOME\Lib\site-packages directory. Put these three lines in it
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Scripts
You should be able to open a cmd window and run Python, see that it's version 2.7.4 and type these lines without error
from osgeo import gdal
import shapely
import arcpy


== Installing gdal and shapely into ArcGIS Python ==
== Installing gdal and shapely into ArcGIS Python ==
Line 20: Line 44:
  PATH              %PYTHONHOME%;%GDALHOME%;%PATH%;%PYTHONHOME%\Scripts
  PATH              %PYTHONHOME%;%GDALHOME%;%PATH%;%PYTHONHOME%\Scripts


Create a file called ArcPy.pth and place it in your %PYTHONHOME\Lib\site-packages directory. Put these three lines in it (for ArcGIS 10.0)
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Scripts


== Wrong path for me: Virtualenv ==
== Wrong path for me: Virtualenv ==
Line 43: Line 62:
http://www.tylerbutler.com/2012/05/how-to-install-python-pip-and-virtualenv-on-windows-with-powershell/
http://www.tylerbutler.com/2012/05/how-to-install-python-pip-and-virtualenv-on-windows-with-powershell/


I had already installed OSGeo4W yesterday, and modified the environment on my desktop Windows 7 (virtual machine) to load it so that I can run python from command line shells. The link above gives help on how to do that so I won't cover it here.
Installation instructions, the compact edition:
* Install OSGeo4W (which includes Python 2.7.4) Get it from http://osgeo4w.osgeo.org/
* Update environment to include these variables
OSGEOHOME        C:\OSGed4W
GDALHOME          %OSGEOHOME%
GDAL_DATA        %GDALHOME%\share\gdal
GDAL_DRIVER_PATH  %GDALHOME%\bin\gdalplugins
PYTHONHOME        %OSGEOHOME%\apps\Python27
PATH              %OSGEOHOME\bin;%PATH%;%PYTHONHOME\Scripts
* Download and install distribute_setup.py '''wget http://python-distribute.org/distribute_setup.py'''
* Download and install distribute_setup.py '''wget http://python-distribute.org/distribute_setup.py'''
* Run it. It will install itself. '''python distribute_setup.py'''
* Run it. It will install itself. '''python distribute_setup.py'''

Revision as of 22:42, 14 June 2013

I have a script that runs fine standalone but I need it to run in ESRI Model Builder. I am doing this for a client. I avoid Model Builder because scripts are maintainable and reliable.

I am using the versions of Python, GDAL, and Shapely.

Installing and using OSGEO4W

For 10.1 you should be able to use this, because arcpy is already bound to python 27. For 10.0???

  • Install OSGeo4W (which includes Python 2.7.4) Get it from http://osgeo4w.osgeo.org/
  • Re-run the installer and do an advanced install, selecting the file geodatabase plugin for Python.
  • Update your environment to include these variables
OSGEOHOME         C:\OSGed4W
GDALHOME          %OSGEOHOME%
GDAL_DATA         %GDALHOME%\share\gdal
GDAL_DRIVER_PATH  %GDALHOME%\bin\gdalplugins
PYTHONHOME        %OSGEOHOME%\apps\Python27
PATH              %OSGEOHOME\bin;%PATH%;%PYTHONHOME\Scripts
  • Create a file called Desktop10.0.pth and place it in your %PYTHONHOME\Lib\site-packages directory. Put these three lines in it
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Scripts

You should be able to open a cmd window and run Python, see that it's version 2.7.4 and type these lines without error

from osgeo import gdal import shapely import arcpy

Installing gdal and shapely into ArcGIS Python

Make sure you select the right bindings for your Python, which is 2.6 in ArcGIS 10. Also use 32-bit since it will match ArcGIS install. Download and install GDAL core http://www.gisinternals.com/sdk/Download.aspx?file=release-1600-gdal-1-10-0-mapserver-6-2-1\gdal-110-1600-core.msi Download and install the file geodatabase add on http://www.gisinternals.com/sdk/Download.aspx?file=release-1600-gdal-1-10-0-mapserver-6-2-1\gdal-110-1600-filegdb.msi This gets you gdal, proj.4, and python bindings. I did a custom install and disabled c# and java bindings as I have no use for them. Where does it install them? ESRI Python still cannot do 'import gdal' or 'from osgeo import gdal'.

Files get installed in C:/Program Files(x86)/GDAL

GDALHOME          C:\Program Files(x86)\GDAL
GDAL_DATA         %GDALHOME%\gdal-data
GDAL_DRIVER_PATH  %GDALHOME%\gdalplugins
PYTHONHOME        C:\Python26\ArcGIS10.0
PATH              %PYTHONHOME%;%GDALHOME%;%PATH%;%PYTHONHOME%\Scripts


Wrong path for me: Virtualenv

This is too complicated to push onto customers! For developers virtualenv looks great but I am thinking that I need to back off and make gdal work with ArcGIS Python 2.6 instead.

Virtualenv installation

My GDAL/Shapely script won't run in Model Builder because it sets up its environment for the ESRI version of Python 2.6. This is the symptom

TypeError: cannot create weak reference to 'classobj' object

I believe this means the python interpreter is getting confused and trying to import from the wrong site packages. I think I can use virtualenv to get around this problem.

I found these instructions and I am following them right now. http://www.tylerbutler.com/2012/05/how-to-install-python-pip-and-virtualenv-on-windows-with-powershell/

The guy that wrote those instructions is apparently a "powershell" user. Why does anyone use powershell? Or Windows?

  1. I went ahead and did pip virtualenvwrapper-powershell on principal even though I won't touch it
  2. but also did pip search virtualenv to find what else was available
  3. and then did pip install virtualenvrwapper-win just in case.

Then my virtual machine rebooted itself. I think it decided it had had enough of me. Now it's installing Windows updates. How annoying.

Virtualenv usage

Next step: I read the instructions on the virtualenv package page.

Create a new environment:

C:\Users\bwilson>virtualenv arcgis
PYTHONHOME is set.  You *must* activate the virtualenv before using it
New python executable in arcgis\Scripts\python.exe
Installing setuptools................done.
Installing pip...................done.

This looks promising. It created a new folder 'arcgis' containing Include, Lib, and Scripts folders. There is no "bin" folder per the instructions but there is an activate.bat in Scripts. So let's try that

cd arcgis Scripts\activate

This changes my prompt to prefix arcgis in front of it and when I do 'echo %PATH%' I see it's put itself first. C:\Users\bwilson\arcgis\Scripts is at the start of the path. To leave I just type deactivate.