Visual Studio Code
I am now using Visual Studio Code for pretty much all coding including Python, PlatformIO and ESPHome, JavaScript.
Today I switched from using the standard Microsoft release to using VSCodium. Then immediately switched back because the Remote Development Extension Pack is not supported.
Settings are in <AppData>/VSCodium
I use VSC to edit Python to create Geoprocessing scripts that I can run in Docker containers.
I use it to develop JavaScript React apps.
I use it to develop apps for Arduino and ESP32 and ESP8266.
Pretty much I use it now for anything that needs coding.
Extensions
The important one is
Settings Sync (uses a Gist on Github)
Use Settings Sync to install the extensions automatically. See also https://vscode.pro/settings-sync
In a browser, log in to github with the correct account. brian32768 for me. Start VSCode and install Settings Sync. Click "Login with Github". Click on the Gist to load, there should be only one. It should be able to figure everything out from here.
I normally use a different github account at work and at home and as a byproduct this means my home VSCode can have extensions related to IoT that are unwanted at work.
I had to set it temporarily to "Force Download" to get it to sync up the first time.
List of extensions
Probably out of date by the time I look at it again.
Bookmarks DotENV ESLint HTML Preview HTML snippets Jupyter Jupyter Keymap Jupyter Notebook Renderer markdownlint MinifyAll npm npm Intellisense PHP Debug PlatformIO Powershell Prettier Pylance Python Remote - Containers Remote - SSH Remote - SSH: Editing Configuration Files Svelte 3 Snippets Svelte for VS Code Svelte Preview vscode-database XML XML Tools YAML ZipFS - a zip file system
Settings
Bash on Windows (Terminal Profiles)
Edit C:\Users\bwilson\AppData\Roaming\Code\User\settings.json and add
Terminal profiles: https://code.visualstudio.com/docs/terminal/profiles
Python profiles: https://code.visualstudio.com/docs/python/environments
"terminal.integrated.profiles.windows": {
"Bash": {
"path": ["C:\\ProgramData\\Git\\bin\\bash.exe"],
"icon": "terminal-bash"
},
"PowerShell": {
"source": "PowerShell",
"icon": "terminal-powershell"
}
},
"terminal.integrated.defaultProfile.windows": "Bash",
"python.terminal.activateEnvironment":false
Restart VScode and open a terminal and it should start a bash shell now instead of the stupid default "Power" Shell. The "activateEnvironment" line prevents it from running broken commands in the shell like "conda activate Miniconda2". I still don't know where those come from.
Conda
Esri used to include a very old version of conda, as of ArcGIS Pro 2.8 it's not so bad, but for other reasons I usually download Miniconda and install it.
Running "conda init" as directed by conda always throws errors at me. Editing the .bash_profile manually worked. Put something like this at the end of your .bash_profile
CONDA_PATH=/c/Users/bwilson/Miniconda3
export CONDARC=$HOME/bin/condarc
source ${CONDA_PATH}/etc/profile.d/conda.sh
I keep bin/condarc in a github repo at brian32768/windows_bin
You should be able to list available environments with
conda env list
and you should be able to activate the standard Esri environment with
conda activate arcgispro-py3
Your prompt will change to include [arcgispro-py3].
In VSCode, remember to activate the interpreter in the WORKSPACE not the User settings, that's where the conda environments will show up.
My condarc file now looks like this
There are many problems fixed here. Using condarc works from shells but I found I needed an environment variable CONDARC too so I set that in bashrc as described above.
# This was not working for me so I commented it out.
#channels:
# - esri
# - conda-forge
# - defaults
ssl_verify: true
envs_dirs:
- C:/ArcGISPro/bin/Python/envs
- ${LOCALAPPDATA}/ESRI/conda/envs
pkgs_dirs:
- ${LOCALAPPDATA}/ESRI/conda/pkgs
- C:/ArcGISPro/bin/Python/pkgs
changeps1: true
Confirm it's really reading the right condarc file with "conda info". Today mine looks like this,
active environment : None
shell level : 0
user config file : C:\Users\bwilson\.condarc
populated config files : C:\Users\bwilson\.condarc
J:\.condarc
conda version : 4.10.1
conda-build version : not installed
python version : 3.8.5.final.0
virtual packages : __cuda=10.1=0
__win=0=0
__archspec=1=x86_64
base environment : C:\Users\bwilson\Miniconda3 (writable)
conda av data dir : C:\Users\bwilson\Miniconda3\etc\conda
conda av metadata url : https://repo.anaconda.com/pkgs/main
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\bwilson\AppData\Local\ESRI\conda\pkgs
C:\ArcGISPro\bin\Python\pkgs
envs directories : C:\ArcGISPro\bin\Python\envs
C:\Users\bwilson\AppData\Local\ESRI\conda\envs
C:\Users\bwilson\Miniconda3\envs
C:\Users\bwilson\.conda\envs
C:\Users\bwilson\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.10.1 requests/2.25.1 CPython/3.8.5 Windows/10 Windows/10.0.19041
administrator : False
netrc file : C:\Users\bwilson/.netrc
offline mode : False
Using clone inside ArcGIS Pro fails
Up until ArcGIS Pro 2.8, Esri installed the conda environment with root permissions so normal users could not modify it. The workaround was to clone and then use the clone as the base in future cloning operations. Cloning the environment instead of modifying the base environment is still highly recommended. I typically set up a new environment for each project as part of my workflow.
If you see the Autopep error
Install that autopep thing for syntax highlighting.
conda install autopep8
It's not finding Python!!
Here is a doc telling where it looks.
Put the commonly used variables into your user settings, File -> Preferences -> Settings -> User -> Extensions -> Python -> Conda Path
"C:/Program Files/ArcGIS/Pro/bin/Python/Scripts/conda.exe"
Reload VSCode. Intellisense should work for you now too.
Now when you select a Python version, the right ones should appear. Make sure you pick the right environment (for me I named it "arcgispro-py3-vscode")
If you do it this way you should be able to use conda correctly from a bash shell too.
(base) [/cdn-cgi/l/email-protection [email protected]]:~$ conda env list # conda environments: # base * C:\Program Files\ArcGIS\Pro\bin\Python arcgispro-py3 C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 arcgispro-py3-vscode C:\Users\brian\AppData\Local\ESRI\conda\envs\arcgispro-py3-vscode conda activate arcgispro-py3-vscode (arcgispro-py3-vscode) [/cdn-cgi/l/email-protection [email protected]]:~$ python --version Python 3.6.10h:: Anaconda, Inc.
and so on...
ESRI .PYT files are not treated as Python
I put this in my global settings.json file (F1 File:Associations)
"files.associations": {"*.pyt": "python"}
I don't have a fresh machine to test all this on.
What I have found works best for me is to install miniconda from the Anaconda web site. The full Anaconda has 100's (literally I think something like 1500) packages and extra programs I don't need. So In install miniconda and add packages only when I need them. Many people use the package manager GUI that comes with Anaconda, I don't.
I generally treat each project I am working on separately, so for example if I am working on my arcgis_tools project I have an environment for it called "arcgis_tools". With the project I keep a "requirements.txt" list of required packages and so the environment is always ready to go and if something breaks it I can remove it and recreate it from requirements.txt ("conda create --name=arcgis_tools --file=requirements.txt").
When I give suggestions to other people I usually try to tell them how to use the conda that Esri provides, but I never do that myself. :-) I do lots of development outside of the Esri world, so I use conda on machines that have absolutely no geospatial software installed. So knowing how to use Conda without Esri is important to me personally. Still and all I have never found any problem using the latest Conda alongside the version shipped with Esri. So it's easiest for me to install Miniconda.
You have to go way down to the bottom of Anaconda.org where you will find a link to the Miniconda page, https://docs.conda.io/en/latest/miniconda.html. Scroll down to find the correct download link. You want Miniconda3 which is based on Python3. There are full installation instructions there. It says to let it change the PATH variable when it installs.
Most of the notes I keep for myself are on using Visual Studio Code and the Bash shell, here: https://wiki.wildsong.biz/index.php?title=Visual_Studio_Code I realize that is good for 1% of the people out there but there is still good information there I think. Also I have some older notes here: https://wiki.wildsong.biz/index.php?title=Anaconda
There are some notes on the .condarc file -- there should be one in your home directory after installing, regardless of what shell you use. It controls what channels are used for packages and where profiles live and stuff like that. When you do "conda info" it will show something like "user config file: C:\Users\bwilson\.condarc".
Currently my .condarc has this in it channels: - conda-forge - defaults When running from the command line I need to add the esri channel. Sometimes I put it in .condarc, and I am pretty sure order matters so if you do it like this it will search for generic packages first, then esri. channels: - conda-forge - esri - defaults On this machine it's finding my environments with no extra hints. In config info I see envs directories : C:\tools\miniconda3\envs C:\Users\bwilson\.conda\envs C:\Users\bwilson\AppData\Local\conda\conda\envs It's not searching the Esri space. I built my own "arcgispro29" environment and it's found so I am fine. If I want it to load a specific Esri environment I can use the full path or I can edit .condarc.
The command would be
$ conda activate "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3"
The .condarc could be changed like this. channels: - conda-forge - defaults envs_dirs: - C:/Program Files/ArcGIS/Pro/bin/Python/envs - C:/Users/bwilson/AppData/Local/ESRI/conda/envs This adds the non-writeable Pro folder and the one that Pro can write into in AppData, the others that it found on its own are all still there. From conda info----- envs directories : C:\Program Files\ArcGIS\Pro\bin\Python\envs C:\Users\bwilson\AppData\Local\ESRI\conda\envs C:\tools\miniconda3\envs C:\Users\bwilson\.conda\envs C:\Users\bwilson\AppData\Local\conda\conda\envs And now when I do 'conda env list' I can see the Esri environments too.
In ArcGIS Pro, currently I can only see two environments. I can if I want just move a folder in there, conda is pretty forgiving. So if I move arcgispro29 from its current location to the AppData 'envs' folder and restart Pro, then it will be available in there. I can activate it in pro and then run a notebook
I notice it has arcgis 1.9.1 in there. I tried to upgrade it with "conda update arcgis" but got many compatibility errors so I tried "conda create --name=arcgisnew -c esri arcgis arcpy". Tick tock, it's running. Isn't this where we came in? I suspect it will spin forever saying "Solving environment".
If I do it in two steps, first "conda create --name=arcgisnew arcpy" followed by "conda install --name=arcgisnew arcgis"
The glory of debugging Python Flask apps
I use Conda for python environments, so I create a new env from the command line and then restrt VSCode/ Maybe I could create the env INSIDE VSCode but that's not how I do it. Once it's created and VSCode restarted
conda create -n flask conda activate flask conda install autopep8 conda install flask
To get going quickly I can just pick that version of python when I hit "F5" to run in the debugger. The service starts running on localhost, http://127.0.0.1:5000/ and you can connect via browser.
Console output shows you connections.
C:\Users\bwilson\source\repos\arcgis_rest> cmd /C "C:\Users\bwilson\AppData\Local\ESRI\conda\envs\flask\python.exe c:\Users\bwilson\.vscode\extensions\ms-python.python-2020.10.332292344\pythonFiles\lib\python\debugpy\launcher 57269 -- c:\Users\bwilson\source\repos\arcgis_rest\flask_test.py " * Serving Flask app "flask_test" (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) 127.0.0.1 - - [05/Nov/2020 11:05:50] "GET / HTTP/1.1" 200 - 127.0.0.1 - - [05/Nov/2020 11:05:50] "GET /favicon.ico HTTP/1.1" 404 -
Setting breakpoints works. For example, set a breakpoint in a route handler then hit the associated URL and execution stops so you can examine variables, trace code flow, etc...
VSCode + Flask
Basically, VSCode is a totally awesome tool for developing Flask apps.
I can create a .vscode/launch.json
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Flask",
"type": "python",
"request": "launch",
"module": "flask",
"cwd": "${workspaceFolder}/flask_auth_test",
"env": {
"FLASK_APP": "app.py", // a single py file here or a module name, typically "app"
"FLASK_ENV": "development",
"FLASK_DEBUG": "0"
},
"args": [
"run",
"--no-debugger",
"--no-reload"
],
"jinja": "true",
"console": "internalConsole" // I had to start using this to avoid "launcher timed out" failures in Windows 10.
}
]
}
The further glory of debugging ArcGIS Pro Python Toolboxes
There are notes here for Visual Studio and PyCharm. I based this section on that.
Defeat caching
First off you need to force reloading in ArcGIS Pro else you will have to quit and restart Pro every time you make a change in the Python.
In the PYT file do something like this
import importlib import your_tool_module import.reload(your_tool_module) from your_tool_module import YourTool
and when you are done developing, comment out the first three lines so that caching can work again. In Pro, use right-click + Refresh on the PYT to reload the python module(s).
Attach to a running Python
See Launch vs attach.
- Load up the tool in ArcGIS Pro (open it in other words.)
Remote development
I've now tried this tutorial and learned it works. Python in a Container I got a simple Flask app running and then switched to running even simpler Python scripts in the container.
When I needed to add volume support I discovered this page: VS Code Remote Development It explains the Remote Development extension pack.
- I can use Remote - SSH to treat a remote machine (say, Bellman) as the host for a remote project. (* As recommended by John Sullivan.)
- I can use Remote - Containers to treat a Docker container as the host.
- I can use Remote - WSL too but I don't use Windows Subsystem for Linux currently.
Remote SSH
Once it's set up, working remotely is exactly like working locally. Everything works. It's quite amazing.
The first time you connect successfully over SSH from VSCode, it will install its own service in .vscode-service/ on the remote machine and keep whatever it needs in there.
Python interpreter error
When setting up a new remote machine, you have to manually install the Python extension on the new machine. (You install it just like any extension but tell it to install on the remote machine.)
Until you do, you will get an error whenever you run debug (F5) and not be able to select a python interpreter.
You will notice there is no option at all to select an interpreter in the bottom bar until the extension is installed.
Docker containers
I can keep the code on the local file system or in the container.
My first tests I used the Dockerfile to load my code into the image. It worked fine.
More options:
- clone from github into a running container
- keep code in a volume mounted on the container
Using a volume seems to make the most sense to me. In that setting I can still easily use command line git.
There is a sample based on python, it starts a Debian container and puts a Bash prompt into a Terminal window. It connects when you do F5. Your code is accessible on the local filesystem and in the mounted volume in the container.
git clone https://github.com/Microsoft/vscode-remote-try-python
Write an HTML file
- Install the HTML Preview extension. ctl-k v
- Install Prettifier (see next section on working with JSON.)
Probably I should choose and install some CSS helpers, too.
Working with JSON files
Esri keeps Portal content "source" files in C:/arcgis/arcgisportal/content/items/ Here there be dragons, tread lightly and make backup copies before you edit anything.
I install the 'Duplicate action' extension to create a duplicate as a backup. Select file and then F1-duplicate creates a file with -copy added to its name.
You can attempt to edit these files using the AGO "Arcgis Online Assistant", but it's like doing orthoscopic surgery and I don't get paid enough for that.
Instead I open files in VSCode.
Formatting
Actually everything, not just JSON...
The Esri JSON files are often minified (as in all whitespace removed) making them impossible to edit.
- Install the extension "Prettier" and set it to be the default formatter. This is good because everything you ever do will be formatted properly.
- Install Formatting Toggle, because you don't want Prettier turned on all the time!!! This extension puts a control in the lower right corner of VSCode.
- Install "MinifyAll" so you can go the other direction.
To get Prettier to work, go into VSCode settings and search for "Formatter" and set it to be the Default Formatter". At the bottom of the settings see the Glob thing "Prettier: Document Selectors" and set what kinds of docs you want it to attack.
I also find Insert File extension to be handy and was surprised this is not built in. I put the JSON for a layer in a file like 'precincts.json' and then use Insert File (F1-Insert File) to add it in to the current map that I am editing. It reduces the chances for errors.
To format a doc,
Esri saves things in files that have no extension, so VSCode can't tell what's in the file, it just sees text.
For example, if I open a file named, bb5debcf14db4c48bbd4c54fc0fc207f then it just shows up as grey plain text.
Do this: F1-Change Language Mode and select "JSON" (without support for comments) Now VSCode will treat that file like JSON.
If you have installed a formatter (Prettier), right click to format it and select Format Document. This works even if the formatter is toggled off. You can also just open a minified JSON file and save it. That's right, just do a Ctl-S, and the doc will be completely reformatted, because it's set to do Format On Save by default. If you don't want it automatically formatted, then use Formatting Toggle to turn it off.
To minify a doc, first off, you really don't need to. The next edit-save in ArcGIS Map Viewer will minify it for you. But Ctl-Alt-M will do it, or you can use Ctl-Alt-N to create a "-min" minified doc and leave the json file alone. These are available by right clicking the document or the file in the file explorer pane.
