Tools that make Windows useful

From Wildsong
Jump to navigationJump to search

Formerly this page was "Portable Windows". Once upon a time I needed to be able to work on computers where I had no admin privileges, couldn't get software installed, and had limited Internet access. Yes! I worked for the federal government!

I developed a kit of tools to help me out. When I was a roving code ronin I put them on an external USB drive. Now that I am a minor bureaucrat I install them on my hard drive. I have risen to the rank of trusted minor bureaucrat now and have admin rights so over time this page has moved to be just how to install tools.

But there is an entire site devoted to the portable concept: https://portableapps.com/

I kept everything in one place, a folder called "Portable", the whole folder can be copied a thumb drive. I have to deal with correctly setting PATH to make it work.

Chocolatey was great

Then I started using Conda, and using Bash as my shell and using Conda from Bash... and installing bash from "Git For Windows", and all that is fine but then I started using Chocolatey for everything and...

But chocolatey requires admin privileges... so portability went out the Windows. On the other hand, it works on Windows Server, too, so I am happy with it for now.

For Conda I decided the miniconda script is better. Go to https://docs.conda.io/en/latest/miniconda.html for the downloader.

Installing

Official instructions here https://chocolatey.org/install

You copy a string of text and paste it into an elevated PowerShell. Later on you can upgrade it with itself, 

choco upgrade chocolatey

To run things it installs, add the path C:/ProgramData/chocolatey/bin to the SYSTEM PATH since this will be useable by everyone.

You can install whatever you like (in an elevated shell) /SChannel takes care of the SSL certificate error Here are the things I need. So far it's git, bash, unzip, zip, wget, rsync, conda, emacs, and nodejs. 

choco install -y git.install --params "/GitAndUnixToolsOnPath /SChannel /WindowsTerminal"
choco install -y unzip zip wget rsync miniconda3
choco install -y emacs
choco install -y nodejs

Close your shell window and open a new one in case it changed your PATH settings.

For miniconda I had to add C:/tools/miniconda3/Scripts and C:/tools/miniconda3 to my path environment variable.

Set CONDA_PATH to C:\tools\miniconda3 and add %CONDA_PATH% and %CONDA_PATH%/Scripts to PATH


Choco has a log!

You can see what choco commands do by checking its logs in c:/ProgramData/chocolatey/logs One is verbose and one is a summary, make your choice.

Conda

Normally I have ArcGIS software installed on Windows and I would use the Conda environment that comes with it, but at ArcPro 2.8 it stopped working for me in Bash shells. That's why I always install Miniconda3 using Chocolatey. I then make sure it will run the conda of my dreams by changing my bashrc file. In my case I keep the file (called bashrc not .bashrc) in a Github, https://github.com/brian32768/Windows_bin

Conda is "Package, dependency and environment management for any language: Python, R, Ruby, Lua, Scala, Java, Javascript, C/ C++, FORTRAN"

The full Anaconda installation is huge so I use "mini-conda". "Miniconda is a small �bootstrap� version that includes only conda, Python, and the packages they depend on. "Over 720 scientific packages and their dependencies can be installed individually from the Continuum repository with the �conda install� command." I have not tried using it for anything but Python.

I used to use the IDE called Spyder, which is a reasonably complete Python IDE including a debugger. Normally now I use Visual Studio Code.

GDAL

I install GDAL as part of Chocolatey but 2020-Mar-09 using GDAL on Windows is madness. Use Docker. Seriously? Windows. Gag me.

Download from gisinternals.com -- this is kept up to date and it has a zip file that you can run binaries from (without admin or installation)

With ArcGIS Pro 2.5 I used this:

DOCKER

Back at ArcMap 10.5, this worked:

  1. Download http://download.gisinternals.com/sdk/downloads/release-1500-x64-gdal-2-1-3-mapserver-7-0-4.zip or the latest zip, I chose the 1500 release because it matches ESRI 10.5 python, not sure if it matters
  2. Unpack it someplace convenient, I put it in my home directory under GDAL, so mkdir C:/Users/bwilson/GDAL
  3. Add it to your PATH, I added this to the end C:\Users\bwilson\GDAL\bin;C:\Users\bwilson\GDAL\bin\gdal\apps
  4. If it worked then you should be able to open a cmd window and type 'ogrinfo', and get usage info not an error.

This does NOT install any Python packages, it just gets me the GDAL command line programs, which I can call using Python "subprocess" calls. It also gets me a copy of mapserver, for which I have no use at this time.

There is python stuff in the bin/gdal/python folder which I have not explored yet.

GIMP

There is a portable version, https://portableapps.com/apps/graphics_pictures/gimp_portable I use the standard one now.

Git and Bash shell

Portable Git is awesome.

On Windows 10, for bash and friends there is the WSL thing but again you have to convince the Man to install it for you.

I rely heavily on GIT to manage Python code, so I have a copy of "PortableGit". https://www.nuget.org/packages/PortableGit/

You can run the portable version from a thumb drive because does not require admin level installation and it comes with a copy of Bash that also just runs. (Cygwin and GoW for example require admin rights.)

Still missing a little "zip" (and rsync)

I used to install MinGW but that battled with the Esri version. This is another place Chocolatey shines; if you can, use it.

Credit: I got the idea for this from https://superuser.com/questions/701141/how-to-add-more-commands-to-git-bash

Short answer, go to http://repo.msys2.org/msys/x86_64/ and download the tools yourself and then install them into /usr/bin/ with "tar". Here are the tar commands. I download from the browser.

Download and install zstd so you can use .zst files. 

cd /
tar xJvf /c/Users/bwilson/Downloads/zstd-1.4.7-1-x86_64.pkg.tar.xz

Upgrade zstd 

 tar --zstd xvf /c/Users/bwilson/Downloads/zstd-1.4.8-1-x86_64.pkg.tar.zst

Download and install rsync and its libraries 

tar xv --zstd --file /c/Users/bwilson/Downloads/rsync-3.2.3-1-x86_64.pkg.tar.zst
tar xv --zstd --file /c/Users/bwilson/Downloads/libzstd-1.4.8-1-x86_64.pkg.tar.zst
tar xv --zstd --file /c/Users/bwilson/Downloads/libxxhash-0.8.0-1-x86_64.pkg.tar.zst

Sometimes I install wget, but then other times I make do with curl.

Download and install zip 

tar xvf /c/Users/bwilson/Downloads/zip-3.0-3-x86_64.pkg.tar.xz


QGIS

Download the OSGeo4W network installer (64-bit) and run it. When it asks, tell it to install for "Just Me" and not to "Add icon to Start Menu". Optionally, install to your "Portable" folder.

When you get to the "Select Packages" page, select ONLY the version of QGIS. For me (today anyway) it's 'qgis-dev', the fragile breaking version because I am in that living-on-the-edge mood. You probably want 'qgis'.

On the next page, let it install the dependencies.

Once it's installed it will be found in the Portable folder under OSGEO4W/bin

Retrieved from "https://wiki.wildsong.biz//index.php?title=Tools_that_make_Windows_useful&oldid=13825"