Inside Esri ArcGIS Enterprise

See also Inside Esri ArcGIS GeoEvent Server

Delete service

The problem: Portal says it can't delete an item because it's protected or there are dependent services, and you are pretty sure it is broken.

In my case it's another stupid broken hosted service. Many of my hosted services broke when we upgraded Server from 10.8.1 to 10.9. Many curses to Esri.

Remove from Portal

Commands to remove an item from Portal that can't be deleted from the Web interface

1. Find the item ID from the Portal Content tab
2. cd C:\Program Files\ArcGIS\Portal\framework\runtime\pgsql\bin
3. Make sure the entry exists; there should be just one row: psql -h localhost -p 7654 -U CCGISadmin -d gwdb -c "SELECT id FROM gw_items WHERE id = '<itemID>'"
4. Delete the item: psql -h localhost -p 7654 -U CCGISadmin -d gwdb -c "DELETE FROM gw_items WHERE id = '<itemID>'"
5. Reindex the Portal from Portal admin endpoint (eg https://YOURDOMAIN.COM/portal/portaladmin/) : System> Indexer> Reindex > Full

The folder /c/arcgis/arcgisportal/content/items/a57652c467284f8db3189ef138fbe548 will be left around forever I think. You could just delete the folder yourself, or ignore it. It won't be used anymore.

I did this: I counted the number of folders then did the reindex operation and counted again.

Before, this is an insane number of objects to put in one folder.

ls -1 | wc -l
24127

Remove from Server

Checking in Server Manager (eg ) I can still see the service, but I cannot delete it. The problem is described here: https://community.esri.com/t5/publishing-and-managing-services-questions/unable-to-remove-non-functioning-service-from/m-p/1164214

As described there, I also had a service with a character in the name that Esri did not like. Apparently the youngster who wrote ArcGIS Enterprise has not finished his computer science degree yet. You are allowed to create services that cannot be used or deleted.

See also https://support.esri.com/en/technical-article/000021607

In my case the only thing left hanging around was this:

c:/arcgisserver/directories/arcgissystem/arcgisinput/Hosted/Waterway_Inventory.FeatureServer/

Map Image Layer

After publishing via Python API, I get an error when I load this and I am trying to take it apart and fix it!

The error is when I open it in a Map Viewer.

Error
The layer, Hyperlinked_Surveys, cannot be added to the map.

There is no layer called "Hyperlinked Surveys", so this makes sense! I'd like to know why it's trying to load this though. I cannot find any references to that string in my map! It's called "hyperlinked_surveys".

Code used to publish the services

I used a Notebook running in VSCode to do this. The file config.py contains references to environment settings containing my secrets.

This cell signs in and finds the map and the first layer in the map

from scripts.config import Config
import arcpy
arcpy.SignInToPortal(Config.PORTAL_URL, Config.PORTAL_USER, Config.PORTAL_PASSWORD)

project = arcpy.mp.ArcGISProject("K:/webmaps/basemap/basemap.aprx")
maps = project.listMaps()
mapnames = [map.name for map in maps]
map = maps[mapnames.index('arcgis_utils unit test')]
layer = map.listLayers()[0]
layer

This cell creates the .sddraft and .sd files, it takes 30 seconds to run

def fn(sd_name, server, service, copy=True):
   sddraft = map.getWebLayerSharingDraft(server_type=server, service_type=service,
       service_name=sd_name, layers_and_tables=[layer])
   sddraft_name = os.path.join("C:\\Temp", sd_name + '.sddraft')
   sddraft.overwriteExistingService = True
   sddraft.copyDataToServer = copy
   if server=='FEDERATED_SERVER':
       sddraft.federatedServerUrl = Config.SERVER_URL  # required for FEDERATED_SERVER!
   print(sddraft_name)
   sddraft.exportToSDDraft(sddraft_name)
   arcpy.env.overwriteOutput = True
   arcpy.StageService_server(sddraft_name, os.path.join("C:\\Temp", sd_name + ".sd"))
print(os.getcwd())
fn("Survey_Sample_MIL", "FEDERATED_SERVER", "MAP_IMAGE")
fn("Survey_Sample_FEATURES", "HOSTING_SERVER", "FEATURE")
fn("Survey_Sample_GEODATABASE_FEATURES", "HOSTING_SERVER", "FEATURE", copy=False) THIS DOES NOT WORK
fn("Survey_Sample_GEODATABASE_MAP", "FEDERATED_SERVER", "MAP_IMAGE", copy=False)

This cell creates or overwrites the Map Image Layer, it also runs in about 30 seconds.

arcpy.server.UploadServiceDefinition(in_sd_file="C:\\Temp\\Survey_Sample_MIL.sd", in_server="K:/webmaps/basemap/server (publisher).ags")

Note that this WORKS in this case.

arcpy.server.UploadServiceDefinition(in_sd_file="C:\\Temp\\Survey_Sample_MIL.sd", in_server=Config.SERVER_URL)

and it produces this output:

Output

id	value
0	https://CC-GIS.CLATSOP.CO.CLATSOP.OR.US:6443/arcgis/services/Survey_Sample_MIL/MapServer
1	https://CC-GIS.CLATSOP.CO.CLATSOP.OR.US:6443/arcgis/rest/services/Survey_Sample_MIL/MapServer
2	
3	
4	
5	
6	
7	
8	
9	
10	
Messages
Start Time: Thursday, December 23, 2021 12:08:23 PM
Submitted.
Executing...
Executing (Publish Service Definition): PublishServiceDefinition ia976f338-6563-4da1-b591-f66b40769178 # {"buildInitialCache":false}
Start Time: Thu Dec 23 12:08:24 2021
Submitted.
Executing...
Start Time: Thursday, December 23, 2021 12:08:25 PM
Getting server information (server folders, data stores, etc.).
Validating service definition.
Creating the service.
Succeeded at Thursday, December 23, 2021 12:08:51 PM (Elapsed Time: 25.82 seconds)
Succeeded.
Succeeded at Thu Dec 23 12:08:51 2021 (Elapsed Time: 27.30 seconds)
Succeeded.
Succeeded at Thursday, December 23, 2021 12:08:53 PM (Elapsed Time: 29.60 seconds)

This cell creates or overwrites the Feature Service, it takes a minute

arcpy.server.UploadServiceDefinition(in_sd_file="C:\\Temp\\Survey_Sample_FEATURES.sd", in_server="K:/webmaps/basemap/server (publisher).ags")

Note that this DOES NOT WORK in this case.

arcpy.server.UploadServiceDefinition(in_sd_file="C:\\Temp\\Survey_Sample_FEATURES.sd", in_server=Config.SERVER_URL)

Output is

Output
id	value
0	
1	
2	
3	35886b1683e246638eca2fc2648b962f
4	
5	https://delta.co.clatsop.or.us/server/rest/services/Hosted/Survey_Sample_FEATURES/FeatureServer
6	
7	hyperlinked_surveys|0
8	
9	
10	
Messages
Start Time: Thursday, December 23, 2021 12:13:38 PM
Succeeded at Thursday, December 23, 2021 12:14:34 PM (Elapsed Time: 55.88 seconds)

Finally in this cell, I attempt to publish a service referencing the Geodatabase instead of copying the data.

arcpy.server.UploadServiceDefinition(in_sd_file="C:\\Temp\\Survey_Sample_FEATURES.sd", in_server="K:/webmaps/basemap/server (publisher).ags")

gives this output in about 20 seconds, and I am pretty sure it ignored the "copy=False" and copied data anyway. I will be looking at this. The object being used is a FeatureSharingDraft and the docs say "Creates a sharing draft for a hosted web feature layer". So, how do I share a geodatabase feature service?

Output
id	value
0	
1	
2	
3	151d6134e5574563937257665b41c132
4	
5	https://delta.co.clatsop.or.us/server/rest/services/Hosted/Survey_Sample_GEODATABASE_FEATURES/FeatureServer
6	
7	hyperlinked_surveys|0
8	
9	
10	
Messages
Start Time: Thursday, December 23, 2021 12:59:22 PM
Succeeded at Thursday, December 23, 2021 12:59:45 PM (Elapsed Time: 22.37 seconds)

Finally this seems to work, but I need to switch on feature access.

arcpy.server.UploadServiceDefinition(

   in_sd_file="C:\\Temp\\Survey_Sample_GEODATABASE_MAP.sd", in_server="K:/webmaps/basemap/server (publisher).ags")

Output
id	value
0	https://CC-GIS.CLATSOP.CO.CLATSOP.OR.US:6443/arcgis/services/Survey_Sample_GEODATABASE_MAP/MapServer
1	https://CC-GIS.CLATSOP.CO.CLATSOP.OR.US:6443/arcgis/rest/services/Survey_Sample_GEODATABASE_MAP/MapServer
2	
3	
4	
5	
6	
7	
8	
9	
10	
Messages
Start Time: Thursday, December 23, 2021 1:13:21 PM
Submitted.
Executing...
Executing (Publish Service Definition): PublishServiceDefinition if46ac217-e150-4817-b089-3bdf78d8b6db # {"buildInitialCache":false}
Start Time: Thu Dec 23 13:13:22 2021
Submitted.
Executing...
Start Time: Thursday, December 23, 2021 1:13:23 PM
Getting server information (server folders, data stores, etc.).
Validating service definition.
Creating the service.
Succeeded at Thursday, December 23, 2021 1:13:35 PM (Elapsed Time: 12.03 seconds)
Succeeded.
Succeeded at Thu Dec 23 13:13:37 2021 (Elapsed Time: 15.19 seconds)
Succeeded.
Succeeded at Thursday, December 23, 2021 1:13:38 PM (Elapsed Time: 17.14 seconds)

Files on Portal

When I publish, it creates an entry in Portal. I can see its ID in the URL and then go to the folder on the Portal server, <Portal>/content/items/<ID> where for me <Portal> is C:/arcgis/arcgisportal/ In the folder there is a subfolder called "esriinfo" containing these files. I am exploring with Visual Studio Code with the Remote SSH extension, and formatters installed for XML and JSON. VSCode is awesome.

metadata/metadata.xml
thumbnail/thumbnail.JPEG which I presume is... the thumbnail!! that appears in Portal
item.pkinfo    this is a tiny JSON file that has the ID, a creation date, and the type "Map Service".
iteminfo.json  this has all the properties in it including a GUID, extent, SRS, keywords (tags), a path to the thumbnail, a blank URL, and the fields you see in Portal like name and description. 
iteminfo.xml   is an XML version of a subset of the same properties

Note there is no list of layers in there and no reference to "Hyperlinked Surveys".

In Portal, I see that my MIL has one layer called "Survey_Sample_MIL", and when I click on that I see this URL: https://delta.co.clatsop.or.us/server/rest/services/Survey_Sample_MIL/MapServer Note it's referring to "server" which is the name of my ArcGIS Server instance, so it lives only on ArcGIS Server, not in Portal.

When I check out that link, everything seems okay. I can if I want enable more features in there such as WFS, WMS, and Feature Access.