Hi all,

I'm looking for some suggestions to help me resurrect our ArcGIS Enterprise. Here's what happened -

1. Errors occurred while publishing some very large scene layers.
2. The output services did not start, so services were manually restarted and perhaps the entire site was restarted. This also caused errors due to an expired certificate.
3. The expired certificates were replaced with self-signed certificates.

However, now:

1. The site will not validate through the settings section in the portal
2. The relational data store will not validate in the server manager (but the tile cache does).
3. The data store does validate through the server's rest validate operation (../admin/data/items/enterpriseDatabases/AGSDataStore_ds_blah/server_blah/validate)
4. Some of the services will not shutdown or restart. Eg - the system SceneCachingTools service will not start, and throws error java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: java.net.ConnectException: Connection refused: connect , while our network routing service (map server) does not respond, won't render in the map and will not restart.
5. We cannot publish new feature layers or scene layers, but we can view scene layers and feature layers published before the crash.
6. describedatastore.bat returns a status of started for both relational and tile cache. Mode = readwrite
7. The windows application log has a few errors
   1. Notification of policy change from LSA/SAM has been retried and failed. Error 4312 to save policy change for account

   2. The description for Event ID 0 from source PostgreSQL cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer. The following information was included with the event: Is server running?

Any ideas? Should I try updatesslcertificate.bat with the self-signed certificate (as this is the only certificate I can't view)?

Thanks!

Rob

Thanks to everyone who replied. I resolved it. I followed the suggestions in this document -

Problem: Unable to start or stop services in ArcGIS for Server (esri.com)

I found the word undeploy in one of the logs, which led me to that article.

Once the system was back up and running, and I was able to publish feature and scene layers to the portal, I had a conversation with someone from Esri tech support. The analyst confirmed that the certificates and permissions were fine.

I appreciate all the help from this forum and Esri tech support.

Thanks all!

Rob

Hi @RobStauder ,

You can check you datastore cert  by opening the url https://localhost:2443/arcgis/datastore then looking at the cert info.

You have two problems your ArcGIS Server is not functioning correctly and your Datastore is not validating.

First make a backup of the system

Then can you go to services and look which service account is running your ArcGIS server then reapply the security rights on the folder structure at c:\program files\arcgis\server and the location of you configstore c:\arcgisserver or d:\arcgisserver...

Service account

config-store

Program Files

right click and open server properties then security and your service account make sure it has full control on these folders, there is a new Windows policy that changes the service account to read only.

after you have applied this you can check if your ArcGIS server is functioning.

Then there might be a partial file that has been created during your failed publishing go to  C:\arcgis\arcgisserver\config-store\services\Hosted and see if you can see anything also your rest and services will not work if this is the case

https://dns:6443/arcgis/rest

Datastore
if you can make a backup then @Ikebana suggestion will work and do a nice clean rebuild of datastore but your ArcGIS server must be in a working state.

You can also try the following go to server manager and unregister the datastore then re-register it on https://dns:2443/arcgis/datastore

you only have to unregister relational NB did you make the backups

using the https://dns:6443 admin url

if it still fails then go to the tool

"C:\Program Files\ArcGIS\DataStore\tools\unregisterdatastore.bat"

and try the 2443 url again

If it still fails then the only option left is to restore the backup

to create the backups run
"C:\Program Files\ArcGIS\DataStore\tools\backupdatastore.bat" --store relational

and

"C:\Program Files\ArcGIS\DataStore\tools\backupdatastore.bat" --store tileCache

please make sure that  they did actual make a backup in the backup location

and

something like this for relational

to restore uninstall ArcGIS datastore and remove all the files except backup in c:\arcgisdatastore or your location

relational then edit and run these commands
"C:\Program Files\ArcGIS\DataStore\tools\restoredatastore.bat" --store relational --target most-recent --source-loc D:\arcgisdatastore\backup\relational --data-dir D:\arcgisdatastore --server-url https://dns:6443/arcgis --server-admin adminuser --server-password adminpassword --loaddata true --prompt no

tileCache
"C:\Program Files\ArcGIS\DataStore\tools\restoredatastore.bat" --store tileCache --target most-recent --source-loc D:\arcgisdatastore\backup\tilecache --data-dir D:\arcgisdatastore --server-url https://dns:6443/arcgis --server-admin adminuser --server-password adminpassword --loaddata true --prompt no

don't copy and past the above string type it out please you might get encoding problems

Regards

Henry

Thanks to everyone who replied. I resolved it. I followed the suggestions in this document -

Problem: Unable to start or stop services in ArcGIS for Server (esri.com)

I found the word undeploy in one of the logs, which led me to that article.

Once the system was back up and running, and I was able to publish feature and scene layers to the portal, I had a conversation with someone from Esri tech support. The analyst confirmed that the certificates and permissions were fine.

I appreciate all the help from this forum and Esri tech support.

Thanks all!

Rob