MobileMap: Using Portal for ArcGIS

Overview

While many MobileMap users like the simplicity of ArcGIS Online for their ArcGIS implementation (no server hardware, no SQL database licenses, no need to configure firewalls, etc.), MobileMap works great with Portal for ArcGIS (AKA ArcGIS Enterprise) as well.  For organizations that support ArcGIS Portal, the choice between using ArcGIS Online and Portal can be challenging. There are also hybrid approaches (Hybrid Implementations - ArcGIS Online and Portal) in which users log in via ArcGIS Online, but data are synchronized with Portal, to achieve an on-premises, near real-time (typically synced nightly) ArcGIS Enterprise geodatabase with your inventory data.

Choosing Between ArcGIS Online and Portal (or Both)

Generally organizations that already support Portal choose to use their Portal implementation for InventoryManager. This is not always the case, however. The main considerations include whether the majority of users are in-house staff (instead of contractors), and whether their are sufficient licenses and IT/GIS capacity to support InventoryManager. Sometimes, organizations with Portal choose to use ArcGIS Online for InventoryManager because it will make the system easier to maintain (e.g., maintained by Foresters rather than GIS/IT) or simpler to provide contractor access due to their IT policies. When organizations have not yet implemented Portal, but are considering doing so, this decision can be more challenging. The table below shows some of the consideration for choosing between ArcGIS Online and Portal for organizations with access to both:

Item

Description

ArcGIS Online (AGO)

ArcGIS Portal

Hybrid: ArcGIS Online and Portal

Details

ArcGIS licensing

End-user ArcGIS Licenses for logging in to MobileMap and InventoryManager

Customer

Customer

Customer

Regardless of approach, it is the customer’s responsibility to purchase appropriate licenses for their users.

Backup

Secure, offline, backups of data to support restoration of accidental data deletion or disaster recovery.

Either

Customer

Customer (or both, with WSG backing up the ArcGIS Online portion).

Customer would be expected to backup ArcGIS Enterprise implementations.  For AGO, WSG can support with automation

Database licensing

license(s) for Relational Database Management System (MS SQL Server). Note that PostgreSQL is currently not supported. 

NA

Customer

Customer for ArcGIS Portal portion

Customer would need to provide database licenses.  Not relevant for AGO as this is SaaS and includes database licensing.

Database implementation and management

 

Either

Customer

Customer (or both, with WSG publishing services in the ArcGIS Online portion).

Customer would need to implement and manage databases in ArcGIS Portal. WSG can be granted permission to publish databases (feature services) in ArcGIS Online.

ArcGIS Administration

Managing feature service settings, users groups, and sharing

Either

Customer

Customer (or both, with WSG publishing services in the ArcGIS Online portion).

Typically performed by customer, but WSG can be granted permission to manage users, groups and sharing.

Server hardware*

Physical or virtual servers for database, web server, etc.

NA

Customer

Customer

For ArcGIS Enterprise customer needs to provide suitable database and web server resources.

Network hardware and configuration*

Network hardware including load balancers, firewalls, routers, etc.

NA

Customer

Customer

For AGO only basic public internet connection is needed.  For ArcGIS Enterprise, VPN and or firewall configurations may be needed.

* See https://enterprise.arcgis.com/en/get-started/latest/windows/base-arcgis-enterprise-deployment.htm  for more details on the components and architecture options when deploying ArcGIS Portal

Differences Between ArcGIS Online and Portal

In addition to the components that must be supported, there are a few functional differences for MobileMap and InvevntoryManager users of Portal vs. ArcGIS Online. While all MobileMap functionality are compatible with Portal, there are two specific functions that require some special configuration to work with Portal: 1) the ability to download base maps directly from ArcGIS Online, and 2) support for turn-by-turn driving directions.  In order to enable these functions when using Portal, the Portal administrator will need to configure Portal to enable users to access these ArcGIS Online resources from within MobileMap.

When customers implement MobileMap and InventoryManager using portal, rather than ArcGIS, their users will sign into these applications with their Portal credentials. There is no need to have both a Portal and ArcGIS Online ‘named user’ account, and no need for another login that is specific to MobileMap or InventoryManager - users simply enter their credentials as though they were logging into their Portal to use a Web Map, view a Dashboard, or access a configurable Web App.

Hybrid Implementations - ArcGIS Online and Portal

Hybrid implementations can be useful when an organization has access to both ArcGIS Online and Portal, especially when the organization wants to provide access to contractors without having them join their corporate domain, and they have backend database scripts or other automated processing the require ArcGIS Enterprise and access to the backend SQL Server database. There are two main approaches to hybrid implementations: 1) registering an ArcGIS Portal service with AGOL, and 2) distribute collaboration.

  1. The simplest way to create a hybrid implementation is to publish the service from ArcGIS Enterprise, then register that service with ArcGIS Online, instead of Portal. This allows users to authenticate and access data via ArcGIS Online, even though the data is coming from an ArcGIS Enterprise implementation. In this case ArcGIS Online acts as a proxy, relying the request and response to and from ArcGIS Enterprise. See https://doc.arcgis.com/en/arcgis-online/manage-data/add-item-from-url.htm for more information on registering an ArcGIS Enterprise service with ArcGIS Online. Note that when and ArcGIS Enterprise service is registered with ArcGIS Online, an enterprise user credentials need to be stored. There are two possible downsides to this approach: performance and editor tracking.

    1. Because ArcGIS Online is relaying content, this approach may result in slower responses which can lead to slower behavior in InventoryManager or slower sync operations in MobileMap.

    2. Because ArcGIS Online is acting as a proxy using stored enterprise credentials, all adds/updates will show up as the stored enterprise user in editor tracking fields (created_user, last_edited_user). InventoryManager and MobileMap can be configured to account for this by using the Observer ID (Cruiser_ID) field to record the actual user collecting and editing data.

  2. A more sophisticated approach known as distributed collaboration can be used to create a clone of an ArcGIS Enterprise feature service in ArcGIS Online, and automatically synchronize these services according to a schedule. This approach is documented in https://enterprise.arcgis.com/en/portal/latest/administer/windows/understand-collaborations.htm. If reviewing this document, focus on they typical hybrid implementation: Collaboration between multiple Enterprise portals with a central ArcGIS Online organization. In this scenario the ArcGIS Online organization is the ‘host’ and the ArcGIS Portal instance is the collaboration ‘participant’. When the collaboration between ArcGIS Online and Portal is established, the synchronization schedule can be established.

Workflow for Migrating from AGOL to Portal

WSG has developed a well-defined set of steps for implementation which can be found in. The typical implementation begins with design and testing in WSG’s (or a partner’s) ArcGIS Online instance to help facilitate rapid updates to data models, business rules, application settings, etc. For customers using ArcGIS Portal, once the initial testing and configuration is complete, there is a migration (sometimes called ‘cutover’) event in which data models are transitioned from ArcGIS Online to the customer’s Portal instance. The steps below are for this typical migration, but would be very similar if a customer is using ArcGIS Online in their own organization wanted to migrate from ArcGIS Online to Portal. T

The typical process for implementing MobileMap (and InventoryManager, where relevant) is described in the list below, including who is responsible for each task (WSG, customer, or both).

  1. Define the data model using the process defined in - both

  2. Publish the data model to WSG’s instance of ArcGIS Online using the process defined in - WSG

  3. Test all aspects of data model, settings, data (including Rules, Related Domains, etc.) - both

  4. Deliver data models, map documents, settings, data (typically WSG will deliver a zip file with all of the required components for this, including a File Geodatabase, an ArcMap MXD file, and settings and data in JSON format as text files) - WSG

  5. Implement data model in Portal (using GIS analysts preferred methods - consider export/import of XML workspace when using SQL Server) - customer

  6. Publish data model as Feature Service in Portal (using GIS analysts preferred methods - pay special attention to inclusion of MobileMap settings in the feature service description) - customer

  7. Share Feature Service with appropriate group(s) and ensure that all users (including at least one WSG user) have been invited to a relevant group(s) - customer

  8. Register applications with Portal (and InventoryManager, where relevant) using instructions below - customer

  9. Notify WSG who will assist with testing (and update InventoryManager, where relevant) - customer

Portal Security

See Esri documentation on how to implement authentication and security for your Portal instance. Note that if you follow the best practices described in that documentation link, you will need to ensure that cross domain requests (CORS) is allowed for InventoryManager (or other Spatial CMS sites). More information can be found in this link. You will also need to register MobileMap and InventoryManager (or other Spatial CMS sites) with your Portal instance in order to allow these applications to authenticate via Portal. See instructions below for how to register these applications.

MobileMap

Registering MobileMap with Portal

In order to use MobileMap with your Portal instance, it is necessary to register MobileMap as an application that can authenticate with Portal.  This can be done by going to Content > New Item > An Application.

  • On the 'Add an application' form, select 'Mobile', 'Ready to Use' and 'Android'

  • Add the following URL: https://mobilemap.mbgapplications.com

  • Set the title to 'MobileMap'

  • Set 'MobileMap' as a tag, and add any additional tags that will help administrators search for this item

  • Click 'Add Item'

  • When the new item is created, go to settings, and scroll down to the bottom of the page and click 'Register'

  • On the Register form select 'Native' for App Type

  • Enter the following Redirect URI: urn:ietf:wg:oauth:2.0:oob

  • Click 'Add'

  • Click 'Register

  • The item will now display the App ID.  Copy this and use it as the 'Client ID' in the MobileMap authentication Settings below.

 

Configuring MobileMap Authentication Settings

Once you have registered MobileMap with your Portal, you will need to configured MoibleMap settings to use your Portal:

  • Open Settings > Authentication

  • Check the box for 'Use Portal for ArcGIS

  • in 'Portal for ArcGIS Name' select a name (sed as a title for the sign-in form to help remind you that you are signing into Portal and not ArcGIS Online)

  • In 'Portal for ArcGIS URL' enter the URL to your Portal

  • In ' Portal for ArcGIS Client ID', enter the Client ID that was created by Portal when you registered MobileMap (see details above)

Sign in to Portal

Once you have registered MobileMap with your Portal, and configured MoibleMap settings, sign-in to your Portal by doing the following:

  • Open the Actions Menu

  • Select 'Sign in to ArcGIS'

    • Enter your Portal username and password

    • Tap 'Sign In'

  • If you see a blank form instead of a sign in form:

    • check to make sure you have an interent connection

    • Verify the Portal URL and Client ID in settings (see details above)

  • Note that for typical daily authentication (e.g., upload/download) it is possible to use credentials stored in Settings > Authentication > ArcGIS Username and ArcGIS Password.  Portal sign-in described above is typically required only for refreshing the list of available feature services when selecting a feature service (Actions > Select Feature Services).

Downloading Base Maps in MobileMap

Starting at MobileMap version 4.3.39 it is possible for Portal users to download base maps within MobileMap. Historically this was not possible because the base maps accessible within MobileMap Base Map Download Tool are the default base maps hosted in ArcGIS Online. Portal users cannot authenticate against these base maps because their credentials are valid for Portal only, not ArcGIS Online.

To download base maps when using Portal, there are two steps that are needed

  1. Register ArcGIS Online base map(s) with Portal

  2. Configure MobileMap Authentication settings to include your registered base map(s)

Register ArcGIS Online base map(s) with Portal

To register one or more ArcGIS Online base maps with your portal instance, log into Portal as an administrator and select My Content > New Item > From URL. Complete the form shown shown below (note this image is from ArcGIS Enterprise 10.9.1) supplying the URL to the base map service. The following ArcGIS Online base maps are the most commonly used:

Copy and paste the desired base map url into the URL text box. The ‘Type’ should automatically update too ArcGIS Server web service. Toggle the ‘Store credntials with service item’ switch to on, then click ‘Next’.

 

On the next page in the form, provide the username and password for an ArcGIS Online named user account which will be used by the proxy when users request tile package exports. This is typically a service account that is neither an admin account nor an individual user.

On the final page in the form add a Title, Folder, Categories, Tags and a Summary. Click Save.

In the overview page for this newly added item, copy the URL from the lower right corner. This URL should include your organizations base domain.

 

Configure MobileMap Portal Base Map Settings

Use the Portal item URL copied from the item created in the step above to configure MobileMap to use this base map. To enter this URL in MobileMap settings go to Settings > Authentication > Portal Base Map URLs and enter as “servicename1:serviceURL1,servicename2:serviceURL2” where servicename is the human readable layer name (e.g., Imagery) and serviceURL is the full URL to the base map service.

Inventory Manager / Spatial CMS

Application Registration

If your organization uses InventoryManager or another Spatial CMS application and Portal, it will be necessary to register the web application as well. This process will be similar to MobileMap, but you will need to register the application with the following settings:

Next form

  • Title: InventoryManager

  • Tags: InventoryManager

  • Settings

    • Purpose: Self-configurable

    • API: Other

    • Register: Browser

  • Save

  • App Registration

    • Redirect URI:

      • Callback URL (e.g., https://yourproject.wsgapps.com)

 

Allowed Origins

In addition to the application registration, web applications like InventoryManager need to comply with Cross-origin resource sharing (CORS) security constraints. Use the Security section in Portal to add the URL listed above to the ‘Allow origins’ section. This will ensure that InventoryManager can successfully request data and authentication tokens from your Portal server.

Note: make sure there are no trailing slashes at the end of the URLs (e.g., https://yourportalurl.com) as these will cause CORS errors.

 

In Portal 10.9.1 the user interface looks like:

 

Portal-Specific Challenges

While Portal for ArcGIS provides the same functionality as AGOL from a MobileMap and InventoryManager perspective, some customers have encountered challenges when using Portal. Below is a brief list of issues that customers have encountered when implementing or transitioning our solution to Portal. To help prevent delays or performance problems, it can be helpful to ensure that IT staff are aware of some of these challenges and are included in the planning and implementation. It is also helpful to ensure that you have access to Esri technical support staff to help support your Portal implementation.

  • Versions

    • All ArcGIS Online users are using the same version of ArcGIS Online. This helps ensure consistent behavior for all customers that are using ArcGIS Online. When we go from the initial testing phase (using ArcGIS Online) to the implementation in Portal, there is sometimes a need to adjust configurations to account for differences between versions.

    • Issues that have been encountered have typically been related to subtle differences in the ArcGIS REST API (e.g., handling of exceededTransferLimit, default values) or error handling (more limited error messages in Portal).

  • Authentication & Security

    • When using Portal, both MobileMap and InventoryManager must be ‘whitelisted’ within your web server to prevent CORS issues. You can read more about CORS here, but a good description of the problem is “client-side scripts (e.g., JavaScript) are prevented from accessing much of the Web of Linked Data due to ‘same origin’ restrictions implemented in all major Web browsers.

      While enabling such access is important for all data, it is especially important for Linked Open Data and related services; without this, our data simply is not open to all clients.“ Each web server (e.g., IIS, nginx, Apache) will have a different approach to enabling CORS and whitelisting domains, and may require support from IT or someone who manages the customer’s web servers. IIS is the most common web server for Portal users and brief instructions for enabling CORS in IIS 10 can be found here.

    • When using Portal, both MobileMap and InventoryManager must be whitelisted with Portal to enable authentication via OAuth2.0. While this process is described in detail above, it does mean an extra step in configuring MobileMap and InventoryManager and requires admin privileges within Portal.

    • Ensure support for tokens and refresh tokens. We have seen some cases where users can sign-in as expected but can’t refresh their access tokens. Typically, this has been when the internet server is blocking calls to the refresh token service. The result is that their InventoryManager session only lasts for the length of their token, typically 30 minutes.

    • Security certificates - we have seen cases where a customer’s security certificate (SSL cert) on their ArcGIS Portal instance has timed out, or otherwise been deemed to be insecure. When this happens, MobileMap and InventoryManager will refuse transfer of data since all data are encrypted and thus cannot be sent if the server security cannot be validated

    • Multifactor Authentication (MFA) - we have seen cases where MFA has caused problems with authentication in MobileMap or InventoryManager. In one example, if a user failed to complete the MFA process on the first attempt (e.g., didn’t approve the sign-in on their cell phone) they were locked out of signing in to MobileMap for some period of time. This has to do with the way that the MFA is configured and cannot be controlled within MobileMap or InventoryManager.

  • Performance

    • ArcGIS Online provides sufficient performance for using very large datasets in MobileMap or InventoryManager. It is a highly scalable cloud-hosted SaaS solution built on best-in-class server and networking hardware.

    • Each Portal implementation is unique, and the performance is determined by a wide range of factors including the capabilities of the database server(s), web server(s), load balancer(s), network equipment, etc. ArcGIS settings including the maximum RAM and number of threads can have significant impact on performance. Other factors such as the database RDBMS that is used, database parameters such as block size, database compression, database triggers, etc. may impede performance. It can be hard to predict the performance of Portal prior to testing, but several customers have been surprised to see a significant drop in performance when transitioning from ArcGIS Online to Portal.

  • Database Technology (RDBMS)

    • MS SQL Server is the only RDBMS that is currently supported with our system.

    • Oracle is not supported (but has never been tested) with our system.

    • PostgreSQL is not supported. The ‘default data store’ in Portal is PostgreSQL which is not currently supported due to an issue that prevents explicit control over table and field name capitalization. Esri documentation on PostgreSQL can be found here: . Info on the mixed case limitation can be found here: and

  • Bugs

    • As of mid-November, 2023 Esri has logged at least 2 bugs with ArcGIS Enterprise (Portal) in which Editor Tracking does not update the last_edited_date or last_edited_user when features are updated. This can result in situations where MobileMap is unable to download updated feature data, because MobileMap does not see these data as modified (because their last_edited_date or last_edited_user fields did not update). While we cannot post details on these bugs here, we may be able to provide some additional details to Portal users upon request. This issue may be dependent on specific version(s) of ArcGIS Enterprise and/or the RDBMS used, but we don’t have those details at thist ime. The issue has not been reported for ArcGIS Online to the best of our knowledge.