Appendix - Set up the Universal Orchestrator to Use Client Certificate Authentication via a Reverse Proxy: NetScaler ADC

The Keyfactor Universal OrchestratorClosed The Keyfactor Universal Orchestrator, one of Keyfactor's suite of orchestrators, is used to interact with servers and devices for certificate management, run SSL discovery and management tasks, and manage synchronization of certificate authorities in remote forests. With the addition of custom extensions, it can provide certificate management capabilities on a variety of platforms and devices (for example Amazon Web Services (AWS) resources, Citrix\NetScaler devices, F5 devices, IIS stores, JKS keystores, PEM stores, and PKCS #12 stores) and execute tasks outside the standard list of certificate management functions. It runs on either Windows or Linux servers or Linux containers. can be configured to use client certificate authentication through a reverse proxy or network edge device, such as a NetScaler ADC or F5, that performs TLSClosed TLS (Transport Layer Security) and its predecessor SSL (Secure Sockets Layer) are protocols for establishing authenticated and encrypted links between networked computers. termination.

The reverse proxy authenticates the orchestratorClosed Keyfactor orchestrators perform a variety of functions, including managing certificate stores and SSH key stores. using its client certificate, then forwards that certificate to Keyfactor Command in a request header. Keyfactor Command uses this header to authenticate the orchestrator at the orchestrator endpointClosed An endpoint is a URL that enables the API to gain access to resources on a server.. In both authentication methods described here, this certificate-in-header exchange is identical. The two methods differ only in how the reverse proxy itself authenticates to IIS on the Keyfactor Command server:

  • Basic authentication credentials are stored on the reverse proxy and presented to IIS as part of the request. These same credentials authorize the orchestrator in Keyfactor Command.

  • A second client certificate is stored on the reverse proxy and used to authenticate to IIS. Basic authentication credentials stored in IIS on the Keyfactor Command server authorize the orchestrator instead, so those credentials never travel over the network.

The following instructions cover one method of configuring a NetScaler ADC device to support this.

Complete the following steps and then configure the orchestrator to support client certificate authentication per the installation instructions (see --client-auth-certificate (Client Certificate Authentication) or Install the Universal Orchestrator on Windows).

Create the following two rewrite actions.

Tip:  If you're using a second client certificate to authenticate the proxy to Keyfactor Command, you only need to create the first of these actions.

Capture the client certificate from the orchestrator:

  1. In the NetScaler ADC GUI, browse to AppExpert > Rewrite > Actions.
  2. On the Rewrite Actions page, click Add.
  3. On the Create Rewrite Action page, enter a Name for the action that will take the certificate received from the orchestrator and convert it to PEMClosed A PEM format certificate file is a Base64-encoded certificate. Since it's presented in ASCII, you can open it in any text editor. PEM certificates always begin and end with entries like ---- BEGIN CERTIFICATE---- and ----END CERTIFICATE----. PEM certificates can contain a single certificate or a full certificate chain and may contain a private key. In general, extensions of .cer and .crt are certificate files with no private key, .key is a separate private key file, and .pem is both a certificate and private key. format (for example, CaptureClientCert).
  4. Give the action a Type of INSERT_HTTP_HEADER.
  5. Give the action a Header Name (for example, NS-ClientCert). Be sure to make a note of this header name. You will need it later when you configure certificate authentication for the orchestrator.
  6. Enter an Expression to convert the client authentication certificate to PEM format:
    CLIENT.SSL.CLIENT_CERT.TO_PEM
  7. Enter Comments if desired and click OK to save the action.

Store basic authentication credentials to authenticate the proxy to IIS on the Keyfactor Command server and provide authorization information:

  1. Click Add to add another action.
  2. On the Create Rewrite Action page, enter a Name for the action that will send the basic authentication credentials for the orchestrator to Keyfactor Command (for example, SendServiceCreds).
  3. Give the action a Type of INSERT_HTTP_HEADER.
  4. Give the action a Header Name of Authorization.
  5. Enter an Expression to send Base64-encoded basic authentication credentials to the Keyfactor Command server (where service@keyexample.com and MySecurePassword are the correct service name and password for your environment):
    "Basic "+("service@keyexample.com"+":"+"MySecurePassword").B64ENCODEM
  6. Enter Comments if desired and click OK to save the action.

Create the following two rewrite policies.

Tip:  If you're using a second client certificate to authenticate the proxy to Keyfactor Command, you only need to create the first of these policies.

Put the client certificate from the orchestrator in the header:

  1. In the NetScaler ADC GUI, browse to AppExpert > Rewrite > Policies.
  2. On the Rewrite Policies page, click Add.
  3. On the Create Rewrite Policy page, enter a Name for the policy that will confirm that a certificate has been received from the orchestrator and run the action to convert it to PEM format (for example, NS-GetCert).
  4. Give the policy the Action you created in the previous section to capture the client authentication certificate (for example, CaptureClientCert).
  5. Define a Log Action if desired.
  6. Set the Undefined-Result Action to -Global-undefined-result-action-.
  7. Enter an Expression to validate that the client authentication certificate has been received from the orchestrator:
    CLIENT.SSL.CLIENT_CERT.EXISTS
  8. Enter Comments if desired and click OK to save the policy.

Send the basic authentication credentials to the Keyfactor Command server:

  1. Click Add to add another policy.
  2. On the Create Rewrite Policy page, enter a Name for the policy that will send the basic authentication credentials for the orchestrator to the Keyfactor Command server (for example, NS-SendCreds).
  3. Give the policy the Action you created in the previous section to send the basic authentication credentials (for example, SendServiceCreds).
  4. Define a Log Action if desired.
  5. Set the Undefined-Result Action to -Global-undefined-result-action-.
  6. Enter an Expression to confirm that the authorization header does not already exist in the request header:
    HTTP.REQ.HEADER("Authorization").EXISTS.NOT
  7. Enter Comments if desired and click OK to save the policy.

Create the following responder policy.

Validate that the client certificate presented by the orchestrator was issued by the specified issuing CAClosed A certificate authority (CA) is an entity that issues digital certificates. Within Keyfactor Command, a CA may be a Microsoft CA or a Keyfactor gateway to a cloud-based or remote CA.:

  1. In the NetScaler ADC GUI, browse to AppExpert > Responder > Policies.
  2. On the Responder Policies page, click Add.
  3. On the Create Responder Policy page, enter a Name for the policy that will validate that the certificate received from the orchestrator was issued by the correct CA (for example, NS-ValidateIssuer).
  4. Select an Action of Reset.
  5. Define a Log Action if desired.
  6. Do not configure an AppFlow Action.
  7. Set the Undefined-Result Action to -Global-undefined-result-action-.
  8. Enter an Expression to confirm that the certificate received from the orchestrator was issued from the correct issuing CA (where CorpIssuingCA is the logical nameClosed The logical name of a certificate authority (CA) is the human-readable name assigned to the CA when it is created. This name is used to uniquely identify the CA within its management system and in Keyfactor Command configuration settings. For some CA types, such as Microsoft CAs, the logical name is combined with the CA host name to form a fully qualified identifier (for example, ca1.keyexample.com\Corp Issuing CA One) that may be used in command-line tools and configuration references. The exact format and usage of the logical name can vary by CA type. of your CA):
    CLIENT.SSL.CLIENT_CERT.ISSUER.CONTAINS("CorpIssuingCA").NOT
    Tip:  Connections from the orchestrator will fail if the client authentication certificate was issued by any CA other than the one configured here. You can use AND logic to add more than one CA. For example:
    CLIENT.SSL.CLIENT_CERT.ISSUER.CONTAINS("CorpIssuingCA1").NOT && CLIENT.SSL.CLIENT_CERT.ISSUER.CONTAINS("CorpIssuingCA2").NOT

    With this expression, certificates issued from either one of these CAs would be accepted.

  9. Enter Comments if desired and click OK to save the policy.
Important:  Once you modify the virtual server to require certificates for authentication, many other Keyfactor Command transactions will no longer function if they are sharing the same virtual server. Be sure that you are using a separate virtual server for incoming requests to /KeyfactorAgents on the Keyfactor Command server versus other types of requests. The following instructions refer to setting all policies on a single load balancing virtual server, but your configuration may include multiple virtual servers of other types, which may require slight modifications to these instructions.

Modify the configuration for your load balancing virtual server that is used for Keyfactor Command KeyfactorAgent requests as follows.

Configure the NetScaler ADC device to authenticate the orchestrator using its client certificate:

  1. In the NetScaler ADC GUI, browse to Traffic Management > Load Balancing > Virtual Servers.
  2. On the Virtual Servers page, select your virtual server and click Edit.
  3. In the SSLClosed TLS (Transport Layer Security) and its predecessor SSL (Secure Sockets Layer) are protocols for establishing authenticated and encrypted links between networked computers. Parameters section, click to edit, check the Client Authentication box, and set the Client Certificate dropdown to Mandatory.

Associate the two rewrite policies.

Tip:  If you're using a second client certificate to authenticate the proxy to Keyfactor Command, you only need to associate the first of these policies.
  1. On the Virtual Servers page, under Advanced Settings expand Policies.
  2. In the Policies section, click the plus to add a new policy.
  3. On the Choose Type page, select Choose Policy Rewrite and Choose Type Request and click Continue.
  4. On the Choose Type page, click Add Binding.
  5. On the Policy Binding page, click the Select Policy field and on the Rewrite Policies page select the radio button for the rewrite policy you created to capture the client authentication certificate (for example, NS-GetCert). Click Select to save the selection.
  6. On the Policy Binding page, set a Priority of 110.
  7. Set Goto Expression to Next.
  8. Set Invoke LabelType to None.
  9. Click Bind to save the binding.
  1. On the Choose Type page for Rewrite Request, click Add Binding.
  2. On the Policy Binding page, click the Select Policy field and on the Rewrite Policies page select the radio button for the rewrite policy you created to send the service account credentials to the Keyfactor Command server (for example, NS-SendCreds). Click Select to save the selection.
  3. On the Policy Binding page, set a Priority of 120.
  4. Set Goto Expression to Next.
  5. Set Invoke LabelType to None.
  6. Click Bind to save the binding.
  7. Click Close to return to the virtual server settings page.

Associate the responder policy:

  1. On the Virtual Servers page, in the Policies section, click the plus to add a new policy.
  2. On the Choose Type page, select Choose Policy Responder and Choose Type Request and click Continue.
  3. On the Choose Type page, click Add Binding.
  4. On the Policy Binding page, click the Select Policy field and on the Responder Policies page select the radio button for the responder policy you created to validate the issuer of the client authentication certificate (for example, NS-ValidateIssuer). Click Select to save the selection.
  5. On the Policy Binding page, set a Priority of 100.
  6. Set Goto Expression to END.
  7. Set Invoke LabelType to None.
  8. Click Bind to save the binding.
  9. Click Close to return to the virtual server settings page.

Once you have all the components configured on Citrix, you're ready to configure Keyfactor Command to support client certificate authentication for the orchestrators. Once you do this, all orchestrators connecting to this instance of Keyfactor Command will be required to provide a certificate to authenticate. If you have some orchestrators deployed that do not support certificate authentication (for example, Java agents), you will need to design a solution with multiple Keyfactor Command servers to support multiple authentication types. Contact your Keyfactor representative for assistance with this.

To configure Keyfactor Command to require client certificate authentication for orchestrators:

  1. On the Keyfactor Command server, open the Keyfactor Configuration Wizard.
  2. In the Certificate Authentication section of the Orchestrators tab, check the Enabled box.
  3. In the Certificate Authentication HTTP Header field, enter the Header Name you gave to the rewrite action you created to capture the certificate from the orchestrator (for example, NS-ClientCert). Keyfactor Command uses the certificate supplied in this header to identify the orchestrator attempting to authenticate.
  4. In the Certificate Authentication Username and Certificate Authentication Password fields, enter the credentials for an Active Directory service account that will be used to provide authorization in Keyfactor Command for the orchestrators.
    Tip:  The service account entered here is an Active Directory service account that Keyfactor Command uses to authorize certificate-authenticated orchestrators. This account is required by the Keyfactor Command configuration but does not need to be the same account used by the orchestrator service itself.
  5. Click Verify Configuration and Apply Configuration.

Figure 563: Configure Keyfactor Command for Client Certificate Authentication

If you have opted to configure the NetScaler ADC device to use a client certificate to authenticate from the device to the Keyfactor Command server instead of submitting basic authentication credentials from the device, you will need to configure IIS on the Keyfactor Command server to recognize the client certificate for authentication and then use basic authentication credentials on the Keyfactor Command server to provide authorization to Keyfactor Command. You will also need to configure Keyfactor Command to force it to use the client certificate from the orchestrator stored in the header to authenticate the orchestrator, not the client certificate presented by the proxy in the second hop of the transaction.

On your Keyfactor Command server, install the following additional module:

  • IIS Client Certificate Mapping Authentication (rather than Client Certificate Mapping Authentication)

    Tip:  It's fine to install both IIS Client Certificate Mapping Authentication and Client Certificate Mapping Authentication, but the former is what's needed for this solution.

If you have more than one Keyfactor Command server with separated roles, this only needs to be installed on the server accepting traffic to the /KeyfactorAgents web application.

Figure 564: IIS Module for Client Certificate Authentication

The PowerShell command to install the appropriate module is:

Add-WindowsFeature Web-Cert-Auth

Make the following changes in the IIS Management console on the Keyfactor Command server:

  1. In the IIS Management console, highlight the server name on the left and open Authentication. Make sure Anonymous Authentication is the only allowed method.

    Figure 565: Configure only Anonymous Authentication at the Server Level in IIS

  2. In the IIS Management console, drill down into sites and into the Default Web Site (or other web site if your Keyfactor Command instance has been installed in an alternate web site). Under the Default Web Site, locate the KeyfactorAgents application and open Authentication for this. Set all the authentication methods shown here to Disabled.

    Figure 566: Turn Off Authentication Methods at the Application Level in IIS

    Tip:  If your KeyfactorAgents endpoint is running on a standalone server with no other Keyfactor roles, you should also turn off all authentication methods at the Default Web Site level as in step two. If your server holds other Keyfactor roles, leave this in the default configuration with Anonymous being the only authentication method allowed as in step one.

  3. In the IIS Management console, open SSL Settings for the KeyfactorAgents application. Check the Require SSL box and select either Require or Accept for Client certificates.

    Important:  Select Require only if all the orchestrators you are using support client certificate authentication and you plan to configure them all for certificate authentication.

    Figure 567: Configure SSL Settings in IIS for Client Certificate Authentication

    Tip:  If your KeyfactorAgents endpoint is running on a standalone server with no other Keyfactor roles, you may also configure your server to Require or Accept for Client certificates at the Default Web Site level. It is good security practice to check the Require SSL box. If your KeyfactorAgents endpoint is running on a server with other Keyfactor roles, you do not need to accept client certificates at this level and should not require them at this level.

Make the following changes in the IIS Management console on the Keyfactor Command server:

  1. In the IIS Management console, drill down to the Default Web Site (or other web site if your Keyfactor Command instance has been installed in an alternate web site). In the Default Web Site, open the Configuration Editor tool.

  2. In the Configuration Editor tool at the Default Web Site level, browse to:

    system.webServer/security/authentication/iisClientCertificateMappingAuthentication
    Important:  Don't be tempted to configure this setting only at the application level (KeyfactorAgents) rather than at the Default Web Site level. It will only work if configured at the Default Web Site level and then allowed at the application level.

  3. In the configurations for IIS Client Certificate Mapping Authentication, set the defaultLogonDomain to your forestClosed An Active Directory forest (AD forest) is the top most logical container in an Active Directory configuration that contains domains, and objects such as users and computers. root. Set the manyToOneCertificateMappingsEnabled option to True and the oneToOneCertificateMappingEnabled option to False. Click the dots to the right of the manyToOneMappings setting to open details for this setting.

    Figure 568: Configure IIS Client Certificate Mapping Authentication for the Default Web Site

  4. In the CollectionClosed The certificate search function allows you to query the Keyfactor Command database for certificates from any available source based on any criteria of the certificates and save the results as a collection that will be available in other places in the Management Portal (for example expiration alerts and certain reports). Editor for the manytoOneMappings, click Add and enter appropriate values for the properties. The service account entered here will be used as the identity in Keyfactor Command of all orchestrators that authenticate via client certificate.

    Figure 569: Configure Authorization Credentials for Keyfactor Orchestrators

  5. In the IIS Management console, drill down into sites and into the Default Web Site (or other web site if your Keyfactor Command instance has been installed in an alternate web site). Under the Default Web Site, locate the KeyfactorAgents application and open the Configuration Editor tool for it.

  6. In the Configuration Editor tool at the KeyfactorAgents application level, browse to:

    system.webServer/security/authentication/iisClientCertificateMappingAuthentication

    Turn on the mapping authentication option at this level. The configuration should have replicated down from the Default Web Site level.

When the orchestrator is configured to use a client certificate to authenticate to a proxy and then the proxy is configured to use a separate client certificate to authenticate to the Keyfactor Command server, authentication to the Keyfactor Command application should be done using the original certificate from the orchestrator, not the certificate inserted in the process at the proxy level. This is done by including the original certificate from the orchestrator in the request header to Keyfactor Command. To assure that Keyfactor Command gives priority to this certificate and not the certificate the proxy uses to authenticate, set the Keyfactor Command authentication application setting Always Use Certificate from Header to True.

Figure 570: Configure Application Setting in Keyfactor Command to use the Header Certificate

Tip:  In some proxy configurations, the proxy may be unable to negotiate the client certificate handshake with IIS. IIS won't ask directly for a client certificate, and if, during the handshake, the proxy doesn't send one, the client authentication will fail. If this occurs, you may need to turn on client certificate negotiation at a lower level below IIS. To do this:
  1. On the Keyfactor Command server, open a command prompt using the Run as administrator option.

  2. Execute the following command to output the current configuration for SSL certificate bindings:

    netsh http show sslcert

    Output from this command will look something like this (you may see multiple sections if you have multiple web sites on the server):

    SSL Certificate bindings:
-------------------------
IP:port                      : 0.0.0.0:443
Certificate Hash             : 649dfa6df693583f609af499fe4237f2c1d64224
Application ID               : {4dc3e181-e14b-4a21-b022-59fc669b0914}
Certificate Store Name       : My
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check                  : Enabled
Revocation Freshness Time    : 0
URL Retrieval Timeout        : 0
Ctl Identifier               : (null)
Ctl Store Name               : (null)
DS Mapper Usage              : Enabled
Negotiate Client Certificate : Disabled
Reject Connections           : Disabled
Disable HTTP2                : Not Set
Disable QUIC                 : Not Set
Disable TLS1.2               : Not Set
Disable TLS1.3               : Not Set
Disable OCSP Stapling        : Not Set
Disable Legacy TLS Versions  : Not Set

  3. Look at the value for the Negotiate Client Certificate setting for the web site on which Keyfactor Command is installed.

    If the value is Disabled, retrieve from the output the values for the IP:port, Certificate Hash, and Application ID.

  4. Execute the following commands to remove and re-add the IP:port with Negotiate Client Certificate allowed (referencing the correct values for ipport, certhash, and appid):

    netsh http delete sslcert ipport=0.0.0.0:443
    netsh http add sslcert ipport=0.0.0.0:443 certhash=649dfa6df693583f609af499fe4237f2c1d64224 appid={4dc3e181-e14b-4a21-b022-59fc669b0914} clientcertnegotiation=enable
  5. Execute the show command again to confirm that the setting is now shown as enabled.
  6. Restart the IIS services (iisreset) and try the certificate authentication again.