Configuring an Accelerator Deployment with Distributed Gateway, Control Plane and Traffic Manager

The WSO2 API-M server can be deployed as an all-in-one deployment or as a distributed deployment. In the distributed setup, the API-M server profiles are deployed as separate API-M nodes.

distributed_gateway_control_plane.png

Given below are the API-M nodes you can have in a distributed deployment by default.

API-M Node (Profile) Description
Gateway Worker Node API-M nodes running the Gateway profile.
Control Plane Node API-M nodes running the Control Plane profile. The Control Plane includes the Key Manager, Publisher, and Developer Portal components.
Traffic Manager Node API-M nodes running the Traffic Manager profile.

See WSO2 API Manager Distributed Deployment documentation for more information on the distributed deployment of WSO2 API Manager.

This document provides instructions on how to set up the financial services accelerator with Identity Server and distributed Gateway, Control Plane and Traffic Manager nodes of WSO2 API Manager.

Prerequisites

  • Configure the JAVA_HOME environment variable to reference the directory where the Java Development Kit (JDK) is installed on the server. For more information, see Setting up JAVA_HOME.
  • Ensure that all necessary ports are open on each server to facilitate successful data communication. For more information, see Configuring ports.

Step 1: Setting Up WSO2 Financial Services IAM Accelerator

See WSO2 Open Banking IAM Accelerator documentation for instructions on how to set up the WSO2 Open Banking IAM Accelerator.

Configure WSO2 Identity Server

  1. Add following configurations in the /repository/conf/deployment.toml file. 

    [[event_listener]]
id = "token_revocation"
type = "org.wso2.carbon.identity.core.handler.AbstractIdentityHandler"
name = "org.wso2.is.notification.ApimOauthEventInterceptor"
order = 1

[event_listener.properties]
notification_endpoint = "https://<APIM_CP_HOST>:<APIM_PORT>/internal/data/v1/notify"
username = "${admin.username}"
password = "${admin.password}"
'header.X-WSO2-KEY-MANAGER' = "WSO2-IS"

  2. Download notification.event.handlers-2.0.5.jar and add it to <IS_HOME>/repository/components/dropins folder.

  3. Change the following config to false to disable FAPI with Manual Client Registration. 

    [oauth.dcr]
enable_fapi_enforcement=false

  4. Restart the IS server

Step 2: Setting Up WSO2 API-M Distributed Gateway, Control Plane and Traffic Manager Nodes

Installing base products

  1. Download and install the API-M base product. The following documentation provides the instruction to install below base product combination:
Base Product Combination
WSO2 Identity Server 7.1.0
WSO2 API Manager Components 4.5.0

Note

Three instances are required to set up the AM distributed deployment.

  • Instance 1: AM Gateway Profile
  • Instance 2: AM Control Plane Profile
  • Instance 3: AM Traffic Manager Profile

This document uses the following placeholders to refer to the following products:

Product Placeholder
WSO2 API-M Gateway Instance <APIM_GW_HOME>
WSO2 API-M Control Plane Instance <APIM_CP_HOME>
WSO2 API-M Traffic Manager Instance <APIM_TM_HOME>

Getting WSO2 Updates

The WSO2 Update tool delivers hotfixes and updates seamlessly on top of products as WSO2 Updates. See Getting WSO2 Updates to get the latest updates for the WSO2 API Manager.

Exchanging the Certificates

Generate a separate SSL certificate for each WSO2 API-M nodes and Identity Server (IS) node, and import them into both the keystore and truststore. This will prevent hostname mismatch issues related to SSL certificates.

Note

The same primary keystore should be used for all API Manager instances to decrypt the registry resources. For more information, see Configuring the Primary Keystore.

Follow the steps Exchanging the certificates to exchange the certificates between the WSO2 API-M nodes and Identity Server.

Setting Up WSO2 API-M Server and Start the Profiles

Setting up Databases

You can create the necessary databases for the API-M deployment on a separate server and configure each node to connect to the appropriate databases. Follow the instructions in the Setting up databases to create and configure the databases.

Installing WSO2 Open Banking APIM Accelerator

  1. Download and extract the latest Open Banking Accelerator 4.x.x AM Accelerator.

  2. Place the extracted APIM accelerator to the root directories of both Gateway node, Control Plane node and Traffic Manager node.

  3. Run the merge script in <APIM_HOME>/<OB_APIM_ACCELERATOR_HOME>/bin:

    ./merge.sh
    ./merge.sh
    ./merge.ps1

  4. Run the configure file in <APIM_HOME>/<OB_APIM_ACCELERATOR_HOME>/bin:

    ./configure.sh
    ./configure.sh

    ``` powershell tab='On Windows' ./configure.ps1

Configure the Gateway Node

Configure the Gateway to communicate with the Control Plane.

  1. Open the <APIM_GW_HOME>/repository/conf/deployment.toml file.

  2. Set the hostname of the API Manager: ``` toml [server] hostname = ""

  3. Configure super admin credentials. 

    [super_admin]
username = "<APIM_SUPER_ADMIN_USERNAME>"
password = "<APIM_SUPER_ADMIN_PASSWORD>"
create_admin_account = true

  4. Configure the following configs to enable email as username. 

    [tenant_mgt]
enable_email_domain = true

[user_store]
type = "database_unique_id"
class = "org.wso2.carbon.user.core.jdbc.UniqueIDJDBCUserStoreManager"

  5. Update the datasource configurations with your database properties, such as the username, password, JDBC URL for the database server, and the JDBC driver. Sample shows configuring datasource using MySQL.

  6. Add shared_db and apim db configurations.

    • Given below are sample configurations for a MySQL database. For other DBMS types and more information, see Setting up databases.

      [database.shared_db]
url = "jdbc:mysql://<DB_HOST>:3306/am_configdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
      [database.apim_db]
url = "jdbc:mysql://<DB_HOST>:3306/apimgtdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"

  7. Update the following configurations with the hostname of the Identity Server to connect the Gateway to the Key Manager component in the Control Plane: 

    [apim.key_manager]
service_url = "https://<IS_HOST>:9446/services/"
username = "<IS_SUPER_ADMIN_USERNAME>"
password = "<IS_SUPER_ADMIN_PASSWORD>"

  8. Disable subscription validation as follows. 

    [apim.key_manager]
allow_subscription_validation_disabling = true

  9. Configure the following traffic manager configurations. 

    [apim.throttling]
throttle_decision_endpoints = ["tcp://<APIM_TM_HOST>:5672"]
username= "$ref{super_admin.username}@carbon.super"
password= "$ref{super_admin.password}"

[apim.throttling.policy_deploy]
username = "$ref{super_admin.username}@carbon.super"

[apim.throttling.jms]
username="<NAME_OF_APIM_SUPER_ADMIN_USER>!wso2.com!carbon.super"
password = "$ref{super_admin.password}"

[apim.event_hub.jms]
username="<NAME_OF_APIM_SUPER_ADMIN_USER>!wso2.com!carbon.super"
password= "$ref{super_admin.password}"

[[apim.throttling.url_group]]
traffic_manager_urls = ["tcp://<APIM_TM_HOST>:9611"]
traffic_manager_auth_urls = ["ssl://<APIM_TM_HOST>:9711"]

  10. Add event hub configuration.

    !!!note Event hub configuration is used to retrieve Gateway artifacts. Using event_listening_endpoints, the Gateway will create a JMS connection with the event hub that is then used to subscribe for API/Application/Subscription and Key Manager operations-related events. The service_url points to the internal API that resides in the event hub that is used to pull artifacts and information from the database.

    [apim.event_hub]
enable = true
username = "$ref{super_admin.username}"
password = "$ref{super_admin.password}"
service_url = "https://<APIM_CP_HOST>:9443/services/"
event_listening_endpoints = ["tcp://<APIM_CP_HOST>:5672"]

  11. Add the following config to support JWT content type. 

    [[blocking.custom_message_formatters]]
class = "org.apache.axis2.format.PlainTextFormatter"
content_type = "application/jwt"

[[blocking.custom_message_builders]]
class = "org.apache.axis2.format.PlainTextBuilder"
content_type = "application/jwt"

[[custom_message_formatters]]
class = "org.apache.axis2.format.PlainTextFormatter"
content_type = "application/jwt"

[[custom_message_builders]]
class = "org.apache.axis2.format.PlainTextBuilder"
content_type = "application/jwt"

  12. Add the following to configure transport layer configs. 

    [transport.passthru_https.sender.parameters]
HostnameVerifier = "AllowAll"

[passthru_http]
"http.headers.preserve"="Content-Type,Date"

[transport.passthru_https.listener.parameters]
HttpsProtocols = "TLSv1.2"
PreferredCiphers = "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"

  13. Enable certificate chain validation as the following. 

    [apimgt.mutual_ssl]
enable_certificate_chain_validation = true

  14. Disable certificate_revocation. 

    [transport.passthru_https.listener.cert_revocation_validation]
enable = false
allow_cert_expiry_validation = false
allow_full_cert_chain_validation = false
cache_delay = 1000
cache_size = 1024

  15. Configure the following to skip dropping the AUTH header from gateway and to configure allowed scopes. 

    [apim.oauth_config]
enable_outbound_auth_header = true
white_listed_scopes = ["^device_.*", "openid", "^FS_.*", "^TIME_.*"]

  16. Configure publisher url. 

    [financial_services]
publisher_url="https://<APIM_CP_HOST>:9443/publisher"

  17. Add the following configurations to enable the external service extension. 

    [financial_services.extensions.endpoint]
enabled = true
allowed_extensions = ["pre-process-application-creation", "pre-process-application-update"]
base_url = "<EXTERNAL_SERVICE_URL>"
retry_count = 5
connect_timeout = 5000
read_timeout = 5000

[financial_services.extensions.endpoint.security]
# supported types : Basic-Auth or OAuth2
type = "Basic-Auth"
username = "<EXTERNAL_SERVICE_USERNAME>"
password = "<EXTERNAL_SERVICE_PASSWORD>"

Configure custom error response

  1. Add the following configuration to the gateway node to skip removing the sequence file: 

    [apim.sync_runtime_artifacts.gateway]
skip_list.sequences = ["customErrorFormatter.xml"]

  2. Add this line before the closing of the sequence tag </sequence> in the _cors_request_handler_.xml file found in location <APIM_GW_HOME>/repository/deployment/server/synapse-configs/default/sequences/_cors_request_handler_.xml: 

    <sequence key="customErrorFormatter"/>

  3. Copy the customErrorFormatter.xml file resides in fs-apim-mediation-artifacts-1.0.0/custom-sequences directory to the <APIM_GW_HOME>/repository/deployment/server/synapse-configs/default/sequences.

Tip

Download a sample deployment.toml configuration file here.

Configure the Control Plane Node

Configure the Control Plane to communicate with the Gateway.`

  1. Open the <APIM_CP_HOME>/repository/conf/deployment.toml file.

  2. Set the hostname of the API Manager: 

    [server]
hostname = "<APIM_CP_HOST>"

  3. Configure super admin credentials. 

    [super_admin]
username = "<APIM_SUPER_ADMIN_USERNAME>"
password = "<APIM_SUPER_ADMIN_PASSWORD>"
create_admin_account = true

  4. Configure the following configs to enable email as username. 

    [tenant_mgt]
enable_email_domain = true

[realm_manager]
data_source= "WSO2USER_DB"

[user_store]
type = "database_unique_id"
class = "org.wso2.carbon.user.core.jdbc.UniqueIDJDBCUserStoreManager"

  5. Update the datasource configurations with your database properties, such as the username, password, JDBC URL for the database server, and the JDBC driver. Sample shows configuring datasource using MySQL.

    • Given below are sample configurations for a MySQL database. For other DBMS types and more information, see Setting up databases.

      [database.apim_db]
url = "jdbc:mysql://<DB_HOST>:3306/apimgtdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
      [database.shared_db]
url = "jdbc:mysql://<DB_HOST>:3306/am_configdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
      [database.config]
url = "jdbc:mysql://<DB_HOST>:3306/am_configdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
      [[datasource]]
id="WSO2UM_DB"
url = "jdbc:mysql://<DB_HOST>:3306/userdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
jmx_enable=false
pool_options.maxActive = "150"
pool_options.maxWait = "60000"
pool_options.minIdle = "5"
pool_options.testOnBorrow = true
pool_options.validationQuery="SELECT 1"
pool_options.validationInterval="30000"
pool_options.defaultAutoCommit=true

  6. Update the following configurations with the hostname of the Identity Server. 

    [apim.key_manager]
service_url = "https://<IS_HOST>:9446/services/"
username = "<IS_SUPER_ADMIN_USERNAME>"
password = "<IS_SUPER_ADMIN_PASSWORD>"

  7. Disable subscription validation as follows. 

    [apim.key_manager]
allow_subscription_validation_disabling = true

  8. Configure the gateway environment. 

    [apim]
gateway_type = "Regular,APK,AWS"

[[apim.gateway.environment]]
name = "Default"
type = "hybrid"
gateway_type = "Regular"
provider = "wso2"
display_in_api_console = true
description = "This is a hybrid gateway that handles both production and sandbox token traffic."
show_as_token_endpoint_url = true
service_url = "https://<APIM_GW_HOST>:${mgt.transport.https.port}/services/"
username= "${super_admin.username}"
password= "${super_admin.password}"
ws_endpoint = "ws://<APIM_GW_HOST>:9099"
wss_endpoint = "wss://<APIM_GW_HOST>:8099"
http_endpoint = "http://<APIM_GW_HOST>:8280"
https_endpoint = "https://<APIM_GW_HOST>:8243"
websub_event_receiver_http_endpoint = "http://<APIM_GW_HOST>:9021"
websub_event_receiver_https_endpoint = "https://<APIM_GW_HOST>:8021"

    Info

    This configuration is used for deploying APIs to the Gateway and for connecting the Developer Portal component to the Gateway if the Gateway is shared across tenants. If the Gateway is not used by multiple tenants, you can create a Gateway Environment using the Admin Portal. Note that in the above configurations, service_url points to the 9443 port of the Gateway node, while http_endpoint and https_endpoint points to the http and https nio ports (8280 and 8243).

  9. Configure the following traffic manager configurations. 

    [apim.throttling]
throttle_decision_endpoints = ["tcp://<APIM_TM_HOST>:5672"]
username= "$ref{super_admin.username}@carbon.super"
password= "$ref{super_admin.password}"

[apim.throttling.policy_deploy]
username = "$ref{super_admin.username}@carbon.super"
password = "$ref{super_admin.password}"

[apim.throttling.jms]
username = "am_admin!wso2.com!carbon.super"
password = "$ref{super_admin.password}"

[apim.event_hub.jms]
username="am_admin!wso2.com!carbon.super"
password= "$ref{super_admin.password}"

[[apim.throttling.url_group]]
traffic_manager_urls = ["tcp://<APIM_TM_HOST>:9611"]
traffic_manager_auth_urls = ["ssl://<APIM_TM_HOST>:9711"]

  10. Add the following configuration to increase the header size. 

    [transport.https.properties]
maxHttpHeaderSize = 16384

  11. Configure token exchange.

    [oauth.grant_type.token_exchange]
enable = true
allow_refresh_tokens = true
iat_validity_period = "1h"

  12. Configure cache invalidation. 

    [apim.cache_invalidation]
enable = true
domain = "control-plane-domain"

  13. Add event hub configuration.

    [apim.event_hub]
enable = true
username = "$ref{super_admin.username}"
password = "$ref{super_admin.password}"
service_url = "https://<APIM_CP_HOST>:9443/services/"
event_listening_endpoints = ["tcp://<APIM_CP_HOST>:5672"]

  14. To configure additional application attributes for Manual Client Registration, follow the steps in Configuring Additional Attributes.

Configure the Traffic Manager Node

Configure the Traffic Manager to communicate with the Control Plane.

  1. Open the <APIM_TM_HOME>/repository/conf/deployment.toml file.

  2. Set the hostname of the API Manager: 

    [server]
hostname = "<APIM_TM_HOST>"

  3. Configure super admin credentials. 

    [super_admin]
username = "<APIM_SUPER_ADMIN_USERNAME>"
password = "<APIM_SUPER_ADMIN_PASSWORD>"
create_admin_account = true

  4. Configure the following configs to enable email as username. 

    [tenant_mgt]
enable_email_domain = true

[user_store]
type = "database_unique_id"
class = "org.wso2.carbon.user.core.jdbc.UniqueIDJDBCUserStoreManager"

  5. Add shared_db and apim db configurations.

    • Given below are sample configurations for a MySQL database. For other DBMS types and more information, see Setting up databases.

      [database.shared_db]
url = "jdbc:mysql://<DB_HOST>:3306/am_configdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"
      [database.apim_db]
url = "jdbc:mysql://<DB_HOST>:3306/apimgtdb?autoReconnect=true&amp;useSSL=false"
username = "root"
password = "root"
driver = "com.mysql.jdbc.Driver"

  6. Update the following configurations with the hostname of the Identity Server. 

    [apim.key_manager]
service_url = "https://<IS_HOST>:9446/services/"
username = "<IS_SUPER_ADMIN_USERNAME>"
password = "<IS_SUPER_ADMIN_PASSWORD>"

  7. Disable subscription validation as follows. 

    [apim.key_manager]
allow_subscription_validation_disabling = true

  8. Configure the gateway environment. 

    [[apim.gateway.environment]]
name = "Default"
type = "hybrid"
gateway_type = "Regular"
provider = "wso2"
display_in_api_console = true
description = "This is a hybrid gateway that handles both production and sandbox token traffic."
show_as_token_endpoint_url = true
service_url = "https://<APIM_GW_HOST>:${mgt.transport.https.port}/services/"
username= "${super_admin.username}"
password= "${super_admin.password}"
ws_endpoint = "ws://<APIM_GW_HOST>:9099"
wss_endpoint = "wss://<APIM_GW_HOST>:8099"
http_endpoint = "http://<APIM_GW_HOST>:8280"
https_endpoint = "https://<APIM_GW_HOST>:8243"
websub_event_receiver_http_endpoint = "http://<APIM_GW_HOST>:9021"
websub_event_receiver_https_endpoint = "https://<APIM_GW_HOST>:8021"

!!! info This configuration is used for deploying APIs to the Gateway and for connecting the Developer Portal component to the Gateway if the Gateway is shared across tenants. If the Gateway is not used by multiple tenants, you can create a Gateway Environment using the Admin Portal. Note that in the above configurations, service_url points to the 9443 port of the Gateway node, while http_endpoint and https_endpoint points to the http and https nio ports (8280 and 8243).

  1. Configure the following traffic manager configurations. 

    [apim.throttling]
throttle_decision_endpoints = ["tcp://<APIM_TM_HOST>:5672"]
username= "$ref{super_admin.username}@carbon.super"
password= "$ref{super_admin.password}"

[apim.throttling.policy_deploy]
username = "$ref{super_admin.username}@carbon.super"
password = "$ref{super_admin.password}"

[apim.throttling.jms]
username = "am_admin!wso2.com!carbon.super"
password = "$ref{super_admin.password}"

[apim.event_hub.jms]
username="am_admin!wso2.com!carbon.super"
password= "$ref{super_admin.password}"

[[apim.throttling.url_group]]
traffic_manager_urls = ["tcp://<APIM_TM_HOST>:9611"]
traffic_manager_auth_urls = ["ssl://<APIM_TM_HOST>:9711"]

  2. Add event hub configuration. 

    [apim.event_hub]
enable = true
username = "$ref{super_admin.username}"
password = "$ref{super_admin.password}"
service_url = "https://<APIM_CP_HOST>:9443/services/"
event_listening_endpoints = ["tcp://<APIM_CP_HOST>:5672"]

Start the servers

  1. Start the Traffic Manager Node.

    • Navigate to the <APIM_TM_HOME>/bin directory and run the following command: 
      sh traffic-manager.sh

  2. Start the Gateway Worker Node.

    • Navigate to the <APIM_GW_HOME>/bin directory and run the following command: 
      sh gateway.sh

  3. Start the Control Plane Node.

  4. Navigate to the <APIM_CP_HOME>/bin directory and run the following command: 
    sh api-cp.sh
Top