Guides
R24 AMR | Min(s) read

Pre-Installations for DES

The following pre-installations are performed before AKS creation.

  • Transact pods to be deployed in AKS.
  • DES database details are required.
  • Azure Event Hub to be created and details like primary key, and connection strings are to be noted which would be used when installing DES.

For DES to stream the events emitted by Transact application, you must check if the criterion are available as a part of Transact installation.

  • Enable the commit capture in tafj.properties with the value below.
    temn.tafj.runtime.use.df.cache=true
  • Install the Enterprise Streaming (ES) product in SPF.

  • Set the Data Stream to Yes in the SYSTEM record in SPF.

An image is a standalone, executable software package that is necessary to run an application like code, runtime, system tools, system libraries and settings.

Follow the steps below to run a build environment with internet and target database (both Transact and streaming database) connectivity to build DES images.

  1. Download DES release pack from distribution and unzip the contents in your build machine.
  2. Set Apache Maven and Java home in the system variables.
  3. Go to the artifacts folder in the package and run the command below.
    4. mvn clean install
  4. Open the README.md file in the des-docker folder and set the variables according to the requirement to build DES images as shown below.

    5. Variable required for DES
    DES_DOCKER_ENV=t24
    DES_DATABASE_VENDOR=sqlserver 
    DES_STEAM_VENDOR=eventhub 
    DES_TOOL_BUILD=true

  5. To install database-related scripts, DES requires the source and stream database details (like JDBC URL, username and password). These values can be overridden as environment variables. Read the Overriding Keystore Entries section for more information.
  6. Edit the DES configuration property according to the requirement (for example, changing the table profiles configured for DES to listen, that is, temn.des.epa. data-event.tables=F_DATA_EVENTS, FBNK_DATA_EVENTS_ILP).

    If Event Hub acts as a streaming platform, use the des-eventhub-t24.properties file. This file is available in one of the paths below.

    • Path 1: des-build/src/main/resources/des-config or
    • Path 2: des-docker/src/main/resources/des-config
  7. Run the command below to build DES images.
    des-tool.bat build 
    des-docker.bat build
  8. Tag the built images, for example, the release version must be the release of DES (that is, 202101.0.0).
    9. docker tag mybank/des-config-installer <<ACRName>>/des-config-installer:<<releaseVersion>> 
    docker tag mybank/des-event-cleanup <<ACRName>>/des-event-cleanup:<<releaseVersion>> 
    docker tag mybank/des-event-processor <<ACRName>>/des-event-processor:<<releaseVersion>>
    docker tag mybank/des-event-router <<ACRName>>/des-event-router:<<releaseVersion>>

    docker tag mybank/des-event-pull-adapter <<ACRName>>/des-event-pull-adapter:<<releaseVersion>> 
    docker tag mybank/des-api <<ACRName>>/des-api:<<releaseVersion>> (Only if API is built)

    docker tag mybank/des-tool <<ACRName>>/des-tool:<<releaseVersion>>
		
    For example, ACRNAME - ctsdev.azurecr.io releaseVersion - 202108.0.0
    docker tag mybank/des-config-installer ctsdev.azurecr.io/des-config-installer:202108.0.0
  9. Login to ACR using the command below. 
    az login- az acr login -n <<ACRName>>
  10. Push the images to the required ACR using the command below.

    11. docker push <<ACRName>>/des-config-installer:<<releaseVersion>>
    docker push <<ACRName>>/des-event-cleanup:<<releaseVersion>> 
    docker push <<ACRName>>/des-event-processor:<<releaseVersion>> 
    docker push <<ACRName>>/des-event-router:<<releaseVersion>> 
    docker push <<ACRName>>/des-pull-adapter:<<releaseVersion>> 
    docker push <<ACRName>>/des-api:<<releaseVersion>>
    docker push <<ACRName>>/des_tool:<<releaseVersion>>
    For example, ACRNAME - ctsdev.azurecr.io releaseVersion - 202108.0.0
    docker push ctsdev.azurecr.io/des-config-installer:202108.0.0

  11. Pull and push the third-party images like Kafka, Zookeeper, Schema registry, and so on to install the scripts.

  • Pull
    docker pull confluentinc/cp-schema-registry:5.2.2 docker pull confluentinc/cp-kafka:5.2.2
    docker pull confluentinc/cp-zookeeper:5.2.2
  • Tag
    docker tag confluentinc/cp-schema-registry <<ACRNAME>>/confluentinc/cp-schema-registry:5.2.2 docker tag confluentinc/cp-kafka <<ACRNAME>>/confluentinc/cp-kafka:5.2.2
    docker tag confluentinc/cp-zookeeper <<ACRNAME>>/confluentinc/cp-zookeeper:5.2.2
  • Push
    docker push <<ACRNAME>>/confluentinc/cp-schema-registry:5.2.2 docker push <<ACRNAME>>/confluentinc/cp-kafka:5.2.2
    docker push <<ACRNAME>>/confluentinc/cp-zookeeper:5.2.2

All the images are pushed to ACR successfully.

  1. Install DB scripts and DES configuration by running the command below. The environment running this command should have access to the Transact and streaming databases.

    des-tool.bat/sh install

    2. Alternatively, you can run DES installation through a pod called des-tool. It can be used when starting DES using helm.

Setting up Third-Party Services

Perform the following steps to install third-party services after the availability of Azure resources.

If Kafka, Zookeeper, and Schema registry is already available as a part of the installation, then skip the step 4 and 5.
  1. Create Event Hub in Azure resources manually using the portal or Azure CLI command below. Attached is a sample des_eventhub.json for reference. For all the required topics to be created, read the release notes section 4.3 in the DES release package.

    2. az group deployment create -g <<Resource group>> --template-file <<path to des_eventhub.json>>
    az group deployment create -g FRONT2BACK2-aks --template-file des_eventhub.json

  2. Execute the commands below to get the values of environment variables related to the event hub for creating a connection between DES and Event Hub.
    • Event Hub Shared Access Key
      • az eventhubs namespace authorization-rule keys list -g $(shared.aksResourceGroup) --namespace-name
      $(eventhubNamespace) --name RootManageSharedAccessKey --query primaryKey -o tsv
      az eventhubs namespace authorization-rule keys list -g FRONT2BACK-aks --namespace-name front2backdeshubHA --name RootManageSharedAccessKey --query primaryKey -o tsv export EVENT_HUB_SHARED_ACCESS_KEY=<<value obtained from above execution>>
    • Event Hub Connection URL
      • az eventhubs namespace authorization-rule keys list -g $(shared.aksResourceGroup) --namespace-name
      $(eventhubNamespace) -n RootManageSharedAccessKey --query "primaryConnectionString" -o tsv
      az eventhubs namespace authorization-rule keys list -g FRONT2BACK-aks --namespace-name front2backdeshubHA -n RootManageSharedAccessKey --query "primaryConnectionString" -o tsv export EH_CONNECTION_URL=<<value obtained from above execution>>
    • Event Hub CI shared access key
      • az eventhubs namespace authorization-rule keys list -g $(shared.aksResourceGroup) --namespace-name
      $(eventhubNamespaceCI) -n RootManageSharedAccessKey --query primaryKey -o tsv
      az eventhubs namespace authorization-rule keys list -g FRONT2BACK-aks --namespace-name front2backdeshubCIHA -n RootManageSharedAccessKey --query primaryKey -o tsv export EVENT_HUB_CI_SHARED_ACCESS_KEY=<<value obtained from above execution>>
    • Event Hub CI connection URL
      • az eventhubs namespace authorization-rule keys list -g $(shared.aksResourceGroup) --namespace-name
$(eventhubNamespaceCI) -n RootManageSharedAccessKey --query "primaryConnectionString" -o tsv
      az eventhubs namespace authorization-rule keys list -g FRONT2BACK-aks --namespace-name front2backdeshubCIHA -n RootManageSharedAccessKey --query "primaryConnectionString" -o tsv export EVENT_HUB_CI_CONNECTION_URL=<<value obtained from above execution>>
  3. Go to SQL-managed instances and get the database host details, database name, username, and password for Transact and DES databases. The same details are used to construct the database-related properties that can be overridden using setters (--set) when installing DES using helm.
  4. If third-party helm charts are not available in the des-docker/src/helm/3rdParty folder, then execute the command below to start the third-party services, such as Kafka and Schema registry.
    az acr helm repo add -n $(crName)
    helm repo add confluentinc https://confluentinc.github.io/cp-helm-charts/ helm repo update
    helm upgrade kafka-oss confluentinc/cp-helm-charts --install --version 0.5.0 -n kafka
  5. The charts available in the des-docker/src/helm/3rdParty folder are a copy of confluent helm charts and uses the community edition of confluent images. If confluent images are pushed to a local registry, run the command below.
    helm install kafka-oss 3rdParty/cp-helm-charts --version 0.5.0 -n kafka \
    --set cp-zookeeper.image=<<ACRNAME>>/cp-zookeeper \
    --set cp-zookeeper.imageTag=5.2.2 \
    --set cp-kafka.image=<<ACRNAME>>/cp-kafka \
    --set cp-kafka.imageTag=5.2.2 \
    --set cp-schema-registry.image=<<ACRNAME>>/cp-schema-registry \
    --set cp-schema-registry.imageTag=5.2.2 \
    --create-namespace
    ACRNAME is a container registry where images are available. For example, ctsdev.azurecr.io.

The third-party services are installed successfully.

Copyright © 2020- Temenos Headquarters SA

Contact Us

knowledge@temenos.com

Published on :
Thursday, May 30, 2024 11:58:52 AM IST