CI/CD Integration

This page covers automating the full pipeline from plugin development to production deployment. The workflow has four stages: export the source artifacts to version control, build the bundle from them, host the bundle at a stable URL, and deploy it via Helm.

For the bundle format and installer behavior, see Custom Bundles & Extensibility.

The pipeline

The conncentric/sdk Docker image is the official packaging engine for stage 2. It compiles plugin sources, runs their tests, and zips the definitions/, plugins/, and artifacts/ directories into a single bundle.zip ready for hosting.

Stage 1: Export and push to version control

Operators export adapter definitions from the Portal and developers commit plugin source. Both land in the same repository so a single CI pipeline can build a coherent bundle.

Source	What it is	Who owns it
definitions/	JSON adapter definitions exported from the Portal, with concrete values. Environment-specific values are templated by your pipeline before packaging (for example with envsubst); credentials are ${env:VAR_NAME} runtime references.	Operators / configuration owners.
plugins/ (source)	Custom plugin source projects authored against the SDK.	Plugin developers.
artifacts/	Reference files such as data dictionaries and JSON schemas.	Whoever curates the protocol artifacts.

Push every change to the repository's main branch (or your equivalent integration branch) to trigger the pipeline. See Plugin SDK Getting Started for the recommended repository structure.

Stage 2: Build with the SDK image

The CI pipeline runs the conncentric/sdk image with the repository mounted at /workspace. The bundle command compiles all plugin projects, runs their tests, and produces bundle.zip in the working directory.

- name: Build and package bundle
  run: |
    docker run --user "$(id -u):$(id -g)" \
      -v $(pwd):/workspace \
      ${{ vars.CONNCENTRIC_REGISTRY }}/conncentric/sdk:${{ vars.CONNCENTRIC_VERSION }} \
      bundle

CONNCENTRIC_REGISTRY and CONNCENTRIC_VERSION are set in your CI environment configuration. The image tag must match your deployed platform version so the SDK and the running orchestrator agree on the manifest schema. The --user flag matches the runner's UID and GID so the SDK image can write to the bind-mounted workspace; without it, Gradle fails to create build/ directories on Linux runners.

Stage 3: Upload to hosting

Upload bundle.zip to any URL reachable from your Kubernetes cluster. Rename it with a stable version identifier so restoring a previous version is a one-line URL change.

Cloud object storage

- name: Upload bundle
  run: |
    aws s3 cp bundle.zip \
      s3://${{ vars.BUNDLE_BUCKET }}/custom-bundle-${{ github.sha }}.zip

Substitute the equivalent CLI for GCS (gsutil cp) or Azure Blob (az storage blob upload).

Internal HTTP server

For air-gapped or on-premises deployments, upload to any HTTP server your cluster can reach. No cloud-specific tooling required.

Access control

The Installer job downloads bundles from inside the cluster. Grant it access using your cloud provider's workload identity mechanism.

Cloud	Mechanism	Service account annotation
AWS (EKS)	IRSA	eks.amazonaws.com/role-arn
GCP (GKE)	Workload Identity	iam.gke.io/gcp-service-account
Azure (AKS)	Pod Identity	azure.workload.identity/client-id

Create an IAM policy granting GetObject (or equivalent read access) on your bundle bucket, bind it to a role, and annotate the Installer service account.

kubectl annotate serviceaccount conncentric-installer \
  -n conncentric \
  <your-cloud-annotation>=<your-role-arn-or-service-account>

If workload identity is not available, use pre-signed URLs instead.

aws s3 presign s3://${{ vars.BUNDLE_BUCKET }}/custom-bundle-${{ github.sha }}.zip --expires-in 604800

Use the same object key here that Stage 3 uploaded (custom-bundle-${{ github.sha }}.zip). The commit SHA is an immutable identifier, so each pipeline run produces a distinct, restorable bundle URL.

Stage 4: Deploy via Helm

Update Helm values with the new bundle URL and run helm upgrade. The Two-Pass Installer downloads the bundle and applies it to the cluster.

Construct the bundle URL from the object you uploaded and export it so the deploy step can resolve it. This example presigns the object using the same key Stage 3 uploaded, which lets the in-cluster installer fetch it over HTTPS without embedding credentials. If your bundle is reachable at a stable HTTPS URL that the installer can read via workload identity, set BUNDLE_URL to that URL instead.

- name: Resolve bundle URL
  run: |
    BUNDLE_URL=$(aws s3 presign \
      s3://${{ vars.BUNDLE_BUCKET }}/custom-bundle-${{ github.sha }}.zip \
      --expires-in 604800)
    echo "BUNDLE_URL=$BUNDLE_URL" >> "$GITHUB_ENV"

- name: Deploy
  run: |
    helm upgrade conncentric oci://<your-registry>/conncentric/charts/conncentric --version <release-tag> \
      -n conncentric \
      -f values-production.yaml \
      --set installer.customBundleUrls[0]="${{ env.BUNDLE_URL }}"

Monitor the installer job from the cluster. The installer runs as a Helm post-upgrade hook, and its Job is named conncentric-installer-<revision>, where <revision> is the Helm release revision number. List the jobs to find the current one, then follow its logs.

kubectl get jobs -n conncentric
kubectl logs -f job/conncentric-installer-<revision> -n conncentric

Managing multiple environments

Use the same bundle across environments. Only the Helm values change.

What changes per environment	Where it lives
Infrastructure endpoints (database host, Kafka brokers, FIX venues)	Concrete values in each environment's definitions, produced by your pipeline before packaging.
Bundle URL	installer.customBundleUrls in the environment-specific values-<env>.yaml.
Plugin selection	installer.plugins in the environment-specific values-<env>.yaml.

Definitions carry concrete values per environment; ${env:VAR_NAME} credential references resolve at runtime inside the adapter, and any other unresolved ${VAR} token is rejected at import.

Versioning and configuration restore

Action	How
Deploy a new version	Build a new bundle, upload it with a new version identifier, update the bundle URL in Helm values, run helm upgrade.
Restore a previous config	Point the bundle URL back at the previous version, run helm upgrade. The installer re-applies that bundle.
Pin a version	Use explicit version identifiers in bundle filenames (for example, custom-bundle-v1.2.0.zip) rather than latest or mutable URLs.

The installer is idempotent. Restoring re-applies the older bundle: unchanged items are skipped, changed items are updated.

CI credentials

Credential	Purpose	Storage
Container registry access	Pull the SDK image.	CI environment (the same registry used for deployment).
Kubernetes credentials	Run helm upgrade against the target cluster.	CI secrets manager.
Cloud credentials	Upload bundle to object storage.	CI secrets manager (or use CI-native OIDC federation).

Never hardcode credentials in scripts or commit them to version control.

See also

• Custom Bundles & Extensibility for the bundle format, installer behavior, and variable substitution reference.
• Plugin SDK Getting Started for the SDK image, recommended repo structure, and building custom plugins.