External Validation Webhooks

Overview

External validation webhooks allow you to run IaC validation on your own infrastructure (GitHub Actions, GitLab CI, Jenkins, etc.) instead of using Cloudgeni’s internal runners.

Choose Integration Mode

Three integration modes:

GitHub Actions: Cloudgeni triggers your workflow_dispatch directly. Provide workflow filename. No incoming webhook to handle.
GitLab CI: Cloudgeni triggers your GitLab pipeline with variables. Add a job to .gitlab-ci.yml that runs when CLOUDGENI_UOW_ID is set.
Custom Webhook: Provide your webhook endpoint URL. Cloudgeni sends signed POST requests. You verify signatures, start asynchronous validation and return 202 Accepted.

Step 1: Configure in Cloudgeni

Go to: Settings → IaC Repositories → Select integration → Select repository → Configure Webhook
Choose mode:
- GitHub Actions: Enter workflow filename (e.g., terraform-validation.yml)
- GitLab Pipeline: No additional config needed - uses .gitlab-ci.yml
- Custom Webhook: Enter webhook endpoint URL
Copy CLOUDGENI_WEBHOOK_SECRET
Store securely (GitHub Secrets, GitLab CI/CD Variables, environment variables, etc.)

Step 2: Implementation

GitHub Actions
GitLab CI
Custom Webhook

Cloudgeni triggers your workflow via workflow_dispatch with uowId and callbackUrl inputs. The uowId is the unique identifier for this validation “unit of work” (UOW).

Workflow Requirements

Run IaC steps: init, validate, and plan (or equivalent for your IaC tool, we currently support Terraform and Bicep)
Required: The plan step (or what-if for Bicep) is mandatory when executionFailure=false.
Capture outputs: stdout for init/validate, file for plan JSON
Sign callback payload with CLOUDGENI_WEBHOOK_SECRET
POST results to callbackUrl

Example Workflow

This workflow is a template. You must customize the authentication, environment matrix, and Terraform commands (e.g. init args, variable files) to match your specific infrastructure setup.

# .github/workflows/terraform-validation.yml
name: Cloudgeni External Validation

on:
  workflow_dispatch:
    inputs:
      uowId:
        description: 'UOW execution ID'
        required: true
        type: string
      callbackUrl:
        description: 'Cloudgeni callback URL'
        required: true
        type: string
      branch:
        description: 'Branch to validate'
        required: false
        default: 'main'
        type: string

jobs:
  # ============================================================
  # Job 1: Run Terraform validation for each environment
  # ============================================================
  validate-environment:
    runs-on: ubuntu-latest
    continue-on-error: true  # Don't fail the workflow if validation fails
    strategy:
      fail-fast: false
      matrix:
        # CUSTOMIZE: Replace with your environment names
        env: [staging, prod]

    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ inputs.branch }}

      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: latest
          terraform_wrapper: false

      # CUSTOMIZE: Replace with your cloud provider authentication
      # Example for GCP:
      - name: Setup Cloud Credentials
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}

      - name: Run Terraform (init/validate/plan)
        id: terraform
        shell: bash
        working-directory: environments/${{ matrix.env }}  # CUSTOMIZE: Your path
        continue-on-error: true
        run: |
          set -o pipefail
          
          # Run each command and capture exit codes separately
          terraform init -no-color 2>&1 | tee "$RUNNER_TEMP/init.txt"
          echo "${PIPESTATUS[0]}" > "$RUNNER_TEMP/init.exit"
          
          terraform validate -no-color 2>&1 | tee "$RUNNER_TEMP/validate.txt"
          echo "${PIPESTATUS[0]}" > "$RUNNER_TEMP/validate.exit"
          
          terraform plan -out=tfplan.binary -no-color 2>&1 | tee "$RUNNER_TEMP/plan.txt"
          echo "${PIPESTATUS[0]}" > "$RUNNER_TEMP/plan.exit"
          
          # Generate plan JSON if plan succeeded
          if [ "$(cat $RUNNER_TEMP/plan.exit)" = "0" ]; then
            terraform show -json tfplan.binary > "$RUNNER_TEMP/plan.json" 2>&1 || echo '{}' > "$RUNNER_TEMP/plan.json"
          else
            echo '{}' > "$RUNNER_TEMP/plan.json"
          fi
          
          echo "${{ matrix.env }}" > "$RUNNER_TEMP/env.name"

      - uses: actions/upload-artifact@v4
        with:
          name: results-${{ matrix.env }}
          path: |
            ${{ runner.temp }}/env.name
            ${{ runner.temp }}/init.txt
            ${{ runner.temp }}/init.exit
            ${{ runner.temp }}/validate.txt
            ${{ runner.temp }}/validate.exit
            ${{ runner.temp }}/plan.txt
            ${{ runner.temp }}/plan.exit
            ${{ runner.temp }}/plan.json

  # ============================================================
  # Job 2: Aggregate results and send callback to Cloudgeni
  # ============================================================
  send-callback:
    runs-on: ubuntu-latest
    needs: [validate-environment]
    if: always()  # Always run, even if validation job failed
    steps:
      - uses: actions/download-artifact@v4
        continue-on-error: true
        with:
          path: ${{ runner.temp }}/results

      - name: Send callback to Cloudgeni
        env:
          CALLBACK_URL: ${{ inputs.callbackUrl }}
          WEBHOOK_SECRET: ${{ secrets.CLOUDGENI_WEBHOOK_SECRET }}
        shell: bash
        run: |
          python3 - <<'PYTHON_CALLBACK'
          import os
          import sys
          import json
          import hmac
          import hashlib
          import requests
          
          def send_callback(payload, callback_url, webhook_secret):
              """Send signed callback to Cloudgeni."""
              payload_json = json.dumps(payload, separators=(',', ':'))
              signature = hmac.new(
                  webhook_secret.encode('utf-8'),
                  payload_json.encode('utf-8'),
                  hashlib.sha256
              ).hexdigest()
              
              headers = {
                  'Content-Type': 'application/json',
                  'X-Cloudgeni-Signature-256': f'sha256={signature}'
              }
              
              response = requests.post(callback_url, data=payload_json, headers=headers, timeout=30)
              print(f'Callback status: {response.status_code}')
              return response
          
          def read_file(path, default=''):
              """Safely read a file, returning default if it fails."""
              try:
                  with open(path, 'r', encoding='utf-8') as f:
                      return f.read()
              except Exception:
                  return default
          
          # Get environment variables
          callback_url = os.environ.get('CALLBACK_URL', '')
          webhook_secret = os.environ.get('WEBHOOK_SECRET', '')
          results_dir = os.path.join(os.environ.get('RUNNER_TEMP', '/tmp'), 'results')
          
          # Check required variables
          if not callback_url or not webhook_secret:
              print('ERROR: Missing CALLBACK_URL or WEBHOOK_SECRET')
              sys.exit(1)
          
          try:
              environments = []
              
              # Process each environment's results
              if os.path.isdir(results_dir):
                  for env_dir in sorted(os.listdir(results_dir)):
                      env_path = os.path.join(results_dir, env_dir)
                      if not os.path.isdir(env_path):
                          continue
                      
                      # Read results for this environment
                      env_name = read_file(os.path.join(env_path, 'env.name')).strip() or env_dir.replace('results-', '')
                      
                      init_out = read_file(os.path.join(env_path, 'init.txt'))
                      init_exit = read_file(os.path.join(env_path, 'init.exit'), '1').strip()
                      init_ok = init_exit == '0'
                      
                      val_out = read_file(os.path.join(env_path, 'validate.txt'))
                      val_exit = read_file(os.path.join(env_path, 'validate.exit'), '1').strip()
                      val_ok = val_exit == '0'
                      
                      plan_txt = read_file(os.path.join(env_path, 'plan.txt'))
                      plan_exit = read_file(os.path.join(env_path, 'plan.exit'), '1').strip()
                      plan_ok = plan_exit == '0'
                      
                      # Process plan JSON
                      plan_json = '{}'
                      if plan_ok:
                          raw_json = read_file(os.path.join(env_path, 'plan.json'), '{}')
                          try:
                              parsed = json.loads(raw_json)
                              plan_json = json.dumps(parsed, separators=(',', ':'))
                          except json.JSONDecodeError:
                              plan_ok = False
                              plan_txt = plan_txt or 'Invalid plan JSON generated'
                      
                      # Build steps array
                      steps = [
                          {
                              'step': 'init',
                              'success': init_ok,
                              'error': None if init_ok else (init_out or 'init failed'),
                              'data': init_out if init_ok else None
                          },
                          {
                              'step': 'validate',
                              'success': val_ok,
                              'error': None if val_ok else (val_out or 'validate failed'),
                              'data': val_out if val_ok else None
                          },
                          {
                              'step': 'plan',
                              'success': plan_ok,
                              'error': None if plan_ok else (plan_txt or 'plan failed'),
                              'data': plan_json if plan_ok else None
                          }
                      ]
                      
                      environments.append({
                          'environmentName': env_name,
                          'iacType': 'TERRAFORM',
                          'steps': steps
                      })
              
              # Build payload
              if environments:
                  payload = {
                      'executionFailure': False,
                      'executionFailureMessage': None,
                      'environments': environments
                  }
              else:
                  payload = {
                      'executionFailure': True,
                      'executionFailureMessage': 'No environment results found',
                      'environments': None
                  }
              
              # Send callback
              response = send_callback(payload, callback_url, webhook_secret)
              response.raise_for_status()
              print('Callback sent successfully!')
              
          except Exception as e:
              # Always try to send a failure callback
              print(f'ERROR: {e}')
              try:
                  failure_payload = {
                      'executionFailure': True,
                      'executionFailureMessage': f'Workflow error: {str(e)}',
                      'environments': None
                  }
                  send_callback(failure_payload, callback_url, webhook_secret)
                  print('Sent failure callback')
              except Exception as e2:
                  print(f'Failed to send failure callback: {e2}')
              sys.exit(1)
          PYTHON_CALLBACK

Step 3: Send Callback Response

Terraform
Bicep

After validation completes, send signed results to Cloudgeni.Endpoint: POST /api/v1/callback/uow/{uow_id} (use callbackUrl from webhook)Headers:

Content-Type: application/json
X-Cloudgeni-Signature-256: sha256=<hex>

Payload (Terraform):

{
  "executionFailure": false,
  "executionFailureMessage": null,
  "environments": [
    {
      "environmentName": "main",
      "iacType": "TERRAFORM",
      "steps": [
        {
          "step": "init",
          "success": true,
          "error": null,
          "data": "Terraform initialized successfully"
        },
        {
          "step": "validate",
          "success": true,
          "error": null,
          "data": "Success! The configuration is valid."
        },
        {
          "step": "plan",
          "success": true,
          "error": null,
          "data": "{\"format_version\":\"1.2\",\"terraform_version\":\"1.5.0\",\"resource_changes\":[]}"
        }
        // Add any other steps you want to report. If any of these steps fail, the Cloudgeni agent will use that information to iterate on the IaC code during a remediation.
      ]
    }
  ]
}

Example command to generate plan JSON:

terraform plan -out=tfplan.binary && terraform show -json tfplan.binary > plan.json

Each environment MUST include a step called “plan”. This step is mandatory and must contain:

Step name: “plan”
Success status: true/false
JSON data: The data field must be a JSON string (not an object) containing the Terraform plan output in JSON format

Example: { "step": "plan", "success": true, "error": null, "data": "{...plan json...}" }

executionFailure: Set to true only for complete execution failures (cannot run any validation).
When executionFailure=true: executionFailureMessage is required, environments must be null or empty.
When executionFailure=false: environments is required, executionFailureMessage must be null.
steps[].data for plan must be a JSON string, not an object.
environmentName should match your environment (e.g., “main”, “environments/production”).
iacType must be “TERRAFORM”.

Full schema: https://ai.cloudgeni.ai/docs → “External Validation Callback”

Signing the Callback

import hmac
import hashlib
import json
import requests

def sign_callback(payload_dict, secret):
    payload_json = json.dumps(payload_dict, separators=(',', ':'))
    signature_hex = hmac.new(
        secret.encode('utf-8'),
        payload_json.encode('utf-8'),
        hashlib.sha256
    ).hexdigest()
    return f"sha256={signature_hex}", payload_json

def send_callback(callback_url, payload, secret):
    signature, payload_json = sign_callback(payload, secret)
    response = requests.post(
        callback_url,
        data=payload_json,
        headers={
            "Content-Type": "application/json",
            "X-Cloudgeni-Signature-256": signature
        },
        timeout=30
    )
    response.raise_for_status()
    return response

Security Best Practices

1. Protect Your Webhook Secret

Never commit secrets to version control
Use environment variables or secret managers
Rotate secrets periodically through the cloudgeni console

2. Always Verify Signatures

Reject requests without signatures
Verify before parsing JSON
Use timing-safe comparison functions
Log failed verification attempts

3. Use HTTPS

Always use HTTPS for webhook endpoints
Validate TLS certificates
Consider IP allowlisting if possible

4. Implement Timeouts

Set reasonable timeouts for callbacks (30-60 seconds)
Handle network failures gracefully
Implement retry logic with exponential backoff

5. Validate Payload Schema

Verify all required fields are present
Validate data types and formats
Sanitize inputs before processing

Getting Started

Setup & Configuration

Features

Support

Overview

Choose Integration Mode

Step 1: Configure in Cloudgeni

Step 2: Implementation

Workflow Requirements

Example Workflow

Step 3: Send Callback Response

Signing the Callback

Security Best Practices

Getting Started

Setup & Configuration

Features

Support

​Overview

​Choose Integration Mode

​Step 1: Configure in Cloudgeni

​Step 2: Implementation

​Workflow Requirements

​Example Workflow

​Step 3: Send Callback Response

​Signing the Callback

​Security Best Practices

Overview

Choose Integration Mode

Step 1: Configure in Cloudgeni

Step 2: Implementation

Workflow Requirements

Example Workflow

Step 3: Send Callback Response

Signing the Callback

Security Best Practices