Building A Deterministic Multi Cloud Webhook Router With Cloud Run And Aws Lambda

The problem I ran into

I built a hybrid webhook pipeline that had to accept payment events from two different SaaS vendors, then forward them to internal services that were deployed across AWS Lambda and Google Cloud Run (hybrid + multi-cloud).

What surprised me: the same webhook sometimes got delivered multiple times (payment systems often retry), and the vendor’s event IDs were only unique per vendor, not across vendors. So I couldn’t safely use a single “event id” as-is for deduplication.

Even worse, I needed deterministic routing: given an event payload, the router had to pick the same target service every time—because I was doing cost accounting (FinOps) per routed destination and wanted consistent bills.

So I ended up building a tiny “deterministic webhook router” that:

• generates a stable idempotency key from the payload (cross-vendor safe),
• deduplicates requests using Redis,
• routes to either Cloud Run or Lambda based on a hash of normalized fields,
• forwards the request with the exact same payload bytes.

This post is the exact approach I implemented.

---

Architecture (the parts that matter)

• Cloud Run: hosts the webhook entrypoint. It receives the vendor webhook.
• Redis: stores idempotency keys to prevent duplicate processing.
• AWS Lambda: receives routed events.
• Cloud Run service: receives routed events destined for GCP.
• Routing rule: a stable hash of normalized fields chooses destination.

Key concept: idempotency means “if the same logical request arrives multiple times, the system processes it once.”

---

Data model for routing and dedup

I used two vendor-specific fields:

• vendor (string: "stripe" or "adyen" in my case)
• event_type (string)
• created_at (ISO timestamp)
• payload (the raw JSON)

To make dedup safe across vendors, I build an idempotency key from:

• normalized vendor
• normalized event_type
• normalized created_at (parsed and re-serialized so formatting differences don’t matter)
• a hash of the payload content

---

The Cloud Run webhook router (Python)

Below is the working Cloud Run entrypoint using Flask and redis-py.

Install dependencies

flask==3.0.3
redis==5.0.8
requests==2.32.3
boto3==1.34.162
python-dateutil==2.9.0.post0
gunicorn==22.0.0

Router code

# main.py
import os
import json
import hashlib
import hmac
import base64
from datetime import datetime, timezone

import boto3
import redis
import requests
from flask import Flask, request, Response
from dateutil import parser as date_parser

app = Flask(__name__)

# Environment variables
REDIS_URL = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
AWS_REGION = os.environ.get("AWS_REGION", "us-east-1")
AWS_LAMBDA_FUNCTION_NAME = os.environ.get("AWS_LAMBDA_FUNCTION_NAME", "webhook-processor")
AWS_LAMBDA_ROLE_ARN = os.environ.get("AWS_LAMBDA_ROLE_ARN", "")  # optional if needed

# The two destinations I used
GCP_TARGET_URL = os.environ.get("GCP_TARGET_URL", "https://example-a.run.app/ingest")

# A secret used for deterministic hashing (prevents leaking payload structure through routing hash)
ROUTING_SECRET = os.environ.get("ROUTING_SECRET", "dev-not-for-prod-change-me")

# How long to keep dedup keys (seconds)
DEDUP_TTL_SECONDS = int(os.environ.get("DEDUP_TTL_SECONDS", "86400"))

rds = redis.Redis.from_url(REDIS_URL, decode_responses=False)

lambda_client = boto3.client("lambda", region_name=AWS_REGION)

def normalize_created_at(created_at_value: str) -> str:
    """
    Parse a timestamp string and re-serialize it as an RFC3339-like UTC string.
    This prevents formatting differences from producing different idempotency keys.
    """
    dt = date_parser.parse(created_at_value)
    if dt.tzinfo is None:
        dt = dt.replace(tzinfo=timezone.utc)
    dt_utc = dt.astimezone(timezone.utc)
    # Stable format
    return dt_utc.isoformat().replace("+00:00", "Z")

def stable_payload_hash(payload_obj: dict) -> str:
    """
    Hash the payload content deterministically:
    - JSON stringify with sorted keys
    - stable UTF-8 encoding
    """
    normalized = json.dumps(payload_obj, sort_keys=True, separators=(",", ":")).encode("utf-8")
    return hashlib.sha256(normalized).hexdigest()

def idempotency_key(vendor: str, event_type: str, created_at: str, payload_obj: dict) -> str:
    payload_digest = stable_payload_hash(payload_obj)
    normalized_created_at = normalize_created_at(created_at)

    # Cross-vendor safe logical key
    raw = f"{vendor.lower()}|{event_type}|{normalized_created_at}|{payload_digest}"

    # HMAC keeps the key from becoming a plaintext representation in Redis
    digest = hmac.new(ROUTING_SECRET.encode("utf-8"), raw.encode("utf-8"), hashlib.sha256).hexdigest()
    return f"webhook:dedup:{digest}"

def route_destination(vendor: str, event_type: str, created_at: str, payload_obj: dict) -> str:
    """
    Deterministically choose a destination using an HMAC of normalized fields.
    Returns either 'gcp' or 'aws'.
    """
    normalized_created_at = normalize_created_at(created_at)
    raw = f"route|{vendor.lower()}|{event_type}|{normalized_created_at}|{stable_payload_hash(payload_obj)}"
    digest = hmac.new(ROUTING_SECRET.encode("utf-8"), raw.encode("utf-8"), hashlib.sha256).digest()

    # First byte decides destination. 0-127 => gcp, 128-255 => aws
    return "gcp" if digest[0] < 128 else "aws"

@app.post("/webhook")
def webhook() -> Response:
    # Read raw body bytes so we can forward the exact same bytes downstream.
    raw_body = request.get_data(cache=False)
    try:
        payload = json.loads(raw_body.decode("utf-8"))
    except Exception:
        return Response("invalid json", status=400)

    # Expect vendor schema
    vendor = payload.get("vendor")
    event_type = payload.get("event_type")
    created_at = payload.get("created_at")

    if not vendor or not event_type or not created_at:
        return Response("missing required fields", status=400)

    # Use full payload object for dedup; stable hash makes it content-based.
    idem_key = idempotency_key(vendor, event_type, created_at, payload)
    destination = route_destination(vendor, event_type, created_at, payload)

    # Redis SET with NX ensures only the first request "wins".
    # It returns:
    # - True if key was set
    # - False if key already existed
    already = rds.set(idem_key, b"1", nx=True, ex=DEDUP_TTL_SECONDS)
    if not already:
        # Duplicate logical event: short-circuit.
        return Response("duplicate ignored", status=200)

    # Forward downstream
    if destination == "gcp":
        resp = requests.post(
            GCP_TARGET_URL,
            data=raw_body,
            headers={"Content-Type": request.headers.get("Content-Type", "application/json")},
            timeout=10,
        )
        return Response(resp.text, status=resp.status_code)
    else:
        # Invoke AWS Lambda with the raw payload bytes.
        # Payload must be JSON-serializable for Lambda Invoke.
        # We send the original parsed payload to preserve structure.
        lambda_payload = payload
        invoke_kwargs = {
            "FunctionName": AWS_LAMBDA_FUNCTION_NAME,
            "InvocationType": "RequestResponse",
            "Payload": json.dumps(lambda_payload).encode("utf-8"),
        }

        if AWS_LAMBDA_ROLE_ARN:
            invoke_kwargs["ClientContext"] = AWS_LAMBDA_ROLE_ARN

        resp = lambda_client.invoke(**invoke_kwargs)

        # Lambda returns a streaming payload in resp["Payload"]
        lambda_body = resp["Payload"].read().decode("utf-8")
        return Response(lambda_body, status=200)

if __name__ == "__main__":
    # For local dev only; Cloud Run uses Gunicorn typically
    app.run(host="0.0.0.0", port=8080)

What’s happening (line-by-line in the critical parts)

• I read raw request bytes using request.get_data(...).
  • That matters because I forward the exact same bytes to Cloud Run or Lambda.
• I parse JSON and extract vendor, event_type, created_at.
• I compute an idempotency key:
  • normalize timestamp to UTC stable format,
  • hash normalized payload with sorted keys,
  • HMAC it with ROUTING_SECRET so Redis doesn’t store readable content.
• I call redis.set(key, ..., nx=True, ex=TTL):
  • nx=True means “set only if not exists”.
  • This is the dedup guarantee. Only the first duplicate gets processed.
• I compute destination using deterministic HMAC and pick by a byte threshold.
• I forward:
  • to Cloud Run via HTTP POST,
  • to Lambda via lambda_client.invoke.

---

Local test harness

To test determinism and dedup without deploying, I used redis locally.

Run Redis

docker run --rm -p 6379:6379 redis:7

Run the router locally

export REDIS_URL=redis://localhost:6379/0
export AWS_REGION=us-east-1
export AWS_LAMBDA_FUNCTION_NAME=webhook-processor
export GCP_TARGET_URL=http://localhost:8081/ingest
export ROUTING_SECRET=local-secret
export DEDUP_TTL_SECONDS=86400

pip install -r requirements.txt
python main.py

Create a mock payload

cat > payload.json <<'JSON'
{
  "vendor": "stripe",
  "event_type": "payment.succeeded",
  "created_at": "2026-05-06T12:34:56Z",
  "amount": 4200,
  "currency": "USD",
  "details": { "invoice": "inv_123" }
}
JSON

Call twice with the same payload

curl -s -X POST http://localhost:8080/webhook \
  -H 'Content-Type: application/json' \
  --data @payload.json

curl -s -X POST http://localhost:8080/webhook \
  -H 'Content-Type: application/json' \
  --data @payload.json

Expected behavior:

• First call routes and forwards.
• Second call returns duplicate ignored.

Because the idempotency key is derived from content, even if the vendor retries later, it still dedups.

---

Deploying to Cloud Run (minimal)

Dockerfile

# Dockerfile
FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py .

# Cloud Run expects port 8080
ENV PORT=8080

CMD ["gunicorn", "-b", ":8080", "main:app"]

Build and deploy

gcloud run deploy webhook-router \
  --source . \
  --region us-central1 \
  --platform managed \
  --allow-unauthenticated \
  --set-env-vars "REDIS_URL=redis://10.0.0.5:6379/0,AWS_REGION=us-east-1,AWS_LAMBDA_FUNCTION_NAME=webhook-processor,GCP_TARGET_URL=https://example-a.run.app/ingest,ROUTING_SECRET=change-me,DEDUP_TTL_SECONDS=86400"

---

Why this approach worked for FinOps and platform engineering

In practice, I hit two cost-related problems:

1. Duplicate deliveries were inflating invocation counts.
2. Non-deterministic routing made dashboards noisy (events “sometimes” went to AWS, “sometimes” to GCP).

This router fixed both:

• Dedup via Redis made “logical once” processing align with billing events.
• Deterministic routing ensured the same event type/date/vendor combination landed on the same destination consistently, which stabilized cost attribution.

---

Conclusion

I built a deterministic hybrid webhook router where I compute a cross-vendor idempotency key (content + normalized timestamp + HMAC) and use Redis SET NX to process each logical event once, then route to Cloud Run or AWS Lambda using the same normalized fields every time. The result was a webhook system that behaved predictably under retries and produced stable routing for cleaner FinOps and platform engineering metrics.