Authenticating...
Skip to main content

0068: Composite Events — Deriving Goal Triggers from Raw Event Streams

STATUS

Accepted

Supersedes the conditional goals approach in PR #143.

CONTEXT

Business Problem

Campaigns increasingly require conditional reward gating. The canonical example is a "Turbo" campaign: a player must reach Level 10 AND make 2 in-app purchases before earning the premium reward. Today this logic is handled by a batch process running against Tune, which is an ad-hoc composite-event engine with logic scattered across one-off scripts and Tune configuration, owned nowhere coherent. This batch-plus-Tune arrangement caused Incident #248 and represents a class of reliability risk that grows with each new conditional campaign.

PR #143's Approach and Its Limitations

PR #143 proposes modeling this as goals composed of other goals — a "conditional goals" mechanism where a parent goal references child goals, Player API gains a goal_prerequisites pivot table, snapshot-at-click semantics, and an in-document cascade engine.

This framing has three structural problems:

  1. Phantom goals for non-goal conditions. In a Turbo campaign (Level 10 AND 2 purchases), "Make a Purchase" is a raw-event condition, not something that should ever appear as a goal with a reward. The goal-composition model forces the creation of phantom leaf goals for conditions that should never be goals.

  2. Repetition (counting) has no intermediate goal to compose. When the advertiser's MMP emits a generic lvl_complete event (not lvl_10_complete), reaching Level 10 requires counting 10 occurrences. There is no intermediate goal to compose — the shape is "one raw event, counted," which the goal-composition model cannot express.

  3. Redundant layer. Even with goal composition, the system still needs event derivation underneath to turn raw SDK events into leaf-goal completions. Goal composition adds a layer on top of the derivation problem rather than solving it.

The Reframing: Composite Event as the Primitive

The missing primitive is the composite event: a derived event that the ingestion layer synthesizes from a pattern over raw MMP/SDK postbacks, and which a goal subscribes to. The pipeline becomes:

raw events → composite event → flat goal

Four definitions carry the design:

TermDefinition
Raw eventAn event emitted by an advertiser's MMP SDK (AppsFlyer, Adjust), delivered as a postback. Externally defined, atomic, outside our control. Examples: lvl_10_complete, lvl_complete, af_purchase.
Composite eventAn event we derive by evaluating a pattern over raw events. It is the single trigger a goal listens to. Spans a spectrum of pattern complexity but is always one thing: "this combination of raw events occurred."
GoalThe reward-bearing unit in the player/transaction domain. Subscribes to exactly one composite event and carries the reward. Goals are flat — they never reference or compose other goals.
MappingThe campaign-authored definition (owned in AdSuite) of (a) which raw-event pattern produces which composite event, and (b) which composite event each goal subscribes to.

Goal-Trigger Shapes

Every goal completion originates from raw SDK events. What varies is the pattern required. Most goals use a simple 1:1 mapping that the existing Keystone lookup path already handles (ADR-0033). The composite event engine is only needed for the two shapes that require derivation:

ShapeTriggerEngine needed?Example → Goal
Single (1:1)One raw event, direct lookupNo — existing ADR-0033 path (token_namegoal_id)lvl_10_completeComplete Level 10
RepetitionOne raw event, countedYes — counter state requiredlvl_complete ×10 → Complete Level 10
CombinationMultiple raw-event patternsYes — multi-condition state requiredlvl_10_complete ×1 AND af_purchase ×2 → Turbo reward

The single (1:1) shape is the degenerate case — it is conceptually a composite event with a single condition and a threshold of 1, but it requires no counter state and no engine involvement. Single mappings continue to use the existing Keystone lookup path (ADR-0033). Composite event definitions are only authored when the shape requires actual derivation (repetition or combination).

The key property for the shapes that do require the engine: every goal hangs off exactly one composite event, and no goal points at another goal. The "Turbo" case is not special — it is a combination composite event with a bigger reward attached. There is nothing for a goal-composition mechanism to do, and the phantom-goal problem never arises.

Current State (Batch + Tune Prototype)

Today a batch process matches raw events and triggers goals directly in Tune. Functionally, Tune + the batch job is an ad-hoc composite-event engine — it does the derivation, but in batch, with logic scattered and owned nowhere coherent. This is the system that caused Incident #248.

Decision Drivers

  • Player contract integrity. Goals are the reward-bearing primitive that players see. Phantom goals for internal conditions violate the player contract.
  • Player API stays flat. Per ADR-0029 and ADR-0031, Player API was designed as a player-domain transactional service replacing the ad-tracking package — not an event-processing system. Composition logic does not belong there.
  • Single source of truth in AdSuite. AdSuite is where campaigns are designed. Composite-event definitions and goal subscriptions should be authored there, not scattered across Player API and batch scripts.
  • Realtime replaces batch. The batch-plus-Tune prototype is a reliability liability (Incident #248). Processing postbacks as they arrive via the existing SQS → Lambda pipeline with durable capture eliminates the batch failure class.
  • Keystone is the natural home. Keystone (Postback Processor) already receives MMP postbacks via API Gateway → SQS → Lambda (ADR-0009), already performs DynamoDB lookup for server-side goal resolution (ADR-0033), and already forwards to Tune and Firehose. Extending it with composite-event derivation is a natural evolution, not a new system.

Considered Options

Option A: Goals Compose Goals (PR #143)

A goal references other goals; Player API gains a goal-composition engine with a goal_prerequisites pivot table.

Pros:

  • Conceptually simpler for the single shape
  • Changes isolated to Player API

Cons:

  • Phantom goals required for non-goal conditions
  • Cannot express the repetition (counting) shape — no intermediate goal to compose
  • Redundant layer — still needs event derivation underneath for leaf-goal completions
  • Does not replace the batch+Tune prototype
  • Violates Player API's flat-goal domain design (ADR-0029/0031)

Option B: Composite Events Inside Player API

Player API ingests raw SDK events and derives composite events itself.

Pros:

  • Single system end-to-end; no new cross-system contract
  • All three shapes expressible

Cons:

  • Couples Player API to the entire MMP/SDK event taxonomy, per-advertiser quirks, counting state, and dedup
  • Player API stops being a flat goal domain and becomes an advertiser-ingestion system
  • Contradicts the design intent of ADR-0029/0031

AdSuite authors composite-event definitions and goal subscriptions. It emits two artifacts:

  1. Flat goal definitions → pushed to Offer API / player transaction system (goals know only about rewards, never raw events)
  2. Composite-event definitions → pushed to DynamoDB, read by Keystone's existing Lambda consumer. When a postback matches a composite-event condition, the Lambda performs DynamoDB counter updates and fires goal completions into Player API when all conditions are met

Pros:

  • Goals stay flat; Player API ignorant of SDK events
  • All three shapes (single, repetition, combination) handled uniformly
  • No phantom goals
  • Single owned source of truth for definitions (AdSuite)
  • Realtime replaces brittle batch+Tune; closes Incident #248 class
  • Extends existing Keystone infrastructure (ADR-0009, ADR-0033) rather than building new
  • Definition versioning protects in-flight campaigns

Cons:

  • DynamoDB counter state adds operational complexity (drift, orphans, TTL management)
  • New cross-system contract (AdSuite → Keystone definition push)
  • DynamoDB cost increase for counter/state tables
  • Operational surface area grows (monitoring, alerting, counter drift reconciliation)

Comparison

DimensionA: Goals compose goals (PR #143)B: Composite events in Player APIC: Upstream in Keystone
Single shapeYesYesYes (existing path — no new work)
Repetition (counting)No (no intermediate goal)YesYes
Combination with non-goal conditionsPhantom goals requiredYesYes
Removes the derivation problemNo (adds a layer on top)YesYes
Goals stay flatNoYesYes
Player API coupled to MMP taxonomyPartialHeavyNone
Source of truth for definitionsScatteredPlayer APIAdSuite (where campaigns live)
Replaces batch+TuneNoYesYes
Solves the full problemNo (still needs event derivation underneath)YesYes
Extends existing infraNo (new mechanism)No (new concern in Player API)Yes (ADR-0009, ADR-0033)

DECISION

Adopt Option C: composite-event derivation upstream in Keystone, extending the existing SQS → Lambda pipeline with DynamoDB-backed counter state, using composite event as the primitive.

Rationale

The single strongest argument: composition belongs at the event layer, not the goal layer. Once a combination composite event exists, a Turbo Goal is just a flat goal subscribing to it — the goal-composition engine proposed in PR #143 has nothing to do, and the phantom-goal problem never arises. The repetition shape, which PR #143 cannot express at all, falls out naturally.

Keystone is the natural home because it already sits at the postback ingestion boundary (ADR-0009), already performs DynamoDB-backed server-side lookups (ADR-0033), and already forwards to Tune and Firehose. Keystone is the foundational component for advertiser integrations — it is where raw postbacks enter our platform, and composite-event derivation is a natural extension of that responsibility.

Correlation Key: Transaction ID, Not Player ID

The composite event engine correlates postbacks using the click transaction ID (transaction_id), not player_id. This is the right key because:

  • It's already on every postback. Post-install events for a given install/click share the same transaction ID, echoed back from the tracking link. No enrichment or lookup needed.
  • It decouples Keystone from the player domain. Keystone never needs to resolve players — it operates entirely in the advertiser-integration domain (clicks, installs, post-install events). Player identity is Player API's concern.
  • It works with both delivery paths. Whether the composite event is delivered through Tune (initial state) or directly to Player API (future state), transaction_id is the correlation key both systems understand.

Delivery Path: Through Tune Initially, Direct to Player API Later

When a composite event is satisfied, the evaluator needs to deliver the result downstream. This ADR distinguishes between the initial delivery path and the future delivery path:

Initial state: Keystone → Tune → AdGem. The composite event module sends the satisfied event to Tune with the transaction_id, the same way Keystone delivers all postbacks today. Tune performs the event-to-goal mapping and triggers the conversion in AdGem. This is the most incremental step — it changes where composition happens (Keystone instead of batch scripts) without changing the downstream delivery flow.

Future state: Keystone → Player API directly. Once Keystone owns the event-to-goal mapping (via the composite event definitions from AdSuite), it can bypass Tune and call Player API directly with transaction_id + goal_id using the existing POST /v1/transactions/{tx_id}/goals/{goal_id} endpoint (ADR-0031). This is the target architecture but is not required for the initial launch of composite events.

The architecture diagram and processing logic below reflect the initial state (delivery through Tune). The composite event primitive, the counter state, and the unconditional fan-out are the same in both states — only the delivery target changes.

Architectural Separation Within Keystone

Keystone-core vs. Keystone-platform

To keep the boundary clear, we distinguish two layers:

  • Keystone-core is the stateless ingestion path: API Gateway → SQS → Lambda (ADR-0009), normalizing and routing postbacks, with the ADR-0033 1:1 lookup, Tune forwarding, and Firehose delivery. It holds no durable per-transaction domain state. Every message is processed independently. This is the load-bearing property: it is why Keystone scales on raw postback volume, deploys fearlessly, and recovers by replay.
  • Keystone-platform is the broader ecosystem: the paved road (CDK stack, monitoring, deploy pipeline, IAM patterns) onto which we can place additional Lambdas, queues, and stores for advertiser-integration concerns.

The composite event evaluator lives "in Keystone" in the platform sense — it shares the paved road, the repo, and the CDK stack — without compromising Keystone-core. The evaluator is a separate Lambda with its own SQS queue and its own DynamoDB tables.

Unconditional fan-out

Keystone-core already fans out every postback to Tune and to Firehose. The composite event queue is the same pattern: after completing its existing work, the routing Lambda unconditionally publishes the normalized postback to the internal composite event SQS queue — the same way it already feeds Firehose, with no definition lookup. The composite event Lambda decides for itself, against its own definitions store, which postbacks are relevant and which to ignore.

This keeps Keystone-core ignorant of composite events. It never queries composite_event_definitions, never branches on composite logic, and never takes a data dependency on the evaluator's store. A change to composite event definitions has zero impact on the routing path.

Module separation

Keystone-core (existing)Composite Event Module (new)
LambdaExisting routing handlerNew, separate handler
TriggerExisting edge SQS queueNew internal SQS queue, fed unconditionally
DynamoDB tablesADR-0033 lookup tablecomposite_event_definitions, composite_event_counters
StateStateless — no durable per-transaction stateStateful — per-transaction counter records
ResponsibilityReceive postback, forward to Tune + Firehose, resolve 1:1 goal lookup, unconditional fan-outEvaluate composite-event conditions, maintain counter state, deliver satisfied events (to Tune initially, Player API in future)
Failure blast radiusUnchanged — a failure here does not affect composite event processing (postback is already on the internal queue)Isolated — a failure here does not affect Tune forwarding, Firehose delivery, or 1:1 goal completions

The composite event module can be deployed, scaled, and rolled back independently of Keystone-core. If it fails or is disabled, all existing Keystone flows continue unaffected.

Enforcement

Because a shared repo and stack make boundary erosion a small diff, the separation needs mechanical enforcement, not just convention:

  • Separate IAM roles per module. Keystone-core's execution role cannot read or write the evaluator's DynamoDB tables, and vice versa.
  • CI check for cross-module store access. A routing Lambda import of or reference to composite_event_definitions or composite_event_counters fails the build.
  • Separate DynamoDB table credentials. No cross-grants between the modules' data stores.

Target Architecture

DynamoDB Schema Sketches

composite_event_definitions table — pushed by AdSuite, read by Keystone engine:

AttributeTypeDescription
PKStringCAMPAIGN#{campaign_id}
SKStringCOMPEVT#{composite_event_id}
definition_versionNumberMonotonic version; pinned per campaign activation
shapeStringsingle / repetition / combination
conditionsListArray of {token_name, threshold} objects
goal_idNumberThe flat goal this composite event fires
window_ttl_secondsNumberCampaign-defined time window for partial matches
created_atStringISO 8601 timestamp

composite_event_counters table — written by Keystone engine per transaction per composite event:

AttributeTypeDescription
PKStringTX#{transaction_id}#CAMPAIGN#{campaign_id}
SKStringCOMPEVT#{composite_event_id}#COND#{token_name}
countNumberIdempotent counter (deduplicated by event_id)
seen_event_idsStringSetSet of processed event IDs for dedup
definition_versionNumberVersion of the definition this counter was created against
ttlNumberDynamoDB TTL epoch for window expiry
satisfiedBooleanWhether this condition has met its threshold

Processing Logic

Keystone-core (existing Lambda, unchanged except unconditional fan-out):

  1. Durable capture. Raw postback arrives at API Gateway, is enqueued to the edge SQS queue (existing ADR-0009 pattern). The postback is persisted before any processing — a processor outage causes delay, not loss.

  2. Route and forward. The routing Lambda processes the postback as it does today: ADR-0033 lookup resolves goal_id for direct 1:1 mappings, forwards the postback to Tune, and sends raw event data to Firehose. These existing flows are unchanged.

  3. Unconditional fan-out. After completing its existing work, the routing Lambda publishes the normalized postback to the internal composite event SQS queue — unconditionally, for every postback, the same way it already feeds Firehose. No definition lookup, no branching. The routing Lambda does not know composite events exist.

Composite Event Module (new Lambda, separate IAM role):

  1. Filter and match. The composite event Lambda consumes from its internal queue and queries its own composite_event_definitions table to determine if the postback's campaign_id + token_name matches any composite-event condition. If no match, the postback is discarded. The filtering responsibility lives entirely in the evaluator.

  2. Idempotent counter increment. For each matching composite-event definition, the Lambda performs a conditional DynamoDB update on the counter record:

    • Uses seen_event_ids set to deduplicate (ADD to StringSet is idempotent)
    • Increments count only if the event ID was not already in the set
    • Counter operations are order-independent — postbacks that arrive out of order produce the same final count
  3. Evaluate and fire. After incrementing, the Lambda checks whether all conditions in the composite event definition are satisfied (each condition's count >= threshold). If all are met and the composite event has not already been fired (check satisfied flag), it delivers the satisfied composite event downstream. In the initial state, this means sending the event + transaction_id to Tune, which performs the event-to-goal mapping and triggers the conversion in AdGem — the same flow as any other Keystone-to-Tune postback. In the future state, Keystone owns the event-to-goal mapping and calls Player API directly with transaction_id + goal_id.

CONSEQUENCES

Positive

  • Closes the incident class. Durable capture + realtime processing eliminates the batch-failure mode that caused Incident #248.
  • Goals stay flat. Player API remains a player-domain transactional service per ADR-0029/0031. No phantom goals, no composition engine, no new pivot tables.
  • All shapes handled uniformly. Single, repetition, and combination composite events use the same engine, the same definition schema, and the same counter mechanism.
  • Realtime replaces batch. Goal completions fire within seconds of the triggering postback, not on a batch schedule.
  • Single source of truth. Composite-event definitions live in AdSuite alongside campaign configuration. No scattered logic across batch scripts, Tune config, and Player API.
  • Keystone-core stays stateless. Unconditional fan-out means the routing Lambda never queries composite event definitions and never branches on composite logic. It remains stateless, volume-scaling, and recoverable by replay.
  • Extends existing infrastructure. Builds on Keystone's API Gateway → SQS → Lambda pipeline (ADR-0009) and DynamoDB lookup pattern (ADR-0033) rather than creating a new system.
  • Definition versioning protects in-flight campaigns. Pinning a definition version per campaign activation means mid-flight definition edits don't disrupt partial matches.
  • Domain event contract aligns with ADR-0057. AdSuite → Keystone definition pushes can use the domain event pattern established for Offer API syncing.

Negative

  • Multi-system implementation. Spans Keystone, AdSuite, and the definition push contract. Includes DynamoDB tables, engine logic, definition push pipeline, and shadow-mode validation.
  • DynamoDB counter state. Per-transaction counter records introduce operational concerns: counter drift, partial-match orphans, TTL management, and potential hot partitions.
  • New cross-system contract. The AdSuite → Keystone definition push is a new integration surface that both teams must maintain.
  • DynamoDB cost increase. Counter table writes scale with postback volume × composite events per campaign. See Appendix for cost estimate.
  • Unconditional fan-out increases SQS volume. Every postback is enqueued to the composite event queue, not only those that match a definition. This is the same trade-off Firehose already makes and is cheap, but it is not free. The composite event Lambda discards non-matching postbacks quickly.
  • Operational surface grows. New monitoring for counter state, definition sync health, composite-event fire rates, and reconciliation drift.

Risks

RiskLikelihoodImpactMitigation
Counter state drift — counters diverge from actual postback historyLowHighPeriodic reconciliation job compares counter state against Firehose/Redshift raw events; alert on drift beyond threshold
Dropped postbacks — SQS message lost before processingVery LowHighSQS durability guarantee + DLQ (existing ADR-0009 pattern); DLQ alarm triggers investigation and replay
Definition push failure — AdSuite change doesn't propagate to Keystone DynamoDBLowHighAdSuite publishes via SQS/EventBridge with DLQ; Keystone exposes health-check endpoint for definition freshness; reconciliation on deploy
DynamoDB throttling — burst traffic exceeds table capacityLowMediumOn-demand billing mode (per ADR-0033 pattern); monitor consumed capacity; consider DAX cache layer if hot-partition issues emerge
Definition version mismatch — engine uses stale definition for active campaignLowMediumDefinition version pinned at campaign activation; engine validates version on every lookup; stale-version metric with alert
Duplicate goal completions — same composite event fires twiceLowHighsatisfied flag in counter table prevents re-fire; Player API rejects duplicate goal completions on the same transaction
Partial-match orphans — transaction never completes all conditions; counter state accumulatesMediumLowTTL on counter records tied to campaign window; DynamoDB TTL auto-deletes expired records

Migration Path (Strangler Pattern)

Phase 1: Shadow Mode

Deploy the composite-event engine alongside the existing batch+Tune pipeline. The engine processes postbacks and logs "would-fire" decisions but does not fire goal completions. Reconcile its decisions against the batch job output per campaign.

  • Success criteria: Engine's would-fire decisions match batch job output within a configurable tolerance (e.g., 99.5% agreement over 7 days)
  • Duration: 2-4 weeks per campaign shape (start with single, then repetition, then combination)

Phase 2: Dual-Write with DevCycle Flag

Enable the engine to fire real goal completions, gated by a DevCycle feature flag per campaign. Both the engine and the batch job fire; downstream deduplication (Player API idempotency) prevents double-crediting.

  • Rollout: Start with low-risk single-shape campaigns, expand to repetition, then combination
  • Monitoring: Compare fire rates, latency, and player-facing outcomes between the two paths

Phase 3: Campaign-by-Campaign Cutover

For each campaign where Phase 2 has validated parity:

  1. Disable the batch job for that campaign
  2. Remove the DevCycle flag (engine fires unconditionally)
  3. After all campaigns are migrated, retire the batch scripts and the Tune-based derivation logic

NOTES

References

Naming Note

"Composite event" is the engineering primitive and should be the internal model. The campaign-builder UI in AdSuite can present it to advertisers as a goal type or "Turbo Goal" — the internal model and the product surface do not need to share a name.

Event Definition Governance in AdSuite

Composite event definitions create a dependency: active campaigns rely on the events (raw and composite) they reference. AdSuite must govern this as part of its configuration of advertiser apps and their events. The compatibility rules are:

  • Adding a new event (raw or composite) is backwards-compatible. No active campaign is affected.
  • Removing an event or changing the aggregation signals of a composite event that is referenced by an active campaign is a breaking change. AdSuite should either prevent the change while campaigns are active, or require the affected campaigns to be paused and relaunched after the change is applied.

This governance belongs in AdSuite because it is the authoring surface for both event definitions and campaign configuration — it has the information needed to enforce these constraints at edit time.

Open Questions

  1. AdSuite authoring surface. Does AdSuite already hold enough campaign structure to author repetition/combination composite-event definitions, or is a new authoring UI needed?
  2. Counter state storage. Where should composite-event counter state live given existing stream infrastructure — reuse an existing DynamoDB table or create new?
  3. Composite event overlap. Do any current goals genuinely subscribe to the same composite event (single + combination overlap), and does that change the definition contract?
  4. Reconciliation tolerance. What is the acceptable drift threshold between the engine's fire decisions and the batch job during shadow mode? (e.g., 99.5% agreement)
  5. Definition version pinning. Should definition versions be pinned at campaign activation, or should a campaign be able to opt into a "latest" mode for testing?
  6. AppsFlyer integration. ADR-0033 covers Adjust; what is the timeline and shape for AppsFlyer postback integration into Keystone?
  7. Cross-campaign composite events. Can a composite event span conditions across multiple campaigns for the same transaction, or is composition always scoped to a single campaign?

Original Author

Nick Haynes

Approval date

Approved by

Appendix

Worked Examples

Goal (flat, reward-bearing)TriggerShapeHow it fires
Complete Level 10lvl_10_completegoal_idSingle (1:1)Existing ADR-0033 path. Postback arrives; token_namegoal_id DynamoDB lookup fires goal completion directly. No composite event definition needed.
Complete Level 10 (no granular event)10x lvl_completeRepetitionComposite event engine. Each lvl_complete postback increments the counter; on the 10th unique event, threshold met; fires.
Turbo reward1x lvl_10_complete AND 2x af_purchaseCombinationComposite event engine. Each postback increments the relevant condition counter; when all conditions meet their thresholds, fires.

Why PR #143 Cannot Express Repetition

The repetition shape requires counting N occurrences of the same raw event. In the goal-composition model, each "occurrence" would need to be a separate leaf goal — but there is no mechanism to create N identical goals, and the MMP emits the same event token each time with no ordinal. The composition model presupposes that each leaf goal is a distinct, meaningful event. Counting is a fundamentally different operation that the goal layer cannot express.

DynamoDB Cost Estimate

Following the ADR-0033 cost analysis pattern (on-demand pricing, us-east-1):

Assumptions:

  • 2.3M postbacks/month (per ADR-0033 traffic data)
  • Only a subset of postbacks participate in composite events (repetition/combination shapes); single 1:1 mappings use the existing ADR-0033 lookup and do not touch these tables
  • Estimated 30-50% of postbacks match a composite-event definition (varies by campaign mix)
  • Average 1.2 counter writes per matching postback
  • Counter reads for threshold evaluation included in the write transaction

Definitions table:

  • Reads: ~3.5M/month × $0.25/million = ~$0.88/month
  • Writes: ~10K/month (definition changes are infrequent) = negligible
  • Storage: <1MB = negligible

Counters table:

  • Writes: ~2.8M/month × $1.25/million = ~$3.50/month
  • Reads: ~2.8M/month × $0.25/million = ~$0.70/month
  • Storage: ~50MB (assumes 500K active transaction-campaign-condition records with TTL cleanup)

Total estimated cost: ~$5-8/month (~$10-12 during peak periods)

This is consistent with the ADR-0033 DynamoDB cost profile and represents negligible incremental cost.

Sequence Diagram: Combination Composite Event End-to-End