0057: Real-time Offer API Syncing
STATUS
Accepted
CONTEXT
AdGem currently synchronizes offers from the internal database to the Offer API using a scheduled batch script (SyncOfferApiV2). This script runs on a cron schedule:
- At the 20th minute of each hour for some apps
- At the 50th minute of each hour for some apps
- At the 10th and 40th minutes for strategic apps (twice hourly)
While functional, this approach introduces latency between when data changes occur in the dashboard and when those changes are reflected in the Offer API.
Current Architecture
┌─────────────────────────────────────────────────────────────────────┐
│ Current Batch Sync Flow │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ Dashboard Changes Scheduled Task Offer API │
│ ┌──────────────┐ ┌─────────────┐ ┌──────────┐ │
│ │ Campaign │ │ │ │ │ │
│ │ Goal │ ──X──> │ SyncOffer │ ────> │ Offers │ │
│ │ Offerwall │ (wait) │ ApiV2 │ │ │ │
│ │ Margin Ovr │ │ │ │ │ │
│ └──────────────┘ └─────────────┘ └──────────┘ │
│ │ │
│ Runs at :20, :50, or :10/:40 │
│ depending on app tier │
└─────────────────────────────────────────────────────────────────────┘
Problem Statement
The batch synchronization approach has several limitations:
- Latency: Changes to campaigns, goals, payouts, or targeting can take 10-40 minutes to propagate to the Offer API
- Inefficiency: Full app syncs process all offers even when only a single field changed
- Stale Data: Publishers and end-users may see outdated offer information
- Operational Overhead: Increased API calls during batch windows can strain the Offer API
Business Drivers
- Payout Accuracy: When a goal's margin or revenue changes, the corresponding offers should reflect updated payouts promptly to avoid revenue leakage or over-payment
- Offer Integrity: Contract changes (targeting, rewards) should trigger offer supersession to maintain offer integrity
- Publisher Experience: Publishers expect changes made in the dashboard to be reflected quickly
- Operational Efficiency: Processing only changed data reduces system load on the Offer API. During batch sync windows, increased API traffic can degrade response latency for publisher-facing HTTP requests. Event-driven syncing distributes load more evenly and targets only affected offers, promoting more focused use of Offer API resources
Historical Context: Previous Real-time Implementation
A previous attempt at real-time syncing left behind event infrastructure that still exists in the codebase:
MarginOverrideCreated,MarginOverrideUpdated,MarginOverrideDeletedeventsOfferwallCreated,OfferwallUpdatedeventsUpdateOffersForCampaignlistener
This implementation was problematic because it superseded offers for any observed change, regardless of whether the change affected contract fields. The approach was abandoned in favor of the batch sync, but the event definitions remain.
Dashboard Access Patterns Bypassing Model Events
A contributing factor to the previous implementation's failure — and an ongoing limitation for any observer-based approach (Options 1 and 2) — is that the dashboard codebase has many code paths that bypass Laravel's Eloquent model events. Direct queries, bulk updates, raw SQL, and other access patterns that don't go through Eloquent's save() / update() methods will not trigger observers or fire model events.
This means an observer-based real-time sync can silently miss changes, leading to drift between the dashboard database and the Offer API. Addressing this requires either:
- Auditing and refactoring all code paths that modify synced fields to ensure they go through Eloquent (significant effort in a legacy codebase)
- Choosing an architecture where the source of events is explicit (Options 4/5/6), where services intentionally publish domain events rather than relying on model lifecycle hooks
This limitation strengthens the case for Options 4/5/6, where event publishing is an explicit, intentional action rather than a side effect of the ORM.
Current Real-time Capabilities
Some event-driven syncing currently works via Eloquent Observers:
| Trigger | Current Action | Implementation |
|---|---|---|
| Campaign status → active/paused | Enable/Disable offers | CampaignObserver |
| Campaign capped → true | Disable offers | CampaignObserver |
| App status → active/disabled | Enable/Disable offers | AppObserver |
Contract Fields
The Offer API defines "contract fields" that represent the core identity of an offer. Changes to these fields require supersession (delete + recreate with new ID):
total_amount- Total reward amount for the usertotal_payout_usd- Total payout to publishergoals- Goal structure including individual payoutsis_multi_reward- Whether offer has multiple goalslocation_targets- Geographic targetingdevice_targets- Device type targetingos_targets- Operating system targeting
Non-contract field changes can be applied via PATCH (update in place).
Entities Requiring Event-driven Sync
| Entity | Field(s) | Impact | Action |
|---|---|---|---|
| Goal | margin, revenue |
total_payout_usd, goals |
Supersede |
| Goal | description, order, completion times |
Metadata | Update |
| MarginOverride | margin_value, apps, campaigns, publishers, active |
total_payout_usd |
Supersede |
| Offerwall | multiplier |
total_amount |
Supersede |
| Offerwall | currency_name_singular, currency_name_plural |
Metadata | Update |
| CampaignOfferwallCreative | All text/image fields | Metadata | Update |
| TargetingProfile | android, ios, OS versions |
device_targets, os_targets |
Supersede |
| TargetLocation/TargetDevice | Created/deleted | location_targets, device_targets |
Supersede |
| Campaign | start_datetime, end_datetime, etc. |
Metadata | Update |
Scale Considerations
- Some apps have 1500+ offers
- Targeting profiles are designed for one-to-many relationships (though rarely used this way in practice)
- MarginOverrides can match broadly (by publisher, app, or campaign arrays where null means "all")
Considered Options
Option 1: Extend Eloquent Observer Pattern
Leverage the existing Observer infrastructure. Each model gets an observer that dispatches specialized jobs based on which fields changed.
┌─────────────────────────────────────────────────────────────────────┐
│ Option 1: Observer Pattern │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ Model Change Observer Job Queue │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ │
│ │ Goal.margin │ ────> │ GoalObserver │ ────> │ Supersede│ │
│ │ changed │ │ wasChanged() │ │ Campaign │ │
│ └──────────────┘ └──────────────┘ │ Offers │ │
│ └──────────┘ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ │
│ │ Creative.name│ ────> │ Creative │ ────> │ Update │ │
│ │ changed │ │ Observer │ │ Campaign │ │
│ └──────────────┘ └──────────────┘ │ Offers │ │
│ └──────────┘ │
└─────────────────────────────────────────────────────────────────────┘
Pros:
- Uses Laravel's built-in
wasChanged()for field-level detection - Simple to understand for developers familiar with Laravel
- No new infrastructure required
- Gradual rollout via feature flags
Cons:
- Routing logic scattered across multiple observer classes
- Each observer must independently determine supersede vs update
- Harder to see the big picture of what triggers what
- Deduplication and batching must be handled per-observer or in jobs
- Adding new field handlers requires modifying observer classes
- Previous real-time implementation failed using this pattern
Option 2: Centralized Event Bus with Field-Level Routing
Create an OfferSyncEventBus service that receives model change events and routes them to appropriate handlers based on configurable field→action mappings.
┌─────────────────────────────────────────────────────────────────────┐
│ Option 2: Event Bus Pattern │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ Model Change Lightweight Event Bus │
│ ┌──────────────┐ Observer ┌──────────────────────┐ │
│ │ Goal.margin │ ┌─────────┐ │ OfferSyncEventBus │ │
│ │ changed │──>│ Emits │─────────>│ │ │
│ └──────────────┘ │ Event │ │ ┌────────────────┐ │ │
│ └─────────┘ │ │ Field Router │ │ │
│ ┌──────────────┐ ┌─────────┐ │ │ │ │ │
│ │ Creative.name│──>│ Emits │─────────>│ │ goal.margin │ │ │
│ │ changed │ │ Event │ │ │ → Supersede │ │ │
│ └──────────────┘ └─────────┘ │ │ │ │ │
│ │ │ creative.* │ │ │
│ │ │ → Update │ │ │
│ │ └────────────────┘ │ │
│ │ │ │
│ │ ┌────────────────┐ │ │
│ │ │ Deduplication │ │ │
│ │ │ & Batching │ │ │
│ │ └────────────────┘ │ │
│ │ │ │ │
│ └──────────┼───────────┘ │
│ ▼ │
│ ┌──────────────┐ │
│ │ Job Queue │ │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
Pros:
- Centralized routing logic: All field→action mappings in one configurable location
- Single point for batching/deduplication: Prevents duplicate jobs when multiple fields change
- Better visibility: One place to understand all sync triggers
- Cleaner separation of concerns: Observers emit events; bus decides actions
- Easier configuration: Add new handlers without modifying observer code
- Testability: Test routing logic once in the bus, not across multiple observers
- Addresses previous failure: Previous implementation lacked proper field→action routing
Cons:
- More upfront infrastructure to build
- New pattern to learn for the team
- Slightly more complex architecture
Option 3: Database Triggers / Change Data Capture
Use database-level triggers or CDC tools (e.g., Debezium) to capture changes and push to a message queue.
Pros:
- Captures all changes regardless of application code path
- Decoupled from application logic
- Works for changes made outside the dashboard
Cons:
- Significant infrastructure complexity
- Harder to debug and test
- Overkill for current requirements (changes originate from dashboard)
- Database-specific implementation
- Requires additional services (Kafka, Debezium, etc.)
Option 4: Domain Events to Offer API (Hybrid)
Rather than building the Event Bus in the dashboard, the dashboard emits lightweight domain events describing what changed, and the Offer API owns the Event Bus, routing logic, and transformation.
┌─────────────────────────────────────────────────────────────────────────────┐
│ Option 4: Domain Events to Offer API │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Dashboard Message Queue Offer API │
│ ┌──────────────┐ ┌─────────────┐ ┌────────────────┐ │
│ │ Goal.margin │ │ │ │ Event Bus │ │
│ │ changed │──(event)─>│ SQS/Redis │ ────────> │ ├─ Routing │ │
│ │ │ │ │ │ ├─ Transform │ │
│ │ Emits domain │ │ │ │ ├─ Supersede │ │
│ │ event only │ └─────────────┘ │ └─ Persist │ │
│ └──────────────┘ └────────────────┘ │
│ │
│ Dashboard knows: Offer API knows: │
│ - Campaigns, Goals - What changes mean for offers │
│ - Offerwalls, Margins - Field → action mappings │
│ - "What changed" - Supersession vs update logic │
│ - Batching, deduplication │
└─────────────────────────────────────────────────────────────────────────────┘
What are domain events?
Domain events describe what changed in the dashboard's domain without encoding what it means for offers:
// Dashboard emits these events (knows nothing about offers)
GoalMarginChanged { goal_id, campaign_id, old_margin, new_margin }
OfferwallMultiplierUpdated { offerwall_id, app_id, old_multiplier, new_multiplier }
CampaignCreativeUpdated { campaign_id, changed_fields[] }
MarginOverrideCreated { margin_override_id, apps[], campaigns[], publishers[] }
// Offer API translates these to offer operations
"GoalMarginChanged" → recalculate payouts → supersede affected offers
"OfferwallMultiplierUpdated" → recalculate rewards → supersede all app offers
This inverts responsibility: the dashboard knows about campaigns/goals/offerwalls; the Offer API knows about offers.
Pros:
- Future-proofing: Logic lives in the target system (Offer API), not the legacy system (dashboard)
- New clients: Future systems replacing the dashboard use the same event pathway
- Clean separation: Dashboard describes "what changed"; Offer API decides "what to do"
- Single source of truth: Offer API owns all offer-related business logic
- Testability: Offer API can be tested independently with synthetic events
Cons:
- More upfront work: Requires building infrastructure in Offer API
- Domain coupling: Offer API must understand dashboard entities (Goal, Offerwall, etc.)
- Cross-team coordination: Changes may require updates to both systems
- Message queue dependency: Introduces SQS/Redis as critical infrastructure
Trade-offs: Option 2 vs Option 4
| Consideration | Option 2 (Dashboard Bus) | Option 4 (Offer API Bus) |
|---|---|---|
| Time to implement | Faster | More upfront work |
| Future-proofing | Logic in legacy system | Logic in target system |
| New clients | Must replicate or call dashboard | Use same event pathway |
| Domain coupling | Offer API stays "dumb" | Offer API understands source entities |
| Team ownership | Dashboard team owns sync logic | Offer API team owns sync logic |
| Testing | Test in dashboard context | Test with synthetic events |
| Debugging | Logs in one system | Logs span two systems + queue |
Future State: Service-Oriented Architecture
The current dashboard is being decomposed into separate services:
- AdSuite: Campaign and goal management, including margin overrides (admin-facing)
- Publisher Portal: App/property management (publisher-facing)
This evolution affects our architecture choices. Options 5 and 6 below extend Option 4 to accommodate multiple source services.
Why Not AWS Glue?
AWS Glue was considered as a centralized ETL layer for transforming source data into offers. However, Glue is not a good fit for this use case:
- Batch-oriented: Glue is optimized for large batch ETL jobs, not real-time event processing
- Cold start latency: Glue jobs have 30-60 second minimum startup time, incompatible with near-real-time requirements
- Cost model: Glue pricing favors large, infrequent batch operations; frequent small changes would be expensive
- Coupling concern: If source services calculate payouts before sending to Glue, we couple them to offer structure, defeating the separation of concerns
Glue could complement real-time syncing as a periodic reconciliation layer (e.g., nightly verification), but should not be the primary sync mechanism.
Option 5: Hybrid SQS with Service-Specific Queues
Extending Option 4, each source service publishes domain events to its own SQS queue. The Offer API consumes from all queues and owns the transformation logic.
┌───────────────────────────────────────────────────────────────────────────────┐
│ Option 5: Service-Specific SQS Queues │
├───────────────────────────────────────────────────────────────────────────────┤
│ │
│ AdSuite Publisher Portal │
│ (campaigns, goals, (apps, offerwalls) │
│ targeting profiles, │
│ margin overrides) │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ SQS: │ │ SQS: │ │
│ │ campaign- │ │ publisher- │ │
│ │ events │ │ events │ │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ └─────────────────┬───────────────────┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Offer API │ │
│ │ ┌───────────────┐ │ │
│ │ │ Event Bus │ │ │
│ │ │ - Routing │ │ │
│ │ │ - Transform │ │ │
│ │ │ - Dedup │ │ │
│ │ │ - Supersede │ │ │
│ │ └───────────────┘ │ │
│ └─────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────────┘
Domain events by service:
// AdSuite emits campaign-domain events (including margin overrides)
GoalMarginChanged { goal_id, campaign_id, old_margin, new_margin }
GoalRevenueChanged { goal_id, campaign_id, old_revenue, new_revenue }
CampaignCreativeUpdated { campaign_id, changed_fields[] }
TargetingProfileUpdated { profile_id, campaigns[], changed_fields[] }
MarginOverrideCreated { margin_override_id, apps[], campaigns[], publishers[], margin_value }
MarginOverrideUpdated { margin_override_id, changed_fields[], apps[], campaigns[], publishers[] }
MarginOverrideDeleted { margin_override_id, apps[], campaigns[], publishers[] }
// Publisher Portal emits app-domain events
OfferwallMultiplierUpdated { offerwall_id, app_id, old_multiplier, new_multiplier }
OfferwallCurrencyUpdated { offerwall_id, app_id, currency_name, currency_name_plural }
AppStatusChanged { app_id, old_status, new_status }
Pros:
- Service isolation: Each service owns its event schema and queue
- Independent scaling: High-volume services can scale independently
- Failure isolation: One queue's issues don't affect others
- Clear ownership: Each team owns their event contract
- Gradual migration: Services can migrate to new queues independently
Cons:
- Multiple queues to manage: Operational overhead of multiple queues
- Cross-service effects: MarginOverrides (in AdSuite) can affect offers for apps managed in Publisher Portal
- Consumer complexity: Offer API must consume from multiple queues
- Event ordering: No guaranteed ordering across queues (may matter for some scenarios)
Option 6: AWS EventBridge as Central Event Router
Use AWS EventBridge as a central event bus with pattern-based routing. All services publish to EventBridge; the Offer API subscribes to relevant event patterns.
┌───────────────────────────────────────────────────────────────────────────────┐
│ Option 6: AWS EventBridge Central Router │
├───────────────────────────────────────────────────────────────────────────────┤
│ │
│ AdSuite Publisher Portal │
│ (campaigns, goals, (apps, offerwalls) │
│ margin overrides) │
│ │ │ │
│ │ │ │
│ └──────────────────┬───────────────────┘ │
│ ▼ │
│ ┌──────────────────────────────┐ │
│ │ AWS EventBridge │ │
│ │ │ │
│ │ Rules: │ │
│ │ ┌────────────────────────┐ │ │
│ │ │ source: "adsuite" │──┼──┐ │
│ │ │ detail-type: "Goal*" │ │ │ │
│ │ │ | "Margin*" │ │ │ │
│ │ └────────────────────────┘ │ │ │
│ │ ┌────────────────────────┐ │ │ │
│ │ │ source: "publisher" │──┼──┤ │
│ │ │ detail-type: "App*" │ │ │ │
│ │ │ | "Offer*" │ │ │ │
│ │ └────────────────────────┘ │ │ │
│ │ │ │ │
│ │ Features: │ │ │
│ │ - Dead-letter queues │ │ │
│ │ - Event archive & replay │ │ │
│ │ - Schema registry │ │ │
│ └──────────────────────────────┘ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Offer API │ │
│ │ ┌───────────────┐ │ │
│ │ │ Event Handler │ │ │
│ │ │ - Transform │ │ │
│ │ │ - Supersede │ │ │
│ │ │ - Persist │ │ │
│ │ └───────────────┘ │ │
│ └─────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────────┘
Pros:
- Single event bus: All services publish to one place
- Pattern-based routing: EventBridge rules filter events to appropriate targets
- Built-in features: Dead-letter queues, event archive, replay, schema registry
- Decoupled producers/consumers: Services don't need to know about Offer API
- Multiple consumers: Other services can subscribe to same events if needed
- AWS managed: Serverless, auto-scaling, high availability
Cons:
- Event size limit: 256KB per event (may matter for large payloads)
- AWS lock-in: Tight coupling to AWS EventBridge
- Cost: EventBridge charges per event published
- Latency: Adds a hop compared to direct SQS
- Learning curve: Team needs to learn EventBridge patterns and rules
Trade-offs: Option 5 vs Option 6
| Consideration | Option 5 (Service SQS) | Option 6 (EventBridge) |
|---|---|---|
| Infrastructure | Multiple SQS queues | Single EventBridge bus |
| Routing logic | In Offer API consumer | In EventBridge rules |
| Event filtering | Application-level | EventBridge patterns |
| Archive/replay | Manual implementation | Built-in |
| Schema validation | Manual implementation | Schema registry available |
| Multi-consumer | Requires SNS fan-out | Native support |
| AWS dependency | SQS only | EventBridge + rules |
| Event ordering | Per-queue FIFO available | Best-effort (no FIFO) |
DECISION
RECOMMENDED APPROACH: Phased Investment — Minimal Dashboard Fixes + Offer API Event Infrastructure
Rather than building the full event bus in the dashboard (Option 2), we recommend a phased approach that balances immediate business impact with long-term architectural investment. This avoids over-investing in the legacy dashboard while building the real-time infrastructure in the right place.
Phase A: Minimal Dashboard Fixes (Short-term)
Apply targeted, low-cost fixes to the dashboard for the highest-impact pain points:
- Goal payout changes (margin/revenue) — these are the most time-sensitive and directly affect publisher earnings. A focused observer or hook for goal payout changes provides immediate business value with minimal code
- Campaign status/capping — already handled by existing observers (no work needed)
- Retain
SyncOfferApiV2as the safety net for all other changes
This phase keeps dashboard investment minimal and focused only on changes where the 10-40 minute latency causes real business problems. The Option 2 implementation details in this document provide useful reference for the observer and job patterns needed in this phase.
Phase B: Build Event-Driven Infrastructure in Offer API (Medium-term)
Invest primarily in the Offer API's ability to consume domain events (per Options 4/5/6):
- Build the Event Bus / event handler in the Offer API
- Define the domain event schema and contracts
- Stand up the message queue infrastructure (SQS, EventBridge, etc.)
- Have the dashboard emit domain events for the limited Phase A triggers as a proof-of-concept
This positions the Offer API to receive events from any source service — dashboard today, AdSuite and Publisher Portal tomorrow.
Phase C: AdSuite / Publisher Portal Integration (Long-term)
As the new services come online, they publish domain events natively:
- AdSuite publishes campaign, goal, and targeting events
- Publisher Portal publishes app, offerwall, and property events
- Dashboard event publishing is decommissioned as its functionality is absorbed
SyncOfferApiV2is fully retired
Why not build the full Option 2 Event Bus in the dashboard?
While Option 2 provides a well-designed architecture for dashboard-based syncing, the dashboard is being actively decomposed into AdSuite and Publisher Portal. Building the full event bus, routing logic, and deduplication infrastructure in the dashboard risks creating throwaway work. Additionally, the dashboard has known access patterns that bypass Eloquent model events (see Context section), which limits the reliability of any observer-based approach.
Options 4/5/6 — where the Offer API owns the event bus and routing logic — are better long-term investments. The phased approach lets us capture the most urgent business value (Phase A) while directing the real infrastructure investment toward the target architecture (Phase B/C).
Why Centralized Event Bus Over Scattered Observers?
Regardless of where the event bus lives (dashboard in Phase A, Offer API in Phase B), the centralized bus pattern is preferred over scattered observers:
-
Learned from history: The previous observer-based implementation failed because it lacked proper field→action discrimination. A centralized bus makes this routing explicit and configurable.
-
Maintainability: As the number of tracked fields grows, having routing logic in one place prevents the "scattered observers" problem.
-
Deduplication: When a single save touches multiple fields (e.g., campaign edit updates description AND instructions), the bus can deduplicate into a single sync action.
-
Auditability: The bus can log all routing decisions, making it easier to debug why an offer was superseded vs updated.
Phase A Reference: Option 2 Implementation Details
The following implementation details from Option 2 serve as reference material for Phase A's targeted dashboard work (goal payout observer, sync jobs) and as a blueprint for the eventual Offer API event bus in Phase B. Not all of this needs to be built in Phase A — only the components needed for goal payout syncing.
1. OfferSyncEventBus Service
// app/Services/OfferApi/OfferSyncEventBus.php
class OfferSyncEventBus
{
// Configuration-driven field → action mapping
private array $supersessionTriggers = [
Goal::class => ['margin', 'revenue'],
Offerwall::class => ['multiplier'],
TargetingProfile::class => ['android', 'ios', 'min_ios', 'max_ios', 'min_android', 'max_android'],
];
private array $updateTriggers = [
Goal::class => ['description', 'order', 'minimum_completion_time', ...],
Offerwall::class => ['currency_name_singular', 'currency_name_plural'],
CampaignOfferwallCreative::class => ['*'], // All fields trigger update
Campaign::class => ['start_datetime', 'end_datetime', ...],
];
public function handle(ModelChangedEvent $event): void
{
// 1. Check feature flag
// 2. Determine action based on changed fields
// 3. Deduplicate if multiple changes to same entity
// 4. Dispatch appropriate job (or skip if no action needed)
}
}
Scenarios where no job is dispatched:
| Scenario | Example | Reason |
|---|---|---|
| Field not synced to Offer API | Offerwall currency_image, hex, button_accent_color, tab labels |
These fields are UI/display settings not included in offer data |
| Feature flag disabled | App not in useRealTimeOfferSync list |
Real-time sync not enabled for this app |
| No active offers exist | Campaign is paused or app is disabled | No offers in Offer API to update; changes will apply when campaign/app reactivates |
| Internal operational fields | Goal is_capping_goal, conversion_limit, spend_limit |
Used for internal tracking, not part of offer contract |
| Higher-scope job pending | Campaign update while app supersede is queued | Deduplication: higher-scope job will include this change |
| Non-payable goal changed | Progress goal (zero revenue) updated | Progress goals don't affect offer payout calculations |
Fields NOT synced to Offer API (no job needed):
Offerwall:
currency_image- icon asset, not syncedhex,button_accent_color,button_font_color- color themingheader_image,banner_image_path- offerwall UI assetsoffer_filter_setting,category_filter- filtering logicrandomization_chance- display algorithm settingdecimal_setting,use_thousands_separator- number formattingmy_offers_tab_label,all_offers_tab_label,custom_item_label,custom_item_label_plural- UI labelstitle- offerwall title, not per-offer
Goal:
is_attribution,is_purchase_goal,is_capping_goal- internal classification flagsconversion_limit,spend_limit- capping thresholds (handled separately)external_goal_id- third-party reference
Campaign:
- Internal tracking fields, advertiser relationships, and other fields not passed to
OfferSyncFormatter
// Example: Bus determines no action needed
class OfferSyncEventBus
{
private array $ignoredFields = [
Offerwall::class => [
'currency_image', 'hex', 'header_image', 'banner_image_path',
'button_accent_color', 'button_font_color', 'offer_filter_setting',
'category_filter', 'randomization_chance', 'decimal_setting',
'use_thousands_separator', 'my_offers_tab_label', 'all_offers_tab_label',
'custom_item_label', 'custom_item_label_plural', 'title',
],
Goal::class => [
'is_attribution', 'is_purchase_goal', 'is_capping_goal',
'conversion_limit', 'spend_limit', 'external_goal_id',
],
];
public function handle(ModelChangedEvent $event): void
{
$changedFields = array_keys($event->changedFields);
$ignoredForModel = $this->ignoredFields[get_class($event->model)] ?? [];
// Filter out ignored fields
$relevantChanges = array_diff($changedFields, $ignoredForModel);
if (empty($relevantChanges)) {
Log::debug("No relevant field changes for offer sync", [
'model' => get_class($event->model),
'changed_fields' => $changedFields,
]);
return; // No job dispatched
}
// Continue with routing logic...
}
}
2. Deduplication Logic
Deduplication prevents redundant or conflicting API calls when multiple changes occur that affect the same offers. There are two levels of deduplication to consider:
Within-Request Deduplication
When a single form save touches multiple fields, the bus coalesces events before dispatching:
| Scenario | Without Deduplication | With Deduplication |
|---|---|---|
Campaign edit updates description AND instructions |
2 UpdateOffersForCampaignJob dispatched | 1 UpdateOffersForCampaignJob dispatched |
Goal margin AND revenue both change |
2 SupersedeOffersForCampaignJob dispatched | 1 SupersedeOffersForCampaignJob dispatched |
| Multiple goals on same campaign edited | N jobs (one per goal) | 1 job (per campaign) |
Implementation: The bus buffers events during the request lifecycle and flushes once at the end, grouping by sync scope (campaign ID or app ID).
Cross-Request Deduplication
When changes happen in different parts of the system (different requests, users, or processes), we need job-level coordination:
Scenario 1: Offerwall multiplier updated + MarginOverride created for same app (both supersessions)
Request A: Offerwall.multiplier changed → SupersedeOffersForAppJob(app:123)
Request B: MarginOverride created for app:123 → SupersedeOffersForMarginOverrideJob
Timeline Option 1: App job runs first
→ App job fetches current DB state (includes new MarginOverride)
→ All offers superseded with correct payout (both changes reflected)
→ MarginOverride job runs but produces identical offers (redundant work)
Timeline Option 2: MarginOverride job runs first
→ Subset of offers superseded with new payout
→ App job runs, supersedes ALL offers again (redundant for the subset)
Resolution: Use Laravel's ShouldBeUnique with scope-aware unique IDs. The app-level supersede takes precedence:
class SupersedeOffersForAppJob implements ShouldBeUnique
{
public function uniqueId(): string
{
return "supersede:app:{$this->appId}";
}
}
class SupersedeOffersForMarginOverrideJob implements ShouldBeUnique
{
public function handle(): void
{
// Check if app-level supersede is queued/running
foreach ($this->affectedAppIds as $appId) {
if (Cache::has("supersede:app:{$appId}:pending")) {
// Skip this app - will be handled by app-level job
continue;
}
// Process offers for this app
}
}
}
Scenario 2: Offerwall multiplier updated (supersede) + Campaign instructions updated (update)
Request A: Offerwall.multiplier changed → SupersedeOffersForAppJob(app:123)
Request B: CampaignOfferwallCreative.instructions changed → UpdateOffersForCampaignJob(campaign:456)
Resolution: Supersede encompasses update. The supersede job recreates offers with all current data (including updated instructions). The update job should check for pending/recent supersessions:
class UpdateOffersForCampaignJob implements ShouldBeUnique
{
public function handle(): void
{
$appId = $this->campaign->apps->first()->id; // or iterate all apps
// If app-level supersede is pending, skip - it will include our changes
if (Cache::has("supersede:app:{$appId}:pending")) {
Log::info("Skipping update for campaign {$this->campaignId} - app supersede pending");
return;
}
// Proceed with update
}
}
Scenario 3: Offerwall currency names updated (update) + Campaign instructions updated (update)
Request A: Offerwall.currency_name changed → UpdateOffersForAppJob(app:123)
Request B: CampaignOfferwallCreative.instructions changed → UpdateOffersForCampaignJob(campaign:456)
Resolution: Both are updates but at different scopes. Since they patch different fields, both can proceed independently. However, if they race to the Offer API, we should ensure idempotency:
class UpdateOffersForAppJob
{
public function handle(): void
{
// Only patch the fields relevant to this change
$patchData = [
'currency_name' => $this->offerwall->currency_name_singular,
'currency_name_plural' => $this->offerwall->currency_name_plural,
];
foreach ($this->getOffersForApp() as $offer) {
$this->offerApiClient->patch($offer['id'], $patchData);
}
}
}
class UpdateOffersForCampaignJob
{
public function handle(): void
{
// Only patch the fields relevant to this change
$patchData = [
'instructions' => $this->creative->offer_instructions,
];
foreach ($this->getOffersForCampaign() as $offer) {
$this->offerApiClient->patch($offer['id'], $patchData);
}
}
}
Deduplication Summary
| Scope Conflict | Resolution |
|---|---|
| App supersede + Campaign supersede | App supersede wins (encompasses campaign) |
| App supersede + Campaign update | App supersede wins (includes fresh data) |
| App supersede + MarginOverride supersede | App supersede wins; MarginOverride skips affected apps |
| App update + Campaign update | Both proceed (patch different fields) |
| Campaign supersede + Campaign update (same campaign) | Supersede wins |
Key mechanisms:
ShouldBeUnique: Prevents duplicate jobs for the same scope- Pending flags in cache: Allow jobs to detect and defer to higher-scope operations
- Field-specific patches: Update jobs only patch their relevant fields, allowing concurrent updates at different scopes
3. Lightweight Observers
Observers become simple event emitters:
// app/Observers/GoalObserver.php
class GoalObserver
{
public function saved(Goal $goal)
{
if ($goal->wasChanged()) {
ModelChangedEvent::dispatch(
model: $goal,
changedFields: $goal->getChanges(),
originalValues: $goal->getOriginal()
);
}
}
}
4. Core Jobs
Create foundational jobs that the bus will dispatch:
SupersedeOffersForCampaignJob- Deletes and recreates offers for a campaign across all appsSupersedeOffersForAppJob- Deletes and recreates all offers for an app (with batching)UpdateOffersForCampaignJob- Patches metadata for all offers of a campaignUpdateOffersForAppJob- Patches metadata for all offers of an app
5. Feature Flag Control
Add DevCycle feature flag useRealTimeOfferSync to control rollout:
// DevCycleService.php
public function usesRealTimeOfferSync(int $appId): bool
{
$enabledApps = $this->getVariable('useRealTimeOfferSync', []);
return in_array($appId, $enabledApps);
}
6. MarginOverride Handling
Due to the complex matching logic (publishers, apps, campaigns arrays), MarginOverride changes will use a two-phase approach:
┌─────────────────────────────────────────────────────────────────────┐
│ MarginOverride Discovery Phase │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ MarginOverrideUpdated Discovery Batched Jobs │
│ ┌──────────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Event Fired │ ───> │ Identify all │ ───> │ Supersede │ │
│ │ │ │ affected │ │ in batches │ │
│ │ │ │ offers │ │ │ │
│ └──────────────────┘ │ │ │ Log if >100 │ │
│ │ Log count │ │ offers │ │
│ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
7. Batching & Rate Limiting
For apps with large offer counts (1500+):
// config/offer-api.php
'realtime_sync' => [
'batch_size' => 50, // Offers per batch
'batch_delay_ms' => 500, // Delay between batches
'large_impact_threshold' => 100, // Log warning if exceeds
],
8. Scheduled Sync Retention
The SyncOfferApiV2 command will be retained but unscheduled:
- Available for manual execution via
--forceflag - Add
--dry-runflag to preview sync without executing - Can be re-enabled if real-time sync encounters issues
Phased Rollout Plan
This rollout plan aligns with the three-phase approach in the DECISION section:
Phase A: Minimal Dashboard Fixes (Short-term)
├── GoalObserver for payout changes (margin/revenue)
├── Core sync jobs (SupersedeOffersForCampaignJob)
├── DevCycle feature flag (useRealTimeOfferSync)
├── Retain SyncOfferApiV2 as safety net
└── Gradual rollout: test apps → 10% → 25% → 50% → 100%
Phase B: Offer API Event Infrastructure (Medium-term)
├── Event Bus / event handler in Offer API
├── Domain event schema and contracts
├── Message queue infrastructure (SQS or EventBridge)
├── Dashboard emits domain events for Phase A triggers (proof-of-concept)
├── Offer API consumes events and owns routing/transformation
└── Monitoring & alerting
Phase C: Service Integration (Long-term)
├── AdSuite publishes campaign, goal, targeting events
├── Publisher Portal publishes app, offerwall events
├── Dashboard event publishing decommissioned
└── SyncOfferApiV2 fully retired
The Option 2 implementation details above (OfferSyncEventBus, deduplication, batching, etc.) can inform both Phase A's targeted observer work and Phase B's Offer API event bus design.
CONSEQUENCES
Positive Outcomes
- Reduced Latency: Changes propagate to Offer API within seconds/minutes instead of 10-40 minutes
- Improved Accuracy: Payout and contract changes reflect immediately
- Operational Efficiency: Only changed data is processed, reducing API load
- Better Publisher Experience: Dashboard changes visible quickly
- Audit Trail: Centralized bus enables comprehensive logging of sync decisions
- Learned Architecture: Addresses root cause of previous implementation failure
Negative Outcomes
- Initial Complexity: Event bus requires upfront infrastructure investment
- New Pattern: Team needs to learn the event bus approach
- Testing Overhead: Bus routing logic and each observer require testing
- Potential for Cascading Updates: One change could trigger multiple offer updates
Risks
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Offer API overload from burst of changes | Medium | High | Batching, rate limiting, circuit breaker in bus |
| Missed updates due to routing bugs | Low | High | Comprehensive testing, retain scheduled sync as fallback |
| Large blast radius from MarginOverride changes | Medium | Medium | Discovery phase with logging, configurable thresholds |
| Queue backlog during bulk operations | Medium | Medium | Unique job constraints, batch processing |
| Feature flag misconfiguration | Low | Medium | Default to disabled, gradual rollout |
| Team unfamiliarity with event bus | Low | Low | Documentation, code review, pairing |
| Dashboard deprecation investment risk | Medium | High | Phased approach limits dashboard investment (see DECISION) |
| Access patterns bypassing model events | High | High | Phase A is limited in scope; Phase B uses explicit event publishing |
Dashboard Deprecation Risk
The dashboard is being actively decomposed into AdSuite and Publisher Portal. The phased approach in the DECISION section directly addresses this risk by limiting dashboard investment (Phase A) and directing the real infrastructure work toward the Offer API (Phase B) and the new services (Phase C).
Migration Strategy
Phase A Rollout:
- Deploy goal payout observer and sync jobs to all environments
- Enable feature flag for 1-2 test apps
- Monitor for 1 week, verify offer consistency
- Gradually enable for production apps (10% → 25% → 50% → 100%)
SyncOfferApiV2remains scheduled as safety net throughout Phase A
Phase B Transition:
- Build event infrastructure in Offer API
- Dashboard begins emitting domain events alongside existing observer-based sync
- Validate Offer API event processing produces identical results
- Cut over from dashboard observer sync to Offer API event-driven sync
- Unschedule
SyncOfferApiV2once event-driven sync covers all entity types
Phase C Decommission:
- New services publish events directly; dashboard event publishing removed
SyncOfferApiV2fully retired- Retain manual sync command capability for emergency use
NOTES
References
- Laravel Eloquent Observers
- Laravel Queues
- Laravel Events
- Existing implementation:
app/Observers/CampaignObserver.php - Existing implementation:
app/Observers/AppObserver.php - Offer API contract fields:
config/offer-api.php - PR #95: docs(ADR): Real-time Offer API Syncing
- PR #127: docs: backfill PR reference links for existing ADRs
Related Jira Issues
- AGPI-1517: Real-time Offer API Syncing (Epic)
- AGPI-1516: Discovery & ADR (this document)
Original Author
Micah Wierenga
Approval date
Approved by
APPENDIX
Proposed Story Breakdown
Phase A: Minimal Dashboard Fixes
| Story | Title | Priority | Est. Points |
|---|---|---|---|
| A1 | GoalObserver for payout changes (margin/revenue) with field-level routing | P0 | 5 |
| A2 | SupersedeOffersForCampaignJob (core sync job) | P0 | 5 |
| A3 | DevCycle feature flag for real-time sync | P0 | 2 |
| A4 | Gradual rollout and validation (test apps → production) | P0 | 3 |
Phase A Estimated Points: 15
Phase B: Offer API Event Infrastructure
| Story | Title | Priority | Est. Points |
|---|---|---|---|
| B1 | Domain event schema and contracts definition | P0 | 5 |
| B2 | Message queue infrastructure (SQS/EventBridge) | P0 | 5 |
| B3 | Offer API Event Bus: event handler and routing logic | P0 | 8 |
| B4 | Offer API: field→action mapping and supersede/update logic | P0 | 8 |
| B5 | Dashboard emits domain events for goal payout changes (proof-of-concept) | P1 | 5 |
| B6 | Offer API: deduplication and batching | P1 | 5 |
| B7 | Offer API: MarginOverride event handling | P1 | 8 |
| B8 | Monitoring & alerting | P2 | 5 |
Phase B Estimated Points: 49
Phase C: Service Integration
| Story | Title | Priority | Est. Points |
|---|---|---|---|
| C1 | AdSuite publishes campaign, goal, and targeting domain events | P0 | 8 |
| C2 | Publisher Portal publishes app, offerwall domain events | P0 | 8 |
| C3 | Decommission dashboard event publishing | P1 | 3 |
| C4 | Retire SyncOfferApiV2 | P2 | 2 |
Phase C Estimated Points: 21
Total Estimated Points: 85 (across all phases)
Note: Phase A is designed to be small and deliver immediate business value. Phase B represents the bulk of the architectural investment but is built in the right system (Offer API). Phase C depends on the AdSuite/Publisher Portal timeline and can be scoped as those services come online.
Model Reference
The CampaignOfferwallCreative model contains offer display content. For brevity in code examples, we refer to its observer as CreativeObserver.
Current SyncOfferApiV2 Flow
// app/Console/Commands/SyncOfferApiV2.php
// 1. Query DevCycle for apps at specified frequency
// 2. For each app, dispatch UpdateOfferApiForAppJob
// app/Jobs/OfferApi/UpdateOfferApiForAppJob.php
// 1. Fetch AppCampaigns with all relationships
// 2. Format offers using OfferSyncFormatter
// 3. Pass to OfferApiSynchronizationService->sync()
// app/Libraries/OfferApi/OfferApiSynchronizationService.php
// 1. Fetch existing offers from Offer API
// 2. Compare using OfferSyncHelper->determinePatches()
// 3. CREATE missing, UPDATE changed, DISABLE removed
// 4. Contract field changes trigger DELETE + CREATE (supersession)
Configuration Reference
// config/offer-api.php
return [
'url' => env('OFFER_API_URL'),
'internal_api_key' => env('OFFER_API_INTERNAL_API_KEY'),
'contract_fields' => [
'total_amount',
'total_payout_usd',
'goals',
'is_multi_reward',
'location_targets',
'device_targets',
'os_targets',
],
'jobs' => [
'retry_on_error' => true,
'retry_limit' => 10,
'retry_delay' => 2,
'retry_http_statuses' => [500, 502, 503, 504, 429],
],
// Proposed additions for real-time sync
'realtime_sync' => [
'enabled' => false, // Controlled by DevCycle per-app
'batch_size' => 50,
'batch_delay_ms' => 500,
'large_impact_threshold' => 100,
],
];
Questions for Team Discussion
- Service-Specific Queues vs EventBridge (Option 5 vs 6) for Phase B:
- Do we need the advanced features of EventBridge (archive, replay, schema registry)?
- Is event ordering across services important for our use case?
-
What is our team's familiarity with EventBridge vs SQS?
-
Cross-Service Effects: MarginOverrides (managed in AdSuite) can affect offers for apps managed in Publisher Portal. How should the Offer API handle events that have cross-service implications?
-
Existing Event Cleanup: Should we remove the unused
MarginOverrideCreated/Updated/DeletedandOfferwallCreated/Updatedevents, or repurpose them as the foundation for domain events in Phase B? -
Targeting Profile Scope: When a targeting profile changes, should we supersede offers for ALL campaigns using it (could be large), or add a threshold notification and process in batches?
-
Phase B Timeline: What is the realistic timeline for building the Offer API event infrastructure? Should Phase B begin in parallel with Phase A rollout, or sequentially after Phase A is validated?