Team Tag Propagation — Sample PR Plan

Date: 2026-05-12 Author: Ron White Context: Building a per-team operations dashboard (https://app.datadoghq.com/dashboard/ubi-zkn-2c4) surfaced a gap: only team:platform propagates to APM spans, and only on the postgresql service. Service Catalog team association is metadata-only and does not auto-propagate to span tags. All other teams have zero spans carrying a team: tag.

Mechanism

The Datadog PHP tracer (dd-trace-php) accepts a team tag via two mechanisms — pick based on whether a service is single-team or multi-team:

Service shape	Mechanism	Where the change lives
Single-team (one repo = one team)	DD_TAGS=team:<team> env var	Deployment config (CDK / Compose / .env)
Multi-team (one repo serves multiple teams via routes)	Route-group middleware that stamps the team tag on the request's root span	PHP source: middleware + routes file

Both patterns end up at the same place — a team: meta on the root span — so dashboard queries don't need to know which mechanism a given service uses.

team is not part of Datadog's Unified Service Tagging (which covers env, service, version), so it must be added explicitly via one of the patterns above.

Service Triage

Pick the pattern based on the repo:

Service	Owner(s)	Pattern
player-api	player-experience	A — DD_TAGS
offer-api	publisher-integrations	A — DD_TAGS
targeted-api	publisher-integrations	A — DD_TAGS
partners-postback-endpoints	advertiser-integrations	A — DD_TAGS
offer-interceptor	campaign-delivery	A — DD_TAGS
api	mixed (player-experience, campaign-delivery, publisher-integrations)	B — middleware
new_dashboard	mixed	B — middleware

Open the sample PR against one of each shape so the second PR has a clear precedent to follow.

---

Pattern A: Single-Team Service via DD_TAGS

A.1 Confirm the target service uses dd-trace-php

• cat composer.json | rg 'datadog|laravel' — expect datadog/dd-trace and a Laravel framework dep.
• If the tracer isn't installed, this plan doesn't apply — add the package first via a separate PR.

A.2 Locate where Datadog env vars are configured

rg -l 'DD_SERVICE|DD_ENV|DD_AGENT_HOST' --type-add 'iac:*.{yml,yaml,ts,tf,json,env}'

Expected locations, and when each applies:

Location	Use when
provision/ — CDK TypeScript stack, ECS task definition environment: { ... }	The service deploys via CDK/ECS (most of AdAction's stack). Default home for the tag.
Datadog Secret in CDK (see note below)	The service already sources its other Datadog config from a CDK-created Secret. Add team as a new field there so it flows through the existing path.
docker-compose.yml / compose.yml	Local or Compose-based deploys where env vars are declared inline.
.env.example / .env.production	Services that read DD_TAGS from a dotenv file rather than the orchestrator.
Dockerfile ENV directives	Last resort — bakes the tag into the image so it can't vary by environment. Avoid unless nothing else applies.

If env vars live in more than one place, add DD_TAGS at the most specific layer so the diff is self-contained.

AdAction CDK convention (preferred where it fits): for services whose Datadog config is provisioned through a CDK-created Secret, add team as a new field on that Secret and read it in configure-datadog-service-and-agent.sh the same way the other Secret fields are pulled. That keeps the tag alongside the rest of the Datadog wiring instead of scattering a separate DD_TAGS literal across task definitions. (Per Dakota's review note — confirm the exact Secret and field names against the target repo.)

A.3 Add the team tag

CDK / ECS task definition (likely):

environment: {
  DD_SERVICE: 'player-api',
  DD_ENV: 'prod',
  DD_VERSION: process.env.GIT_SHA ?? 'dev',
  DD_TAGS: 'team:player-experience',  // ← new
  // ...rest unchanged
}

Docker Compose:

environment:
  - DD_SERVICE=player-api
  - DD_ENV=prod
  - DD_TAGS=team:player-experience  # ← new

Multiple tags (comma-separated):

DD_TAGS=team:player-experience,cost_center:engineering

---

Pattern B: Multi-Team Service via Middleware

For api and new_dashboard — different controllers belong to different teams.

B.1 Create the middleware

<?php
// app/Http/Middleware/TagTeam.php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;

class TagTeam
{
    public function handle(Request $request, Closure $next, string $team)
    {
        if (function_exists('\\DDTrace\\root_span')) {
            $span = \DDTrace\root_span();
            if ($span !== null) {
                $span->meta['team'] = $team;
            }
        }

        return $next($request);
    }
}

The function_exists guard means the middleware is a no-op when dd-trace-php isn't loaded (local dev without the extension, for example).

B.2 Register the middleware alias

// app/Http/Kernel.php
protected $routeMiddleware = [
    // ...existing...
    'team' => \App\Http\Middleware\TagTeam::class,
];

(In Laravel 10+ it's $middlewareAliases; in Laravel 11+ it's registered in bootstrap/app.php via ->withMiddleware(...). Match whatever the repo already uses.)

B.3 Group routes by team owner

// routes/api.php

Route::middleware('team:player-experience')->group(function () {
    Route::get('/offers', [OfferController::class, 'index']);
    Route::post('/offers/{id}/dismiss', [OfferController::class, 'dismiss']);
});

Route::middleware('team:campaign-delivery')->group(function () {
    Route::resource('campaigns', CampaignController::class);
});

Route::middleware('team:publisher-integrations')->group(function () {
    Route::get('/publishers/{id}/payouts', [PayoutController::class, 'show']);
});

Mixed case (recommended belt-and-suspenders): set DD_TAGS=team:untagged in deployment config as a sentinel fallback for routes not yet wrapped in a team: group. The middleware overrides it on the root span when invoked. Routes that miss a group surface as team:untagged — an explicit, alertable gap — rather than being silently attributed to a real team. (See the ADR's "Fallback for unwrapped routes" for the trade-off vs. a real default team.)

B.4 Optional — CI lint to prevent unowned routes

A small script that loads the routes, walks each one, and fails if any route lacks the team: middleware alias. Worth adding once the initial migration lands.

---

Decision Tree

Is the service repo entirely one team's code?
├── Yes → Pattern A (DD_TAGS in deployment config)
└── No → Pattern B (middleware on route groups)
            └── Set DD_TAGS=team:untagged as sentinel fallback for unowned routes

---

Verification (Both Patterns)

After deploy, from a Claude session with the Datadog MCP loaded:

aggregate_spans
  query: "service:<your-service> team:<your-team>"
  from: "now-30m"
  computes: [{ field: "*", aggregation: "COUNT", output: "count" }]

Acceptance: count > 0.

For Pattern B, also confirm a route from each team group separately:

aggregate_spans
  query: "service:api team:campaign-delivery"
  ...

Reload the Team Overview dashboard with the team selected — APM Top Endpoints should populate.

---

Opening the PR

Use the /pr skill (Ron's convention — adds a gif).

• Title (Conventional Commits):
  • Pattern A: feat(observability): tag <service> spans with team:<team>
  • Pattern B: feat(observability): add team-tagging middleware for <repo> routes
• Body:
  • One-sentence context: Team Overview dashboard surfaced the team-tag gap.
  • The change: env var (A) or middleware + route groups (B).
  • Verification: paste the aggregate_spans result showing tagged spans.
  • Link to the dashboard.
• Reviewer: the team lead per ~/.claude/CLAUDE.md team table.

---

Acceptance Criteria

• Sample PR opened for one Pattern A service AND one Pattern B service
• Pattern A: single env var addition; no PHP source modified
• Pattern B: middleware + Kernel.php + at least one route group wired up
• After deploy, aggregate_spans returns non-zero count for service:X team:Y
• Team Overview dashboard APM section shows endpoints when filtered to that team

---

Rollout Template

Once the two sample PRs merge:

• Single-team services: one PR per service (or one per team if services share deployment config)

• Multi-team services: middleware lands once; subsequent PRs incrementally wrap route groups

• Weekly progress check:

  aggregate_spans
    query: "operation_name:<entry-span-op>"   # scope to entry spans — see note
    group_by: { fields: ["team"], limit: 20 }
    computes: [{ field: "*", aggregation: "COUNT", output: "count" }]

  Goal: all 6 teams appear in the result, and total team:untagged spans trend down.

  Scope the check to entry spans (confirm the exact operation name against your spans, typically laravel.request or web.request). Pattern B tags only the root span, so a raw query: "*" drops every child DB/HTTP span into a null-team bucket and drowns the signal. Entry-span scoping counts requests, which is what "is this team represented" actually means. See the ADR's "Why root-span tagging is sufficient."

---

Alternatives Considered

Option	Why we didn't pick it
PHP 8 #[Team('x')] attribute on controllers	Nicer for greenfield code; more boilerplate to retrofit existing controllers. Route-group middleware is the smallest reviewable diff for existing repos.
__construct sets tag in every controller	Easy to forget on new controllers, clutters every class.
Datadog Agent tag-processing rules (regex on resource_name)	Zero PHP change but brittle, hard to keep in sync with route refactors, no compile-time check.
Split into separate DD_SERVICE values per team	Heavier refactor; covered separately as a future direction — gives each team its own first-class APM service entry, SLOs, deployment tracking. Worth revisiting once basic team tagging is in place.

Out of Scope

• Multi-tenant services where team varies per request payload (rare — would need runtime tagging keyed on request data)
• Non-PHP services (Node/Python tracers also respect DD_TAGS, but env-var plumbing may differ)
• Backfilling team metadata into Service Catalog (already populated — that's how Datadog knows the team list)