AI Paid Media Agent: A Human-in-the-Loop LangChain Design

An AI paid media agent should not receive an account token and start changing campaigns. The useful design is less dramatic: collect data, validate it, explain performance changes, rank possible actions, and wait for approval. That workflow gives media teams speed without hiding decisions inside an autonomous loop. This guide maps that approach to LangChain components and provides a Python structure you can adapt to paid search or paid social accounts.

What an AI Paid Media Agent Should—and Shouldn’t—Automate

Paid media involves two different classes of work.

The first is analytical. Your agent can fetch reports, normalize fields, calculate performance metrics, detect anomalies, and draft recommendations. These operations are inspectable. You can rerun them against the same data.

The second changes an external system. Examples include:

• Updating campaign budgets
• Changing bids
• Pausing ad groups
• Modifying targeting
• Removing placements
• Replacing creative
• Changing campaign status

These actions affect spend and delivery. They need stronger controls than report generation.

Warmly distinguishes independent agents from copilots. A copilot proposes actions while leaving the final decision to the user. That is the safer default for a paid media workflow.

Autonomy is only one design criterion. Warmly also recommends evaluating operational fit, learning and customization, integrations, reporting, scalability, and cost. Those criteria translate into concrete engineering questions:

• Can the agent connect to every required advertising platform?
• Does it preserve the source platform’s attribution context?
• Can an operator inspect the evidence behind a recommendation?
• Can you limit write permissions by account and action type?
• What happens when an API request fails halfway through a run?
• Can you identify the prompt, model, data snapshot, and policy version used?
• Is each execution cheap enough to run on your intended schedule?

Do not collapse collection, reasoning, and execution into one tool-enabled prompt. Treat them as separate workflow stages with typed inputs and outputs. This follows the same principle used in production AI agent workflow automation: each stage should have a clear contract and failure mode.

Step 1: Collect, Normalize, and Validate Cross-Platform Ad Data

Meta, TikTok, Google Ads, and other sources expose different names and definitions. Your agent needs a common internal schema before it compares campaigns.

A practical record can include:

from dataclasses import dataclass
from datetime import datetime
from decimal import Decimal
from typing import Optional


@dataclass(frozen=True)
class CampaignMetric:
    platform: str
    account_id: str
    campaign_id: str
    campaign_name: str
    status: str
    currency: str
    window_start: datetime
    window_end: datetime
    attribution_window: str
    spend: Decimal
    impressions: int
    clicks: int
    conversions: Optional[Decimal]
    conversion_value: Optional[Decimal]
    source_updated_at: datetime

    @property
    def cpa(self) -> Optional[Decimal]:
        if not self.conversions:
            return None
        return self.spend / self.conversions

    @property
    def roas(self) -> Optional[Decimal]:
        if not self.spend or self.conversion_value is None:
            return None
        return self.conversion_value / self.spend

The schema alone does not make the data comparable. Preserve the information needed to detect invalid comparisons:

• Attribution window: Do not merge conversions reported under different windows without labeling them.
• Timestamps: Store the reporting window and the source update time.
• Currency: Convert through an explicit, recorded process or compare only matching currencies.
• Missing values: Keep missing data as missing. Do not silently turn an unavailable conversion count into zero.
• Duplicates: Use a stable key based on platform, account, campaign, and reporting window.
• Status: Separate active, paused, archived, and removed entities.
• Platform definitions: Retain source metadata where clicks, conversions, or value have channel-specific meanings.

Validation should happen before any LLM call. Deterministic code is better at rejecting malformed records.

def validate_metric(metric: CampaignMetric) -> list[str]:
    errors: list[str] = []

    if metric.window_end <= metric.window_start:
        errors.append("reporting window is invalid")
    if metric.spend < 0:
        errors.append("spend cannot be negative")
    if metric.impressions < 0 or metric.clicks < 0:
        errors.append("delivery metrics cannot be negative")
    if metric.source_updated_at < metric.window_end:
        errors.append("source may not include the full reporting window")
    if not metric.attribution_window:
        errors.append("attribution window is missing")

    return errors

A collection run should produce both normalized records and a validation report. If a source is stale or incomplete, the next stage must see that warning. It should not infer that a campaign declined when the actual problem is a partial report.

Step 2: Diagnose Campaign Performance with Explicit Evidence

Keep calculations outside the language model. Compute CPA, ROAS, spend deltas, pacing, and comparison-period changes in Python or SQL. Give the model validated results plus their provenance.

The LLM can then help classify and explain issues such as:

• Unexpected performance changes
• Budget pacing outside your policy
• Possible creative fatigue
• Potential audience overlap
• Tracking gaps
• Conflicting actions across channels

These should remain hypotheses unless the available data proves them. For example, falling click-through performance may support a creative-fatigue hypothesis. It does not prove the creative caused the change. The agent should request creative history or placement-level evidence when needed.

Every diagnosis should contain an evidence block:

{
  "diagnosis": "Possible tracking gap",
  "campaign_ids": ["campaign-example"],
  "evidence": [
    {
      "metric": "conversions",
      "current_period": "missing",
      "comparison_period": "available",
      "source": "platform-report"
    }
  ],
  "data_quality_warnings": [
    "Current conversion data is incomplete"
  ],
  "status": "needs_review"
}

The comparison period must be explicit. So must the source and freshness of each metric. This supports the visibility Warmly recommends across real-time metrics, engagement, attribution, and return on investment.

Do not ask the model, “What is wrong with this account?” Ask bounded questions:

• Which validated rule triggered?
• What evidence supports the diagnosis?
• What evidence contradicts it?
• Which required fields are unavailable?
• Can the issue be resolved analytically, or does it need human review?

This makes the response easier to evaluate and reduces unsupported narratives.

Step 3: Generate Ranked Recommendations Instead of Direct Changes

A diagnosis describes what may be happening. A recommendation describes a proposed response. Keep those objects separate.

Use a structured recommendation schema:

from typing import Literal
from pydantic import BaseModel


class Recommendation(BaseModel):
    action: Literal[
        "adjust_budget",
        "adjust_bid",
        "change_targeting",
        "change_placement",
        "review_creative",
        "no_action",
    ]
    rationale: str
    expected_impact: str
    confidence: Literal["low", "medium", "high"]
    affected_campaign_ids: list[str]
    evidence_refs: list[str]
    risk: str
    rollback_plan: str

Rank recommendations by evidence quality, risk, and policy compatibility. Do not rank solely by the model’s confidence label.

Run deterministic policy checks after generation. A policy layer can reject a recommendation when:

• It references an unknown campaign.
• Its evidence belongs to another reporting window.
• Required metrics are missing.
• It exceeds the operator’s permitted action scope.
• It lacks a rollback plan.
• It conflicts with another accepted recommendation.
• It proposes an action on stale data.
• The requested platform tool does not support the action safely.

Budget, bid, targeting, placement, and creative recommendations need different validation rules. A creative review might create a ticket. A budget update would require an exact current value, proposed value, currency, account, and rollback value.

The result remains a copilot workflow. The agent prepares a decision packet. It does not treat natural-language output as authorization to spend money.

Step 4: Add Human Approval, Audit Trails, and Reversible Execution

Represent approval as workflow state, not as a message buried in a chat transcript.

Useful states include:

• pending
• approved
• rejected
• edited
• deferred
• expired
• executing
• succeeded
• failed
• rolled_back

Require approval for material changes to budgets, bids, targeting, or creative. Your policy can also route ambiguous diagnoses to review without generating an executable action.

For each decision, record:

• The normalized input snapshot
• Validation results
• The diagnosis and supporting evidence
• The original recommendation
• Human edits
• The approver identity and timestamp
• The policy and prompt versions
• The outbound API request
• The platform response
• The post-change verification
• The rollback method and result

Credentials should follow least privilege. Collection tools need read access. Execution tools should receive only the write permissions they require. Keep platform secrets outside prompts and persisted workflow state. The production agent security guide covers secrets injection, IAM isolation, and least-privilege access in more depth.

Execution also needs an idempotency key. A retry must not apply the same approved change again. Before writing, fetch the current platform state and verify that it still matches the approved snapshot. If another operator changed the campaign, stop and request fresh approval.

A Reproducible LangChain Architecture and Evaluation Plan

Map the workflow to small LangChain-compatible components:

• Platform tools: Read reports and submit narrowly scoped changes.
• Validation tools: Check schema, freshness, duplicates, and comparability.
• Structured outputs: Produce typed diagnoses and recommendations.
• Workflow state: Store snapshots, warnings, approvals, and execution results.
• Checkpoints: Persist state after collection, recommendation, and approval.
• Orchestration: Route failures and human decisions without restarting the whole run.

The orchestration loop can remain simple:

def run_paid_media_workflow(account_refs, services):
    state = services.collect(account_refs)
    state = services.validate(state)

    if state.has_blocking_errors:
        return services.finish(state, status="needs_data_review")

    state = services.diagnose(state)
    state = services.recommend(state)
    state = services.apply_policy(state)
    state = services.request_approval(state)

    if state.approval_status != "approved":
        return services.finish(state, status=state.approval_status)

    state = services.verify_preconditions(state)
    state = services.execute(state)
    state = services.verify_result(state)

    return services.finish(state, status="completed")

In LangChain, the diagnosis and recommendation stages can use prompt templates plus structured parsers. Standard Python functions should handle validation and policies. If the workflow needs durable pauses for approval, persistent checkpoints become part of the runtime design rather than an optional logging feature.

Evaluate the system against a versioned reference dataset. Include normal campaigns, missing attribution data, stale reports, duplicated records, conflicting channel recommendations, rejected actions, API failures, and changed preconditions.

Track:

• Diagnostic precision
• Recommendation acceptance
• Business impact after approved changes
• Error rate
• End-to-end latency
• Model and infrastructure cost
• Operator hours saved

Acceptance is not proof that a recommendation was correct. Business impact is not enough either, because campaign outcomes can have multiple causes. Review trace evidence, policy results, human edits, and execution outcomes together. Use a reproducible agent evaluation process, then connect it to logging, tracing, and production monitoring.

An AI paid media agent works best as a controlled decision pipeline. Normalize the data first. Separate calculations from interpretation. Require evidence for diagnoses. Generate typed, reversible recommendations. Put a human gate before consequential changes.

Once those boundaries are explicit, the workflow can run as a scheduled production job without becoming an opaque campaign operator. A managed agent runtime can then handle scheduling, isolated secrets, checkpoints, and logs while your application keeps control of the decisions that matter.