Pi Coding Agent AI Observability installation

Contents

  1. Prerequisites

    Required

    Pi is an open-source coding agent that runs in your terminal. The @posthog/pi extension captures LLM generations, tool executions, and conversation traces as $ai_generation, $ai_span, and $ai_trace events and sends them to PostHog.

    You need:

    • Pi coding agent installed. The extension requires Node.js 22 or later.
    • A PostHog account with a project token.

  2. Install the extension

    Required

    Install the PostHog extension globally:

    pi install npm:@posthog/pi

    Or install it for the current project:

    pi install -l npm:@posthog/pi

  3. Configure PostHog

    Required

    Set environment variables with your PostHog project token and host. You can find these in your PostHog project settings.

    export POSTHOG_API_KEY="<ph_project_token>"
    export POSTHOG_HOST="https://us.i.posthog.com"

    Then start Pi as normal:

    pi

    The extension initializes and captures events for each LLM call, tool execution, and completed agent run.

    Tip: Add these variables to your shell profile, such as ~/.zshrc or ~/.bashrc, so they persist across sessions.

  4. Configuration options

    Optional

    Configure the extension with environment variables or a ~/.pi/agent/posthog.json config file. Environment variables take precedence over config file values.

    VariableDefaultDescription
    POSTHOG_API_KEY(required)Your PostHog project token
    POSTHOG_HOSThttps://us.i.posthog.comPostHog ingestion host
    POSTHOG_PRIVACY_MODEfalseWhen true, LLM input and output content is not sent to PostHog. Token counts, costs, latency, and model metadata are still captured.
    POSTHOG_ENABLEDtrueSet to false to disable the extension
    POSTHOG_TRACE_GROUPINGmessagemessage: one trace per user prompt. session: group all generations in a session into one trace.
    POSTHOG_SESSION_WINDOW_MINUTES60Minutes of inactivity before starting a new session window
    POSTHOG_PROJECT_NAMECurrent directory nameProject name included in all events
    POSTHOG_AGENT_NAMEAgent nameAgent name. Defaults to the project name and detects subagent names when available.
    POSTHOG_TAGS(none)Custom tags added to all events in key1:val1,key2:val2 format
    POSTHOG_MAX_ATTRIBUTE_LENGTH12000Maximum length for serialized tool input and output attributes

    Trace grouping modes

    • message (default): Each user prompt creates a new trace. Multiple LLM turns within one prompt, including tool-use loops, are grouped under the same trace.
    • session: All generations within a session window are grouped into one trace. A new trace starts after POSTHOG_SESSION_WINDOW_MINUTES of inactivity.

    Privacy mode

    When POSTHOG_PRIVACY_MODE=true, all LLM input and output content, user prompts, tool inputs, and tool outputs are redacted. Token counts, costs, latency, and model metadata are still captured.

    Even with privacy mode off, sensitive keys in tool inputs and outputs, such as api_key, token, secret, password, and authorization, are redacted.

    What gets captured

    The extension captures three types of events:

    • $ai_generation: Each LLM call, including model, provider, token usage, cost, latency, and input and output messages in OpenAI chat format.
    • $ai_span: Each tool execution (read, write, edit, bash, and others), including tool name, input parameters, output result, and duration. Learn more about spans.
    • $ai_trace: Completed agent runs with aggregated token totals and latency. Learn more about traces.

  5. Verify traces and generations

    Recommended
    Confirm LLM events are being sent to PostHog

    Let's make sure LLM events are being captured and sent to PostHog. Under AI Observability, you should see rows of data appear in the Traces and Generations tabs.


    LLM generations in PostHog
    Check for LLM events in PostHog

  6. Next steps

    Recommended

    Now that you're capturing AI conversations, continue with the resources below to learn what else AI Observability enables within the PostHog platform.

    ResourceDescription
    BasicsLearn the basics of how LLM calls become events in PostHog.
    GenerationsRead about the $ai_generation event and its properties.
    TracesExplore the trace hierarchy and how to use it to debug LLM calls.
    SpansReview spans and their role in representing individual operations.
    Anaylze LLM performanceLearn how to create dashboards to analyze LLM performance.

Was this page useful?