Report AI Crawler Visits

Tell Pendium when an AI crawler fetched a page on a site you render yourself, so the brand's AI visit counts include it.

POST /api/analytics/hit

Authentication: Required (signature with the brand's signing secret)

AI crawlers such as GPTBot and ClaudeBot don't run JavaScript, so the page tag never sees them. If you render the brand's site yourself, report those requests from your server instead. Each report counts toward the brand's AI visits in Site Analytics, the same as a visit to a site Pendium hosts.

Send reports in the background. Don't hold up the page you're serving while one is in flight, and don't retry a failed one.

Credentials

Both values are on the brand's Site Analytics page, under Running your own front end?

Value	What it's for
Site key (`ps_…`)	Names the brand. It's public: it's also in the page tag.
Signing secret (`phs_…`)	Signs each report. Keep it on your server. It can only report visits for this brand's site key.

The site key is also in the Site Export response as site.analytics.siteKey.

Request

Send JSON with a signature header.

Header	Value
`Content-Type`	`application/json`
`X-Pendium-Signature`	`t=<unix seconds>,v1=<signature>`, where the signature is the hex HMAC-SHA256 of `"<t>.<raw body>"` with the signing secret as the key

The timestamp must be within five minutes of Pendium's clock.

Field	Type	Description
`siteKey`	string	Required. The brand's site key.
`host`	string	Required. The host the request came in on, such as `www.acme.com`.
`path`	string	Required. The page path, starting with `/`.
`userAgent`	string	Required. The crawler's `User-Agent` header, unchanged.
`ts`	string	Required. When the request arrived, as an ISO 8601 timestamp.

Only report GET requests for pages, not assets or API calls, and only when the user agent looks like an AI crawler. Pendium classifies each one again against its own list, so a short allowlist is enough: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended.

Responses

Status	Meaning
`202`	Accepted. A report for a host the brand isn't served on is also answered `202` and not counted.
`401`	Missing, stale, or wrong signature.
`404`	The site key isn't valid.
`413`	The body is larger than 8 KB.
`422`	The body isn't valid JSON or a field is missing or malformed.
`429`	Too many reports for this brand right now.

A report counts when its host is one the brand is served on: its verified custom domain, its verified subpath host, or the website on its company profile (with or without www). Sending the same report twice counts it once.

Example: Next.js middleware

// middleware.ts
import { NextResponse, type NextRequest } from "next/server";
 
const AI_CRAWLERS =
  /GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|PerplexityBot|Perplexity-User|Google-Extended/i;
 
async function sign(secret: string, body: string, t: number) {
  const key = await crypto.subtle.importKey(
    "raw",
    new TextEncoder().encode(secret),
    { name: "HMAC", hash: "SHA-256" },
    false,
    ["sign"]
  );
  const mac = await crypto.subtle.sign(
    "HMAC",
    key,
    new TextEncoder().encode(`${t}.${body}`)
  );
  return [...new Uint8Array(mac)]
    .map((b) => b.toString(16).padStart(2, "0"))
    .join("");
}
 
export function middleware(request: NextRequest, event: { waitUntil(p: Promise<unknown>): void }) {
  const userAgent = request.headers.get("user-agent") ?? "";
  if (request.method === "GET" && AI_CRAWLERS.test(userAgent)) {
    const body = JSON.stringify({
      siteKey: process.env.PENDIUM_SITE_KEY,
      host: request.nextUrl.hostname,
      path: request.nextUrl.pathname,
      userAgent,
      ts: new Date().toISOString(),
    });
    const t = Math.floor(Date.now() / 1000);
    event.waitUntil(
      sign(process.env.PENDIUM_SIGNING_SECRET!, body, t).then((v1) =>
        fetch("https://pendium.ai/api/analytics/hit", {
          method: "POST",
          headers: {
            "Content-Type": "application/json",
            "X-Pendium-Signature": `t=${t},v1=${v1}`,
          },
          body,
        }).catch(() => {})
      )
    );
  }
  return NextResponse.next();
}
 
// Pages only: skip Next.js assets, API routes and files with an extension.
export const config = {
  matcher: ["/((?!_next/|api/|.*\\.[a-z0-9]+$).*)"],
};

Set PENDIUM_SITE_KEY and PENDIUM_SIGNING_SECRET in your hosting provider's environment settings.

Credentials

Request

Responses

Example: Next.js middleware

On this page