Salience

Salience is server log intelligence for search engines and AI crawlers. It turns the server and CDN data a website already generates into a live view of every request, response and issue coming from people, search engines and AI bots. A user connects a source they already run — Cloudflare, Vercel, CloudFront, Netlify, Kinsta, Shopify, Apache, Nginx or raw log files — and Salience starts reading, verifying and classifying requests within seconds. The dashboard shows which pages Googlebot, GPTBot, ClaudeBot and other recognised crawler identities requested, the exact response the infrastructure served, and where crawl budget was wasted. Salience is built for search and organic growth teams, AI and content teams, security and platform teams, and engineering and commerce operations. The problem Salience addresses is a visibility gap between what analytics reports and what actually reaches a site. Google Analytics measures users, sessions and events collected by its tag and filters known bots out by design, so crawler behaviour never appears in it. Search Console answers some questions but not in real time: as one quoted scenario puts it, sitemap changes deployed yesterday may go unseen for 48 hours. Meanwhile raw access logs contain the answers but are difficult to read. Four situations recur in the content: not knowing whether Googlebot has seen sitemap changes, not knowing which AI systems read your content or whether they are genuine, discovering that traffic claiming to be Googlebot is not and that the real crawler is being rate-limited by a WAF, and shipping a URL change only to have Googlebot hit thousands of 404s before anyone notices. Salience exists to make those situations visible immediately. Two of the four capabilities in the Salience engine are Observe and Verify. Observe means seeing what is actually reaching the server: Salience analyses requests arriving through a CDN or web server, using server-side data only, with no browser script and no sampling by Salience. Every request that reaches the source is ingested, humans and bots alike, and each is classified as human or bot with more than 200 crawler identities recognised. Verify addresses impersonation: supported search, AI and automated identities are checked against official provider IP ranges wherever verification data is available, so legitimate GPTBot, ClaudeBot or PerplexityBot traffic can be separated from scrapers that merely reuse the User-Agent string. Verification status, per-bot crawl frequency and top URLs live on the Bots and Crawlers page. The other two capabilities are Explain and Act. Explain connects each request to the path, status code, site section, historical pattern, sitemap, robots.txt rule or index state, so a change in behaviour has a cause rather than just a number. Act moves from evidence to action: the affected bots, IPs and URLs are shown alongside the likely issue and a recommended next step. This surfaces through alerting that names the crawler, the URLs and the next step instead of saying that something happened. Example alerts in the content include 15,000 404s on /products/* from Googlebot since a 14:00 deploy, with a suggested next step to roll back or fix the URL structure, and ClaudeBot refused with 403 on 32 URLs under /collections since a WAF rule change, with a suggested next step to review the rule. Alerts can be marked reviewed, snoozed or dismissed as expected, and every alert checks that the site is reachable first. AI crawler activity gets its own view. Requests are grouped into training and other crawlers, AI search crawler requests and user-triggered AI fetches, with totals, share of traffic, unique crawlers and pages crawled. The AI Crawlers panel lists each bot with its category, verification status and unique page counts, and can be exported to CSV. Salience states clearly that recorded requests are classified by agent and do not prove training, indexing, answers or citations. For day-to-day work the dashboard gives a live view per site with shared read-only links for clients and colleagues, alerts arrive by email, Slack and webhooks, and reports provide a weekly summary per site plus CSV exports of the record behind it. A REST API with a personal API key is available on every plan including Free, a CLI lets teams query the same record from a terminal or script, and an MCP server connects Claude, Cursor or ChatGPT to the crawler data so users can ask questions of it. The overall method is a pipeline from raw request to recommended action. Signals feed the engine from search crawlers such as Googlebot and Bingbot, AI and LLM bots such as GPTBot, ClaudeBot and PerplexityBot, platform signals from Google Search Console and Google Analytics, and the primary source, CDN and web server logs. Requests move through raw, normalised, enriched and understood stages before producing Observe, Verify, Explain and Act. Setup follows three steps: connect what you already use, after which Salience starts reading requests and makes them available in the dashboard within seconds, and then act on problems that become obvious. Nothing is added to the page, there is no frontend change, and CDN sources need no agent on the origin. The stated benefits are complete visibility of real requests from people, search engines and AI agents using existing infrastructure; finding errors, blocked crawlers, missing pages and suspicious traffic early without manually digging through logs; understanding which content gets crawled, how the site responds and where problems need attention; and setup in minutes. Salience also runs 16 detectors checked every 5 minutes. On data protection, Salience states that it processes data only to provide the service, with the customer remaining the controller; core data is hosted in AWS London (eu-west-2) and encrypted at rest and in transit; collection is server-side with no client-side pixel; and logs are not sold, used for advertising or shared between customers, with a DPA available on request. Concrete workflows described in the content map to specific teams. Search and organic growth teams use it to check whether Googlebot has seen sitemap changes without waiting for Search Console. AI and content teams use it to see which AI systems read their content, whether the traffic is training or retrieval, and whether it is genuine. Security and platform teams use it to separate spoofed Googlebot traffic from the real crawler and to catch the real one being rate-limited by a WAF. Engineering and commerce operations use it to spot a URL change that produced thousands of 404s. It is also positioned for e-commerce intelligence and publisher intelligence. Historical imports support retrospective investigation: Apache, Nginx or CloudFront logs, or an export from another log analyser, can be dragged in, are auto-detected, deduplicated against real-time data and merged into the dashboard without counting against the monthly request quota. Setup is guided in about ten minutes or less for Cloudflare, Vercel and Kinsta; Apache and Nginx need someone with root or sudo for roughly ten minutes, using a read-only agent that reads the log file and sends it over HTTPS without sitting in the request path. Where a Worker collects logs, on Cloudflare and Shopify, it adds a small amount of edge compute. CloudFront real-time logs arrive at the sampling rate the customer configures. Plans scale with traffic: Free at $0 covers up to 12K visitors a month, one website, 30 days of history and two email alerts; Solo at $19/mo covers up to 125K visitors with six months of history and all 16 alerts by email and Slack plus sitemap and Search Console; Starter at $49/mo covers up to 500K visitors, five websites, one year of history, webhooks and shared dashboard links; Growth at $149/mo covers up to 2.5M visitors, 15 websites and two years of history with priority support; Pro at $499/mo covers up to 12M visitors, 50 websites, four years of history and dedicated support with a shared Slack channel; Enterprise is custom with unlimited websites, SSO/SAML, audit log, uptime SLA, DPA and custom MSA. Salience does not replace a WAF, CDN or observability tool; it reads the access logs those systems already produce and answers a narrower question well: what did search engines and AI crawlers request, what did the infrastructure return, and were they genuine. The core value is evidence from your own logs, turned into a live view and a clear next step.