HN Showcase
Back to browse
AgentCheck – AI bot posture leaderboard from robots.txt and llms.txt

AgentCheck – AI bot posture leaderboard from robots.txt and llms.txt

by MK_Phoenix·Feb 20, 2026·1 point·1 comment
Visit ProjectView on HN

AI Analysis

●MidBold Bet

Transparent AI bot posture tracking, but 'what bots touch you' is already solved by uBlock Origin.

Strengths
  • •Clever use of robots.txt and llms.txt standards as public signals—no proprietary crawling required.
  • •Weekly delta tracking surfaces policy changes, useful for compliance researchers.
  • •Device monitor (planned) addresses a real gap beyond domain-level audits.
Weaknesses
  • •Conflates declared policies (robots.txt) with actual behavior—no verification that sites honor their own rules.
  • •uBlock Origin, Ghostery, and Sequoia browser extensions already show live trackers; this adds metadata layers but not fundamentally new visibility.
Category
Security
Target Audience

Privacy-conscious users, website operators, AI bot policy researchers

Similar To

uBlock Origin · Ghostery · BuiltWith

Post Description

Hi HN — I built AgentCheck to make the agentic internet more legible.

AgentCheck is a public “AI bot posture leaderboard” built from declared public signals: - robots.txt allow/deny rules - public capability/interface files (e.g. /llms.txt and /.well-known/agents.json where present) - weekly deltas so you can see policy changes over time

It answers: which bots a site declares it blocks/allows (using a fixed reference bot set), whether agent-readable interface files exist, and how posture changes week to week.

Important: this is not a claim about actual crawling activity — it’s posture + public interface signals.

Link: https://www.agentcheck.com/leaderboard/ai-bots

I’d love feedback on: - other public signals worth adding - how you’d define “agent readiness” - edge cases where robots.txt parsing should be handled differently

Similar Projects