# digible.ai is deliberately kept OUT of search results — every page carries # ``. # # Crawling stays ALLOWED, and that is not a contradiction. `Disallow` blocks the # fetch, so a crawler never sees the noindex tag; a URL discovered from an # external link can then still be indexed, title-only, with no way to remove it. # Allow + noindex is the combination that actually keeps the site unlisted. # # It also protects the reason this site exists: Google's OAuth verification # reviewer and its crawler both fetch the home page and the privacy policy. A # `Disallow` here reads downstream as "your home page does not explain the # purpose of your app" — a rejection reason with no obvious cause. Verification # does not care whether the page is indexed, only whether it can be read. # # No `Sitemap:` line: a sitemap is an invitation to index. User-agent: * Allow: /