Page MenuHomeIssueTracker

Strengthen phorge robots.txt
Open, P3 LowPublic

Description

Current setup as of 2023-12-31:

Possible reinforcement:

  1. Pray to Upstream to implement proper robots.txt management
  2. Host robots.txt file elsewhere and proxy it via cloudflare worker

1 is probably the way to go but too lazy to file a task.

Event Timeline

revi triaged this task as P3 Low priority.Dec 31 2023, 09:28
revi created this task.

Added…

# No ChatGPT
User-agent: ChatGPT-User
Disallow: /
User-agent: GPTBot
Disallow: /