垃圾蜘蛛太多了 有啥要补充的 robots.txt
2023/06/07 14:41:28
- User-agent: SemrushBotDisallow: /User-agent: DotBotDisallow: /User-agent: MegaIndex.ruDisallow: /User-agent: MauiBotDisallow: /User-agent: AhrefsBotDisallow: /User-agent: MJ12botDisallow: /User-agent: BLEXBotDisallow: /