日志中有这个记录，爬虫是禁止没
2025/05/17 16:07:46
173.252.83.18 - - [17/May/2025:07:53:49 +0800] ＂GET /robots.txt HTTP/2.0＂ 444 0 ＂-＂ ＂facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)＂173.252.83.7 - - [17/May/2025:07:53:51 +0800] ＂GET /robots.txt HTTP/2.0＂ 444 0 ＂-＂ ＂facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)＂173.252.83.7 - - [17/May/2025:07:53:54 +0800] ＂GET /robots.txt HTTP/2.0＂ 444 0 ＂-＂ ＂facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)＂