由于目前国内外AI爬虫正全网海量爬站以训练数据模型,导致90%的网站每天都面临着超负荷运行,部分用户反映的CPU负载率急速攀升问题,可以通过以下设置方法让你的服务器重获新生,从100%负载状态直降到20%。
如果你是宝塔面板,登录后台-》网站-》编辑-》配置文件,以下是我们在实战中总结归纳出来的有效的屏蔽方式,请将以下规则代码,粘贴到server{中间}


if ($http_user_agent ~* (AwarioBot|Amazonbot|Dalvik|ClaudeBot|GPTBot|meta-externalagent|SemrushBot|DuckDuckBot|TLM-Audit-Scanner|Mediapartners-Google|OAI-SearchBot|Googlebot|SERankingBacklinksBot|GoogleOther|Owler|facebookexternalhit|Go-http-client|Bytespider|Apache-HttpClient|UMichBot|Barkrowler|MauiBot|MTRobot|Python|AhrefsBot|DotBot|FeedDemon|BOT|CrawlDaddy|Java|Jullo|Feedly|UniversalFeedParser|ApacheBench|Swiftbot|WinHttp|WebZIP|FetchURL|Python-urllib|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|EasouSpider|Scrapy|Curl|YisouSpider|PetalBot)) {
    return 403;
}


if ($request_method !~ ^(GET|HEAD|POST)$) {
    return 403;
}


以上规则配置后,建议重启下nginx服务