2020
10-13
10-13
nginx 防盗链防爬虫配置详解
新建配置配置文件(例如进入到nginx安装目录下的conf目录,创建:agent_deny.conf)禁止Scrapy等工具的抓取if($http_user_agent~*(Scrapy|Curl|HttpClient)){return403;}禁止指定UA及UA为空的访问#forbiddenScrapyif($http_user_agent~*(Scrapy|Curl|HttpClient)){return403;}#forbiddenUAif($http_user_agent~"Bytespider|FeedDemon|JikeSpider|IndyLibrary|AlexaToolbar|AskTbFXTV|AhrefsBot|Crawl...
继续阅读 >