Plugin Tag: crawler
-
-
Block Common Crawl via robots.txt
(Viso įvertinimų: 0)Blocks the Common Crawl bot (CCBot) that gathers data that is used to train AI algorithms like ChatGPT through the WordPress virtual robots.txt file
-
AI Deny
(Viso įvertinimų: 0)Easily block over 30 of the most common AI user-agents known for crawling websites, keeping unauthorized bots at bay.
-
Robots.txt Extender
(Viso įvertinimų: 0)Dynamic robots.txt for Multisite! Change parameters, or don't, for each site of your network without losing the defaults from WordPress Includes.
-
rexCrawler
(Viso įvertinimų: 0)rexCrawler is a regular expression web-crawler plugin. Able to parse websites, save data and show them on pages.
-
JaspreetChahal’s wordpress bot detector lite
(Viso įvertinimų: 1)This plugin detects a few bots (e.g. Google, Google ads, Alta vista etc) and sends an email based on interval set by you. you can choose what format e …
-
Page Modified
(Viso įvertinimų: 1)Display the results of your Page Modified crawls right within your WordPress Admin.
-
SEObot Monitor for Googlebot, Bingbot and search engine spiders
(Viso įvertinimų: 0)With SEObot Monitor for Googlebot you will be able to dump the server logs to Google Analytics, easily and automatically. This will allow you to dete …