dana No Further a Mystery
the worth from the identify attribute (robots) specifies that the rule applies to all crawlers. both equally the identify as well as the articles characteristics are circumstance-insensitive. To address a specific crawler, change the robots value of the title attribute Together with the person agent token from the crawler that you'll be addressing.