SEO Indexation Control Fundamentals
Earlier this week Google announced effective from 1st September, it will no longer observe unsupported and unpublished rules in robots.txt files. One notable example of this is advisory that Google will no longer support inclusion of the noindex directive within a robots.txt file. Google advised the decision to discontinue support of such unsupported and unpublished rules was to maintain a “healthy ecosystem” and prepare for potential future open source releases ahead of the open-sourcing of Google’s production robots.txt parser.
In this post I’m going to explore alternative more optimal indexation control methods and more crucially which methods are best suited to specific scenarios.
