Comments on: Noindex Meta tag versus robots.txt disallow – which to use? http://www.webstrategies.co.nz/2009/09/04/noindex-meta-tag-versus-robotstxt-disallow-which-to-use/ WebStrategies SEO experts in Auckland, New Zealand Sat, 26 Sep 2009 08:58:21 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Webstrategies http://www.webstrategies.co.nz/2009/09/04/noindex-meta-tag-versus-robotstxt-disallow-which-to-use/comment-page-1/#comment-313 Webstrategies Sat, 26 Sep 2009 08:58:21 +0000 http://www.webstrategies.co.nz/?p=718#comment-313 Thanks for the question. The Robots disallow will still allow the URL to display in the SERPS, although there wil be no page content dispayed in the SERPS. As the URL is available people can still go the URL and see the pages. So in this scenario Google will not consider the page content, so you should not have a diplicate content issue, however people can see the URL and follow it. The meta noindex tag will keep everything out of Google including the URL thus making it a bit more hidden from humans as well (but not fully of course!) Thanks for the question.

The Robots disallow will still allow the URL to display in the SERPS, although there wil be no page content dispayed in the SERPS. As the URL is available people can still go the URL and see the pages. So in this scenario Google will not consider the page content, so you should not have a diplicate content issue, however people can see the URL and follow it.

The meta noindex tag will keep everything out of Google including the URL thus making it a bit more hidden from humans as well (but not fully of course!)

]]> By: The Therapist http://www.webstrategies.co.nz/2009/09/04/noindex-meta-tag-versus-robotstxt-disallow-which-to-use/comment-page-1/#comment-300 The Therapist Fri, 25 Sep 2009 21:55:35 +0000 http://www.webstrategies.co.nz/?p=718#comment-300 I'm currently caught in this very dilemma. In fact I just did a Google search for "meta tag noindex google" and your article popped up. I'm about to add about 10,000 pages to my site, but they will all be stock pages. However, once my customers input data into their control panels, these pages will be full of original content. My worry is- I don't want Google to know about the pages UNTIL customers enter data, yet I have to create them all in advance (its done dynamically). It is easy for me to just programmatically insert a meta tag for "noindex" on any of the pages that do not contain data, however, I am worried if that may not be enough? The last thing I want is our already excellent search rankings to be hurt. Is a Robotx.txt file superior to the Meta tag? I'm still not clear :( I’m currently caught in this very dilemma. In fact I just did a Google search for “meta tag noindex google” and your article popped up. I’m about to add about 10,000 pages to my site, but they will all be stock pages. However, once my customers input data into their control panels, these pages will be full of original content. My worry is- I don’t want Google to know about the pages UNTIL customers enter data, yet I have to create them all in advance (its done dynamically). It is easy for me to just programmatically insert a meta tag for “noindex” on any of the pages that do not contain data, however, I am worried if that may not be enough? The last thing I want is our already excellent search rankings to be hurt. Is a Robotx.txt file superior to the Meta tag? I’m still not clear :(

]]>