 |
|
02.24.10 Which SEO Robot Is Better For Your Site  By Brandon Leibowitz It has become common practice to implement robot.txt files to prevent the search engine spiders from crawling your webpage and having it show up in the search engine results page (SERP). Unfortunately, robots.txt does not prevent your site from showing up in the search engine results. That means your web page is passing on link juice that is unused and wasted. It does prevent your site from being indexed, but it could show a result in the SERP's. It will not have a description of your site, only a URL and strange title. Matt Cutt's from Google explains this in his video A better way to prevent a web site from being crawled and indexed in the search engines is to use a robots meta tag a X robots-tag http header .htaccess file to your website.
Meta Tag  X robots-tag http header .htaccess The new tag allows you to create a HTTP header called X-Robots-Tag and set the value as you would the meta robots tag value. This will set up the file for the entire site making your life much easier. So, if your site is running on Apache, and mod_headers is enabled (it usually is), you could add the following single line to your .htaccess file: Header set X-Robots-Tag "noindex, nofollow" Comments About the Author: Brandon Leibowitz has been working in the internet marketing field, mainly focusing in on SEO and SEM for over 5 years. Check out his website SEO Optimizers and his blog SEO Optimizers Blog. |
|||||||
|
| ||
-- DevWebProBR is an iEntry, Inc. publication -- iEntry, Inc. 2549 Richmond Rd. Lexington KY, 40509 © 2010 iEntry Inc. All Rights Reserved Privacy Policy Legal
|
