WordPress and Robot.txt - Help please
-
webmann
- Posts: 1
- Joined: 20 Aug 07
- Location: New Brunswick, Canada
- Trust:
25 Aug 07 9:35 am
WordPress and Robot.txt - Help please
Hello everyone, I am really glad to be here and Mark you rock.
I recently converted my web sites to WordPress blogs and started seeing my traffic increase from a few visitors a day to about 100 a day. But...
... I have been trying to optimized. That is for another post though.
for now I need to know about the robot.txt
I found I was getting links in search engine results to pages with the WP folders so I decided to disallow some in the robot txt file.
These are the folders I have disallowed.
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /category/*/*
Disallow: */trackback
Disallow: */*/trackback
Disallow: */*/feed/*/
Disallow: */feed
Disallow: /*?*
Disallow: /*?
Disallow: /docs
Disallow: /docs/
Disallow: /docs/*
Allow: /wp-content/uploads
This was done based on things I have found on the net but I am not sure if perhaps I have done something wrong.
I did this about a week or a little more ago and have noticed that my se positioning for keywords in google have dropped there position by about a hundred or more.
Anyone have some helpful info for my? I thank you all in advance.
I recently converted my web sites to WordPress blogs and started seeing my traffic increase from a few visitors a day to about 100 a day. But...
... I have been trying to optimized. That is for another post though.
for now I need to know about the robot.txt
I found I was getting links in search engine results to pages with the WP folders so I decided to disallow some in the robot txt file.
These are the folders I have disallowed.
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /category/*/*
Disallow: */trackback
Disallow: */*/trackback
Disallow: */*/feed/*/
Disallow: */feed
Disallow: /*?*
Disallow: /*?
Disallow: /docs
Disallow: /docs/
Disallow: /docs/*
Allow: /wp-content/uploads
This was done based on things I have found on the net but I am not sure if perhaps I have done something wrong.
I did this about a week or a little more ago and have noticed that my se positioning for keywords in google have dropped there position by about a hundred or more.
Anyone have some helpful info for my? I thank you all in advance.
