Want to work your way up in SERPs? Get and share ideas on link building, on/off page optimizing, social SEO and more.

How long should it take for site to start moving up the SERP
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
26 Dec 09 5:28 am

How long should it take for site to start moving up the SERP

Hi,

This is my first post here. I have been a premium member for about 6 weeks now and i have to say i love it here! Mark and team, you do a great job, and i'm looking forward to learning a lot more with Affilorama. This forum is also a great resouce.

My question is, how long roughly should it take for a site to start making it's move up the SERPs after doing some backlinking? I know this is impossible to put an accurate answer on, but a ball-point answer would help.

I put a site up about 3 weeks ago, and i've done some backlinking myself and also outsourced some backlinking, but at the moment i can just see the main domain on the 19th QUOTED page for one of my keywords. It just seems to be stuck there and hasn't moved.

Thanks, i hope you're all enjoying the holidays and i wish you all a prosperous New Year!

Cheers,
Paul
  • 1
PremiumMember
promocode
Posts: 201
Joined: 05 Sep 06
Trust:
26 Dec 09 2:48 pm

 Hi Paul,

The length of time that it would take to start ranking on the first page varies with each keyword/page. It depends on the how strong your competitors are in the eyes of the search engines. To get an idea, you can analyze your keyword with Traffic Travis. The "SEO Analysis" > "Analyze Keyword" section will give you some insight into the time and effort it would take to get top rankings for your keyword/page.

ex: If you chose a keyword that TT rates a 1 star - may take months and thousands of backlinks to grab a spot on the first page - conversely if you chose a 5 star keyword you may get on the front page with some proper on page SEO the day or even hour that you upload the page.

Does that make sense?

Heidi
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
27 Dec 09 6:27 am

 Hello Heidi,

Thanks for your reply, yes it makes perfect sense.

I used TT, and most of the keyword phrases i used rated a 4, so they should be pretty easy to rank highly for (or that's what i thought). Also, looking at the competition, they don't have that many backlinks to their sites to make them rank high on the first page. I will continue to add more backlinks to my site, because like i said, it seems to be 'stuck' in it's current position and hasn't moved at all.

I was just wondering if this is normal behaviour for a site, or should it have done some moving by now if the backlinking had been sufficient.

Thanks again,
Paul
  • 1
mtindell2
Posts: 3
Joined: 24 Nov 09
Trust:
27 Dec 09 7:14 am

 One of the things I would recommend to get your site listed higher quicker is to open a google webmaster tools account if you don't have one already. You can get it listed faster if you put some videos on your pages. I also use digg, twitter, facebook, and linkedin to get listed faster. Google loves social media sites. I used digg to rise to #1 in about 3 days.

Mike
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
29 Dec 09 6:34 am

 Hi Mike,
Thanks for the info. I just opened a webmaster tools account,and entered the main domain a couple of days ago.

When i checked the report today, it shows 7 "Restricted by robots.txt" in the crawl errors? I have no idea what this is, but it certainly sounds like it is affecting the site.

I downloaded the CSV error file and it says "URL restricted by robots.txt" for the main domain and each of the individual posts too.

My obvious question is what do i need to do to sort this out? Any help is greatly appreciated, thanks!
Paul
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
29 Dec 09 7:33 am

 Hi Mike,

I think i may have just answered my own question. Upon further checking in my webmaster tools account, it showed that in the /sitemap.xml.gz file in the root directory, it wasn't allowing the bots to crawl the site, it was set to the following:

User-agent: *
Disallow: /

Sitemap: "http://mysite.org/sitemap.xml.gz"

From what i can gather, the "Disallow:/" will not allow the bots to crawl. I downloaded the /sitemap.xml.gz file, but i couldn't open it to edit. So i went back to webmaster tools and it showed that this was the problem. So i created a robots.txt file and have no uploaded it to the root domain. I am hoping this will now allow the bots to crawl the site, my only concern is that the two files and directions will conflict, with the /sitemap.xml.gz not allowing the bots to crawl, while the robots.txt is trying to let them crawl.

Will this pose a problem do you think?

Thanks,
Paul
  • 1
PremiumMember
promocode
Posts: 201
Joined: 05 Sep 06
Trust:
30 Dec 09 5:03 am

 Hey Paul,



It sounds like you're saying that this statement was found in the sitemap file but the above is robots.txt code. I'm no expert in robots.txt but it appears to me that the "Disallow: / " is telling spiders to stay away from all the directories and files but it's showing where the sitemap is.

Weird, I'm sure someone will pop in with the answer. You said you created and uploaded a new robots.txt, what does it say?

Heidi
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
30 Dec 09 6:56 am

 Hey Heidi,

Yes, exactly right. When i checked my webmaster tools account, and checked the crawl errors, it showed that the "URL was restricted by Robots.txt,, and then pointed to the /sitemap.xml.gz file and showed it had this code in:
User-agent: *
Disallow: /

Sitemap: mysite.org/sitemap.xml.gz


My interpretation of that is the same as your, it is telling the bots not to read any of the site. I didn't have a Robots.txt file in the domain before, but i created one through my webmaster tools and uploaded it to the root domain. The robot.txt file contains the follow code, that should allow the spiders to crawl the site i think:
User-agent: *
Allow: /

User-agent: Googlebot
Allow: /

But i'm not sure what will happen, with the /sitemap.xml.gz file telling the spiders not to crawl and the robot.txt file telling them they can...

I'm guesing that the /sitemap.xml.gz file is created by the XML Sitemap plugin, but i don't understand why it would be set to User-agent: *
Disallow: /

...it doesn't really make much sense to me. I have checked the settings for the XML sitemap plugin and i can't see anything that i think would allow the spiders to crawl the site.

Hopefully i'm missing something and someone can tell me what i'm doing wrong. This is the first time i have even heard of robot.txt, so i'm learning all of this as i go :-)

Thanks again Heidi,
Paul
  • 1
Last edited by ampie g on 03 Mar 10 3:26 am, edited 2 times in total.
Reason: removed link as URL is only used for illustration and not an actual link
 

PremiumMember
promocode
Posts: 201
Joined: 05 Sep 06
Trust:
30 Dec 09 12:32 pm
paultim71 wrote:
But i'm not sure what will happen, with the /sitemap.xml.gz file telling the spiders not to crawl and the robot.txt file telling them they can...

I'm guesing that the /sitemap.xml.gz file is created by the XML Sitemap plugin, but i don't understand why it would be set to User-agent: *
Disallow: /



I don't believe that it was the sitemap that was creating the disallow rule. The webmaster tool that your using must have created the file (the robots.txt) now that you've created one with the allow settings you should be good.

Here's a good tutorial on the robots file: "http://janeandrobot.com/library/managing-robots-access-to-your-website"
,it makes them easy to understand.

Best,
Heidi
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
30 Dec 09 7:30 pm

 Hey Heidi,

Thanks for your help with this, and the link to the tutorial, i really appreciate it.

I'm still trying to get my head around it all though. When i first checked the crawl errors in my webmaster tools account, it showed that it was set to Disallow. So i then created a new robot.txt with Allow for Googlebot and saved the txt file and uploaded it to my domain...

...but reading further, it says that a robot.txt file should ONLY be uploaded if you want to block Googlebot (or any other bot) from accessing certain pages. I don't want anything blocked at the moment, i want the Googlebot to access all the pages.

Another thing that has me concerned, is that when i locate my site in Google, it just shows as the domain "http://www.mysite.org".
It has no anchor text or and short description, just the domain name. Why is this?? Is this because Googlebot is unable to crawl the site and get any more information?

I have just launched another site a couple of days ago too, and the same thing is happneing with this. It;s getting frustrating because i really can't get to the bottom of it.

I have outsourced a lot of backlinking to the first site, and intend to do so for the second one soon too, but i would like to try and get to the bottom of this problem first.

Thanks again Heidi,
Paul
  • 1
PremiumMember
promocode
Posts: 201
Joined: 05 Sep 06
Trust:
31 Dec 09 7:47 pm

 Hi Again Paul,

You're very welcome, I'm glad I'm able to help you.

paultim71 wrote:
...but reading further, it says that a robot.txt file should ONLY be uploaded if you want to block Googlebot (or any other bot) from accessing certain pages. I don't want anything blocked at the moment, i want the Googlebot to access all the pages.


Well there you go - that would explain the puzzling disallow rule ;)
just for ha ha's take a look at your robots.txt by putting

in your browser (it should be there in the root of your domain)

is it your new robots file or the old one?

paultim71 wrote:
Another thing that has me concerned, is that when i locate my site in Google, it just shows as the domain "http://www.mysite.org".
It has no anchor text or and short description, just the domain name. Why is this?? Is this because Googlebot is unable to crawl the site and get any more information?



What happens if you search for it with this command:

site:"yoursite.org"
do you see any pages listed?

I know it can be incredibly frustrating getting the first website worked out but it gets easier and and problem solving will come very naturally to you soon.

And even then mistakes will happen....

Today I found out that a page that I published went screaming up to the front page of the serps overnight. My affiliate links were getting a 54% CTR but when I checked my affiliate stats I saw zero so I looked at my page source...

and found that I had used the wrong affiliate link and was sending the traffic to a completely unrelated offer no less! It happens. I just keep building and learning everyday :)

Wishing you a healthy and Happy New Year Paul.

Heidi
  • 1
hans1804
Posts: 3
Joined: 31 Mar 09
Trust:
01 Jan 10 6:52 pm

 Hi Heidi,

and found that I had used the wrong affiliate link and was sending the traffic to a completely unrelated offer no less! It happens. I just keep building and learning everyday :)


Grin I just found out that I was sending Restaurant Recipes traffic to Zygors WoW guide, and wondered why it didn't convert :-) Your confession made my day, thanks.!!

Hans
  • 1
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
02 Jan 10 4:36 am

 Hello Heidi.

Yes, when i go to the URL, "http://www.mysite.org/robots".txt it is showing the Allow rule. Also, when i search for: site:mysite.org it
just brings up the URL again, no anchor text or description at all....very strange!

Yes it is frustrating, but also i'm glad i'm coming across thee problems becuase then i'll know what to do in the future when i eventually find out what it wrong now. I know i must have done something wrong because it is doing the same for two sites i now have launched.

When i go to my webmaster tools account, and look in Diagnostics / Crawl Errors, it says URL restricted by Robots.txt for all of the seperate posts as well as the main domain, so it is just very confusing for me at the moment because i'm really at a loss as to what it can be.

Oooh....thats sounds bad about your wrong affiliate link!!! At least you found it quite early on, i hope the site is still performing the same now with your correct affiliate link :-)

Happy New Year to you Heidi, and all. All the best for 2010!

Thanks,
Paul
  • 1
tbrim20
Posts: 67
Joined: 30 Nov 09
Trust:
14 Jan 10 2:29 pm

 I thought I would add to this post since there was never an answer provided.

The robots.txt file tells the bots what you want them to look at and what not to look at.

When you get this problem in Google Webmasters Tools, it can be from a couple of things.

1. If you are using wordpress, there is a setting to keep your blog private which for me is the default setting. This will create the "Disallow" in your robots.txt Just uncheck the box and you'll be good to go.

2. If it is standard alone non-wordpress/blog website then you'll need to edit (or delete) the robots.txt file and change the "Disallow" to "Allow".

3. Google Webmasters Tool does take time to refresh these errors after you fix it. So if you fix it, it can still take a couple of weeks before the error is removed from your Webmaster Tools page.

Cheers,

-Tom
  • 1
www.RookietoRich.com
Get My Free Method to making some Fast Cash and Kickstart your Internet Marketing Business today.
 
paultim71
Posts: 8
Joined: 17 Aug 09
Trust:
15 Jan 10 12:37 pm

 Hi Tom,

Thanks for your reply. I actually found the page about a week ago in my Wordpress admin area, that had the two options for the Blog visability "Blog visable to everyone, including search engines" and "block the search engines". This was under
Settings/Privacy. By default it was set to "block the search engines"

I did feel a bit stupid when i saw it, although i knew i must have been doing something like that wrong as it was having the same effect on two different sites. But a good lesson learned anyway

Now one of my sites is on page 2 for one of the keywords, and the other sites is moving up slowly but surely too.

Thanks Tom,
Paul
  • 1

This topic was started on Dec 26, 2009 and has been closed due to inactivity. If you want to discuss this topic further, please create a new forum topic.

Topic locked