Want to work your way up in SERPs? Get and share ideas on link building, on/off page optimizing, social SEO and more.

How spiders find your site
addisoncave
Posts: 58
Joined: 21 Jun 16
Trust:
17 Sep 16 7:50 am

How spiders find your site

.What are the ways in which spiders can find the web site?
  • 0
Site Admin
maryt
Posts: 3225
Joined: 16 Apr 12
Trust:
18 Sep 16 10:43 pm

 You just have to make sure that your website is indexable, which means you need to check your sitemap, making sure the pages are set to do-follow and do-index.
  • 0
Not sure which Affilorama tool you need? Take the quiz! https://www.affilorama.com/redirect/quiz

Web: https://www.affilorama.com
Twitter: https://twitter.com/affilorama
Facebook: https://www.facebook.com/affilorama

***
"Do not dwell in the past, do not dream of the future, concentrate the mind on the present moment." ~Sidhhartha Guatama
 
antwerp203
Posts: 76
Joined: 02 Aug 16
Trust:
21 Sep 16 5:21 am

 Hi there! First you have to understand how SE spiders work. They are robots and they don't read websites the way human does. They check only certain stuff and don't mind any extras like Flash or Javascript...

When they arrive, they usually go to the main root or the robot.txt and look for any indexables files. Then it will go and check for the title of the web page, the keywords used, meta tags, etc. within the content. Once these areas are checked, your website will be ranked after...:)
  • 0
Get to know more on how to effectively earn money through affiliate marketing here:

http://bit.ly/2EPitwC
 
pankajvmgupta
Posts: 2
Joined: 10 Sep 16
Trust:
26 Sep 16 11:05 am

 Google Can crawl your data very well though its a css, javascript, image, content, links, databases everything..
  • 0
http://www.ewebac.com | EWEBAC is one the fastest growing SEO Company based in mumbai, helping businesses to grow online.
 
carolinemurphy
Posts: 105
Joined: 10 Jun 15
Trust:
10 Oct 16 12:15 pm

 Hi there,

Basically, all search engine spiders function on the same principle – they crawl the Web and index pages, which are stored in a database and later use various algorithms to determine page ranking, relevancy, etc of the collected pages. While the algorithms of calculating ranking and relevancy widely differ among search engines, the way they index sites are more or less uniform and it is very important that you know what spiders are interested in and what they neglect.

Search engine spiders are robots and they do not read your pages the way a human does. Instead, they tend to see only particular stuff and are blind for many extras (Flash, JavaScript) that are intended for humans. Since spiders determine if humans will find your site, it is worth to consider what spiders like and what don't.
  • 0
magentoocodewire2015
Posts: 94
Joined: 07 Apr 16
Trust:
17 Oct 16 4:20 am

 Spider can found your site with the help of following tools like Google webmaster tool etc. You have to use the best ON -page SEO technique for the better result. Spider can crawl your website if your are having quality backlinks, indexing etc.
  • 0
rhcalvin
Posts: 34
Joined: 20 Oct 16
Trust:
25 Oct 16 5:26 am

 Web spiders are automated search engine programs that have their own way to fetch for new and updated webpages to crawl and provide information to search engines.
  • 0
$1 VPS Hosting - http://vpshostingdeal.com
Cheap Dedicated Servers - http://reprisehosting.com/dedicated-server
 
tampata3
Posts: 27
Joined: 27 Dec 16
Trust:
27 Jan 17 11:50 am
carolinemurphy wrote:Hi there,

Basically, all search engine spiders function on the same principle – they crawl the Web and index pages, which are stored in a database and later use various algorithms to determine page ranking, relevancy, etc of the collected pages. While the algorithms of calculating ranking and relevancy widely differ among search engines, the way they index sites are more or less uniform and it is very important that you know what spiders are interested in and what they neglect.

Search engine spiders are robots and they do not read your pages the way a human does. Instead, they tend to see only particular stuff and are blind for many extras (Flash, JavaScript) that are intended for humans. Since spiders determine if humans will find your site, it is worth to consider what spiders like and what don't.


Thanks for your comment. I also agree with you.
  • 0
Meet with a licensed tax attorney Tampa.

https://www.taxlawtampa.com/
 
nickchernets
Posts: 55
Joined: 14 Mar 17
Trust:
14 Apr 17 11:43 pm

 If it is indexed it will be crawled by google spider
  • 0

This topic was started on Sep 17, 2016 and has been closed due to inactivity. If you want to discuss this topic further, please create a new forum topic.

Topic locked