vBulletin Modifications

Spiders Spiders Spiders

Welcome to vBHackers.com! - vBHackers Updates:

Go Back   vBulletin Modifications > Your Forum > Growing Your vB Forum

Reply
 
LinkBack Thread Tools
Old 10-15-2007, 09:12 AM   #1
vB Newbie
Join Date: Oct 2007
Real Name: Allan
Location: Wirral, Uk
Mr-Ice-Man is on a distinguished road

Question Spiders Spiders Spiders

Can anyone tell me please what i need to do to gt spiders coming to my site please.
Ive heard on some forums people say about makeing a file called robots.txt but what should i put in it?

Hope someone can help.
Thankyou.




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Sponsored Links
Old 10-15-2007, 11:49 AM   #2
Michael Biddle
Administrator
Michael Biddle's Avatar
Join Date: Feb 2005
Real Name: Michael Biddle
Location: Anaheim
Michael Biddle is on a distinguished road

Default

robots.txt is for spiders not to crawl in certain locations. to get them their in the first place, you need to have content
__________________
Support will ONLY be offered through forums


Michael Biddle / vBHackers.com
vBSEO 3.1 GOLD Released vB Blogs Compatible | Non-Encrypted | Branding Removal Option | Sitemaps 2.1 | Language Packs

Crawlability Network: vBulletin SEO | vBulletin Hackers
Affiliates: vBulletinSetup
Are you interested in a Recipe mod for vBulletin
Free Modifications: Latest X Posts on Forumhome | Default Signature | Default Signature Per Usergroup




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Old 10-15-2007, 06:49 PM   #3
Colonel Ingus
Master Coder
Colonel Ingus's Avatar
Join Date: Jun 2006
Real Name: The Wizard of Oz
Location: ...in a van, down by the river.
Colonel Ingus is on a distinguished road

Wink

Quote:
Originally Posted by whitemike View Post
to get them their in the first place, you need to have content
Not necessarily. They will still go to the site without content, they just won't have anything to catalog

Mr-Ice-Man, the bots will go to your site automatically in around a month or two from when you first registered it, unless it's on a ban list. I'd suggest you add that content that Mike was talking about or it will hurt you. If they don't see fresh content they will stay away longer, defeating what you're trying to accomplish
__________________

Colonel Ingus - N.A.P.P Member   

Vbulletin Coding Guru / vBHackers.com
vBSEO 3.1 GOLD Released vB Blogs Compatible | Non-Encrypted | Branding Removal Option | Sitemaps 2.1 | Language Packs

Crawlability Network: vBulletin SEO | vBulletin Hackers




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Old 10-23-2007, 08:35 AM   #4
vB Newbie
Join Date: Aug 2007
night2dark2 is on a distinguished road

Default

hmm yes that is quite intersting question. Spiders crawl my site for at least 42-90 hours a month. With iuno abot 20 Spiders. Same for yahoo and all other search engines.




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Old 10-23-2007, 03:45 PM   #5
loco.m
vB SEO Nutcase
loco.m's Avatar
Join Date: Nov 2005
Real Name: Brandon Sheley
Location: Kansas
loco.m is on a distinguished road

Default

Add back links to your site, and quality content.
__________________
Brandon Sheley / vBHackers.com
vBSEO 3.1 GOLD Released vB Blogs Compatible | Non-Encrypted | Branding Removal Option | Sitemaps 2.1 | Language Packs

Crawlability Network: vBulletin SEO | vBulletin Hackers


We offer vBulletin Services @ vBulletin Setup - = - Barcode Signature
I run a few Directories, submit your sites: Quality Link Directory, Link Directory, Link Directory
Have you seen Crowdgather yet? it's where you can Find it on Forums




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Old 10-23-2007, 04:19 PM   #6
Overclocking Elite
Join Date: Oct 2007
BlazingPC is on a distinguished road

Default

Submit your URL(s) to all the major search engines and sign up for Googles webmasters tools and create a site map.
This will also tell you how to create a robot.txt file, Then its a matter of getting your links out there on other sites to help the bots find your site from other places on the net.




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Old 10-24-2007, 03:28 PM   #7
Colonel Ingus
Master Coder
Colonel Ingus's Avatar
Join Date: Jun 2006
Real Name: The Wizard of Oz
Location: ...in a van, down by the river.
Colonel Ingus is on a distinguished road

Wink

Yes, don't forget about that site map. This is a great tool left out by many webmasters.
__________________

Colonel Ingus - N.A.P.P Member   

Vbulletin Coding Guru / vBHackers.com
vBSEO 3.1 GOLD Released vB Blogs Compatible | Non-Encrypted | Branding Removal Option | Sitemaps 2.1 | Language Packs

Crawlability Network: vBulletin SEO | vBulletin Hackers




Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply




Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

Similar Threads
Thread Thread Starter Forum Replies Last Post
eXtend Show Spiders Tim vBulletin 3.5 / 3.6 Hack Releases 19 11-11-2007 04:08 PM
Spiders have same rights as members. deadeye vBulletin Modification Requests 0 12-07-2006 08:09 AM
Spiders as members Vaz General vBulletin Support 6 07-18-2006 02:29 PM
Do spiders love archives? Rex Growing Your vB Forum 7 06-03-2006 03:48 PM


All times are GMT -3. The time now is 05:51 PM.


SEO by vBSEO 3.1.0 ©2007, Crawlability, Inc. (Patent Pending)