Thread: What is in your robots.txt file?
View Single Post
Old 05-04-2008, 05:13 PM   #2 (permalink)
TechSup
Contributing Member
 
TechSup's Avatar
 
Join Date: 12-27-07
Location: North Carolina
Posts: 655
iTrader: 1 / 100%
TechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really niceTechSup is just really nice
Mine just includes the directories allowed and not allowed, specific spiders to crawl and that is it. Very low in text.
__________________
Best Regards, TechSup
Google your question(s) before you ask.
TechSup is offline   Reply With Quote