Thread: robots.txt help!!!
View Single Post
Share |
  #1 (permalink)  
Old 01-04-2011, 08:19 AM
csu csu is offline
Junior Member
  
Join Date: 01-02-11
Posts: 14
iTrader: 0 / 0%
Exclamation robots.txt help!!!
In Joomla if I set robots.txt to:

Allow: /components/com_content/
Disallow: /

would this allow crawlers access articles and categories only and will the crawler get the SEF URL or the system URL please?
 
Reply With Quote