View Single Post
Share |
  #1 (permalink)  
Old 01-04-2011, 09:19 AM
csu csu is offline
Junior Member
Join Date: 01-02-11
Posts: 14
iTrader: 0 / 0%
Exclamation robots.txt help!!!

In Joomla if I set robots.txt to:

Allow: /components/com_content/
Disallow: /

would this allow crawlers access articles and categories only and will the crawler get the SEF URL or the system URL please?
Reply With Quote