Htaccess is sending bots to bogus pages?
i use htaccess to rewrite from /product_info/product_id=00 to:
Product+Category/Product_Brand.html
the problem:
msn, yahoo and google are indexing pages THAT DO NOT EXIST and never have.
they're indexing VALID category names, but they are also linking them to other categories. for instance:
Old+Antiques/1920s.html (valid)
the spiders find this no problem.
but for the past month, they have been finding url's like this:
Old+Antiques/1920s__1930s.html
what they are doing is merging one category with another, and this is producing a 200/OK response.
i can't figure out why.
could it be in my htaccess or my code?
if any part of the codes used needs to be posted.. please let me know.
this is from an oscommerce-based website, but i have already hit the osc forums and it has been determined to not be an oscommerce issue.
the htaccess and rewrite script i use is not stock oscommerce.
i have sent a bot to pull every page linked on my site, none of these url's are showing up anywhere.
for reference, here is my htaccess rule:
RewriteEngine on
RewriteBase /
RewriteRule ^([^/]*)\.html$ $1.php?%{QUERY_STRING} [NC]
RewriteRule ^/?(category)/([^/]*)\.html$ index.php?cPath=$2&%{QUERY_STRING} [NC]
|
Linear Mode
