Hey everyone,
I run a forum, its a phpbb, and it was suggested here that I use a mod_rewrite to change my dynamic urls to static. I had to switch hosts and it took me awhile to redo the mod_rewrite and a few of my pages got index under the wrong URL. I was wondering how I can write a robots.txt file to exclude the non static pages. I want to keep the non-static urls because they are used by members here are some examples of the links...
I dont want google to see these pages...
http://www.always420forum.com/forum/viewforum.php?f=*
http://www.always420forum.com/forum/viewtopic.php?t=*
I do want them to see these pages...
http://www.always420forum.com/forum/forum-*.html
http://www.always420forum.com/forum/about*.html
How would on write the robots.txt file? Thanks for the help!
edit- just realized I probably should have put this in the SEO section, I was worrying about duplicate content specifically for google so this is where I was looking...sorry!