User-agent: * # disallow all files with ? in url Disallow: /*?* Disallow: /*? Disallow: /admin Disallow: /register