# Allow W3C's Link Checker User-agent: W3C-checklink Disallow: # Disallow all other robots User-agent: * Disallow: /OLD/ Disallow: /development-pages/ Disallow: /RedHat/