User-agent: TurnitinBot Disallow: / #Will disallow all urls on your site User-agent: BingPreview Disallow: /CCNEngine/ #disallows access to CCNEngine directory User-agent: * Disallow: /CCNEngine/ #disallows access to CCNEngine directory User-agent: * Disallow: /EC/ #disallows access to EC directory User-agent: Mediapartners-Google Allow: /CCNEngine/ #allows access to CCNEngine directory for Google AdSense ads crawler