# /robots.txt file for crawlers User-agent: * Disallow: /documents Disallow: /download Disallow: /images User-agent: Googlebot-Image Disallow: / User-agent: Googlebot-Mobile Disallow: / User-agent: Googlebot Disallow: /documents Disallow: /download Disallow: /images