User-agent: * Disallow: /assets/css/ Disallow: /assets/flash/ Disallow: /assets/images/ Disallow: /assets/scripts/ Disallow: /404/