User-agent: * Crawl-Delay: 60 User-agent:Googlebot Crawl-Delay: 1 User-agent: AhrefsBot Crawl-Delay: 60 User-agent: MJ12bot Crawl-Delay: 60 User-agent: SemrushBot Crawl-Delay: 60 User-agent: * Disallow: /m/js/ User-agent: * Disallow: /m/locale/ User-agent: * Disallow: /p/js/ User-agent: * Disallow: /p/locale/ User-agent: * Disallow: /*.webp