Home » Labs » How to optimize WordPress robots.txt file  

How to optimize WordPress robots.txt file
Filed under Labs

Below is a example for robots.txt file which can effectively use on a WordPress website / blog. Just open notepad and add below contents to it and just save it as robots.txt. Once created upload it to root of your server. Below format will give a overall idea for optimizing robots.txt for WordPress and you have to edit it according your needs.

	User-agent: *
	Allow: /
	Disallow: /cgi-bin
	Disallow: /wp-admin
	Disallow: /wp-includes
	Disallow: /wp-content
	Disallow: /e/
	Disallow: /show-error-*
	Disallow: /xmlrpc.php
	Disallow: /trackback/
	Disallow: /comment-page-
	Allow: /wp-content/uploads/
 
	User-agent: Mediapartners-Google
	Allow: /
 
	User-agent: Adsbot-Google
	Allow: /
 
	User-agent: Googlebot-Image
	Allow: /
 
	User-agent: Googlebot-Mobile
	Allow: /
 
	Sitemap: http://yoursite.com/sitemap.xml
Tags:

Posts you may like: