Robot.txt file for Blogger, WordPress and For Custom Website: Perfect Robot.txt File For Blogger, WordPress and For a Custom HTML, PHP based Website: Hi, in Article there is Perfect Robot file for your blog or website.
What is the Robot.txt file:
Robot.txt file is the main file in the Search Engine Optimization (SEO), robot.txt file allows the search engines to crawl your website and search engines automatically crawl your posts URL to Webmaster.
Also Read: How to Add Contact Form in Blogger
Robot.txt Files Type:
For Custom Website(HTML, PHP, etc.):
User-agent: * Allow: / Allow: /sitemap.htm Sitemap: https://example.com/sitemap.xml
For Blogger:
User-agent: * Allow: / Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED
For WordPress:
User-Agent: * Disallow: /wp-admin/ Sitemap: https://example.com/sitemap_index.xml
Download Text File: Download
Why Robot.txt File:
Robot.txt file is an important part of every website, which actually maintains the SEO of the website, it allows or disallows the search engine to visit, and also if you did not want to show a particular page, by robot.txt file we can easily block it.
Thanks For Visiting, Team NoobSpot.Com