Site owners will always be looking for getting their website indexed by search engines like Google. However, if you are in the initial development stage of a web a page you might not want search engines to crawl through your website. In such cases, you can discourage search engines from indexing your site by giving some instructions via codes. This piece will illustrate how to halt search engines from crawling the content of WordPress websites.
Why Would You Want To Stop Google From Indexing Your Site?
There are some cases where website developers want to stop search engines from indexing their sites:
- Unfinished websites — at this stage of errors and trials, it’s best not to have your website indexed by search engines to stop its visibility to the public.
- Restricted websites — if you are running a private or invite-only website, you do not want to get it in Google or other search engines results page.
- Test accounts — website owners sometimes create a copy of the original website for testing purposes and during this time they don’t want to get it indexed by search engines.
So how do you block search engines from indexing your website? Well, take a look at several options below and try it yourself.
Stop Google Indexing Using the WordPress Built-In Feature
This is possible by editing the robots.txt file as you only need to use a WordPress built-in feature. Here’s the way:
- Login to WordPress admin area and go to Settings -> Reading.
- Move down and find the Search Engine Visibility option.
- Check the option’Discourage search engines from indexing this site’.
Save Changes, and that’s it! WordPress will modify the robots.txt file for you.
WordPress will also modify the robots.txt file of the website and add the following code to it:
User-agent: *
Disallow: /
These codes tell robots (web crawlers) not to index your pages. However, it all depends on whether the search engine accepts this request or ignores it. Even though most search engines follow this rule, some pages or random pictures on the site may still be indexed.
Stop Search Engines Using WordPress Plugins
There are many plugins available that can help to password protect your site so that no one access the site. The Password Protected plugin might be one of the best solutions and it’s very easy to use
After installing and activating the plugin, go to Settings -> Password Protected and configure the settings to match your needs.
How to Remove an Indexed Page From Google?
What if Google has already indexed your web pages and you don’t want it to be there. Don’t worry even if Google has indexed your website. You will be able to submit a request to remove it from SERPs by following these steps:
- Set up Google Search Console for your website.
- Login to Google Search Console and scroll down to Legacy tools and reports -> Removals.
- Press the ‘Temporarily hide button’ and submit the URL you want to remove from Google.
- On a new window, choose ‘Clear URL from the cache and temporarily remove from search’, then ‘Submit Request’.
And that’s it! Google will temporarily remove your website pages from search results.
