Normally you have to wait until a crawler finds your new page, crawls it, and passes it to the indexer, so finally your new page could be found in the search engine result pages. A long and cumbersome way. IndexNow wants to change that! With this service, you could tell search engines directly that you have a new URL that should be indexed. So search engines know immediately that there is a new URL and can prioritize crawl for these URLs.
Contents
Which search engines support IndexNow?
With the start of IndexNow in November 2021, both search engines, Microsoft Bing and Yandex, support IndexNow.
And what’s with Google?
At the moment Google doesn’t support IndexNow. But the company confirmed it is testing the protocol. Google told Search Engine Land:
“We take a holistic approach to sustainability at Google, including efforts to develop the most efficient and accurate index of the web. We have been carbon neutral since 2007 and we are on pace to be carbon-free by 2030 across both our data centers and campuses. We’re encouraged by work to make web crawling more efficient, and we will be testing the potential benefits of this protocol.”
But it could be a long way for the adoption of IndexNow. Google’s John Mueller said that he wouldn’t assume from a limited test that this will launch broadly anytime soon.
How works IndexNow?
IndexNow works like a ping. You have only to submit a URL with an HTTP request in the form like:
https://<searchengine>/indexnow?url=url-changed&key=your-key
A crawler should soon be crawling the URL. But keep in mind: Indexing is not guaranteed! Also consider that there is a crawl quota (other term for crawl budget) for your site.
The good thing is, you only have to submit your URL to one search engine. All “IndexNow-enabled search engines will share immediately all URLs submitted to all other IndexNow-enabled search engines, so when you notify one, you will notify all search engines.”
How I could use IndexNow?
Using IndexNow is very simple:
1.) Generate API Key
You have to authorize yourself as the owner of the domain. For that, you have to generate an API Key. Visit the following page: https://www.bing.com/indexnow and copy the generated API Key. Let’s assume the API key is: 9912f54d967e46d2bdad1df0c07a2018
2.) Host API Key
Now you have to host this API key on your page. Create a txt-File with the name of your API key and paste in there also the API key. In our example the file should be named: 9912f54d967e46d2bdad1df0c07a2018.txt with the content “9912f54d967e46d2bdad1df0c07a2018”. You can also click on the download icon on the API Generation Key page and you have this file already. Upload this file in the root directory of your page.
3.) Submit new URLs
Now you are ready for submitting new URLs. For the search engine Bing you can now send new URLs with the following URL schema:
https://www.bing.com/indexnow?url=http://www.example.com/product.html&key=9912f54d967e46d2bdad1df0c07a2018
You have only to replace the URL and the key with your data.
Be careful: You don’t get direct visual feedback if the submission was successful. You have to look in the developer tools after the response code. An HTTP code 200 indicates, that the URL was submitted successfully. The other status codes tell you the following:
- HTTP Code 400: Invalid format of the URL
- HTTP Code 403: The key is not valid
- HTTP Code 422: URLs don’t belong to the host or the key is not matching the schema in the protocol
- HTTP Code 429: Too Many Requests (potential Spam)
Now you can also monitor the crawl rate and indexing through Bing Webmaster Tools.