If you have problems with indexing of new pages, although the quality of the pages is at a high level, then perhaps the Google Indexing API can help. IMPORTANT: Google recommends the use of the Indexing API ONLY for Job Posting and Live Streaming. But tests showed that you can index normal website content with the API as well. So you are doing this at your own risk, if you use the API for other content. Following you find a step-by-step tutorial how to set up the Google Indexing API with Python.
Contents
Create a new project on the Google Cloud Platform
First you need access for the Google Indexing API and for that you need a new project on the Google Cloud Platform. You can do that by clicking here.
Create a Service Account
Now you need a Service Account for your new project. Visit for this the service accounts page, select your project and click now in the following window on the “+ Create Service Account” button on the top of the page.
Create a private key
Click now in your Service Account Overview on the three points on the right side and click on “Manage Keys”
Click here on “Add Key” and choose “Create new Key”, “JSON” and “Create”. Now save the file in your python project folder.
Add your Service Account E-Mail address to your Search Console Property
Thereby Google can understand the connection between the Service Account with Indexing API access and the Domain which pages should be indexed, you need the legitimization. For this you have to add the Service Account E-Mail address as an owner of your Google Search Console Property.
Login into you search console account, choose the right property and click in the sidebar on the left on “Settings”. Now click on “Users and permissions” and there on the blue button in the right corner “Add User”. Insert your Service Account E-Mail address and change the permission to “Owner”.
Create the python script
Now you can copy and paste the following python script and replace the JSON_KEY_FILE variable:
from oauth2client.service_account import ServiceAccountCredentials
import httplib2
SCOPES = [ "https://www.googleapis.com/auth/indexing" ]
ENDPOINT = "https://indexing.googleapis.com/v3/urlNotifications:publish"
# service_account_file.json is the private key that you created for your service account.
JSON_KEY_FILE = "service_account_file.json"
credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES)
http = credentials.authorize(httplib2.Http())
# Define contents here as a JSON string.
# This example shows a simple update request.
# Other types of requests are described in the next step.
content = """{
\"url\": \"http://example.com/jobs/42\",
\"type\": \"URL_UPDATED\"
}"""
response, content = http.request(ENDPOINT, method="POST", body=content)
print(response)
print(content)
Now you can run the script. Perhaps you have to install oauth2client (pip install oauth2client)
See more here: https://developers.google.com/search/apis/indexing-api/v3/prereqs
You can check the notification status with a simple GET request:
Send a HTTP GET request to https://indexing.googleapis.com/v3/urlNotifications/metadata?url=url-encoded_url
(Replace / with %2F in the URL, _ with %5F or – with %2D)
Can we host this python script online on heroku, vercel etc and use it from there
Hi Vineet,
if you can run Python scripts, then this should not be a problem. On Heroku, for example, this should be possible.
Best,
Christian