Google is Deprecating Its AJAX Crawling Scheme

In a blog post in the Google Webmaster Central Blog, Google announced that it no longer is recommending the AJAX crawling proposal it made back in 2009. In other words, the major search engine is expressing its disapproval (that’s what deprecate means) of the AJAX crawling scheme.

What is the AJAX Crawling Proposal?

Six years ago, in October 7, 2009, Google proposed a new standard to make AJAX-based websites crawlable. AJAX, short for asynchronous JavaScript and XML, is a group of interrelated Web development techniques used on the client-side to create asynchronous (a form of input/output processing that permits other processing to continue before the transmission has finished) Web applications. There are websites that use AJAX on their pages because of its advantages.

AJAX Advantages

With AJAX, the following are made possible:

1. Update a web page without reloading the page
2. Request data from a server – after the page has loaded
3. Receive data from a server – after the page has loaded
4. Send data to a server – in the background

AJAX-based applications is a great development for users because it makes applications much faster and richer. And, back in 2009, it was proposed by Google to make these pages crawlable. But making applications more responsive has come at a huge cost: crawlers are not able to see any content that is created dynamically. Consequently, the most modern applications are also the ones that are often the least searchable. Thus, the proposal by Google to deprecate its AJAX crawling strategem.

What Now?

Google realized that the assumptions they made back in 2009 are no longer valid. They are now recommending strategies for web design that emphasize accessibility, semantic HTML markup, and external stylesheet and scripting technologies. An example given was using the History API pushState() to ensure accessibility for a wider range of browsers and Google’s systems.

Q & A with Kazushi Nagayama

Q: My site currently follows your recommendation and supports _escaped_fragment_. Would my site stop getting indexed now that you’ve deprecated your recommendation?
A: No, the site would still be indexed. In general, however, we recommend you implement industry best practices when you’re making the next update for your site. Instead of the _escaped_fragment_ URLs, we’ll generally crawl, render, and index the #! URLs.

Q: Is moving away from the AJAX crawling proposal to industry best practices considered a site move? Do I need to implement redirects?
A: If your current setup is working fine, you should not have to immediately change anything. If you’re building a new site or restructuring an already existing site, simply avoid introducing _escaped_fragment_ urls. .

Q: I use a JavaScript framework and my webserver serves a pre-rendered page. Is that still ok?
A: In general, websites shouldn’t pre-render pages only for Google — we expect that you might pre-render pages for performance benefits for users and that you would follow progressive enhancement guidelines. If you pre-render pages, make sure that the content served to Googlebot matches the user’s experience, both how it looks and how it interacts. Serving Googlebot different content than a normal user would see is considered cloaking, and would be against our Webmaster Guidelines.

If this AJAX-crawling update is too advanced for you and you have more questions, feel free to post them in the webmaster help forum.