For web marketing training, presentations and workshops visit Pacifica Training. If you wish to ask questions about website marketing please head over and register free at the website marketing forum, a knowledge base and community driven discussion group.
AJAX and SEO
Published: December 8, 2005
AJAX and Search Engine Optimization
AJAX, Web2.0... whatever it's called, is fundamentally a problem for accessibility from a users point of view but from a business point of view, it's immediately less search engine friendly and this gives us major problems in the marketing department.
AJAX, for those who haven't heard the term, is a new age way of bringing software-like usability to the web. It uses existing technologies but usefully pulls them together to create powerful online applications. The most famous examples of AJAX, or Web2.0 is Google's Suggest feature and Google's Maps tool.
In a nutshell, it's a means of using asynchronous JavaScript and XML to query a remote data source and render the content without refreshing or reloading the page.
The danger of this, from a search engine optimisation point of view is that we are doing away with unique URL's and we have few options to manipulate TITLE tags and page headers, the main requirement for an effective SEO campaign.
As much as SEO is dismissed as secondary to design, a website is a tool for business and should fundamentally reach its market first and foremost.
How to search engine optimise Ajax
The technique I wish to propose contains two main rules and will fundamentally change the way the AJAX application is constructed.
Rule #1
The initial load of the AJAX application must contain the optimised elements such as TITLE and headers and also must be reachable from a fixed address.
Rule #1 is all about using the following two techniques.
- Server side script
- .htaccess mod_rewrite
If you're not based on the Apache server when it comes to hosting then mod_rewrite may not be an option but there are always solutions. Check out your hosting platforms options with regards to URL rewriting.
The following links offer an example of rewriting an URL to form a fixed address that a search engine can index safely and rank.
- http://labs.tn38.net/ajax_seo/company/1/
- http://labs.tn38.net/ajax_seo/company/2/
- http://labs.tn38.net/ajax_seo/company/3/
The above URL's are a rewritten version of:
- http://labs.tn38.net/ajax_seo/index.php?company=1
- http://labs.tn38.net/ajax_seo/index.php?company=2
- http://labs.tn38.net/ajax_seo/index.php?company=3
The mod_rewrite instructions that allow for this are as follows:
Options +FollowSymLinks
RewriteEngine on
RewriteRule ajax_seo/(.*)/(.*)/$
/ajax_seo/index.php?$1=$2
Although the real URL is a dynamically generated single page, the mod_rewrite instruction gives the appearance (to a search engine) that there is a directory with different and more importantly unique content at the destination.
Promotion and marketing of AJAX
This is the second important rule that needs to be followed in order for a great listing of the products or resources with an AJAX application.
Rule #2
As with all websites, we need inbound links which are keyphrase descriptive that point to a particular resource, not just the home page.
The root URL for the demo AJAX application is:
What we need is a list of links that point to relevant sections within this virtual directory that describe what the resource is and uses the static friendly address that mod_rewrite allows us. I have knocked up a quick directory for demo purposes.
Use the links supplied to test for Google search results for the phrases highlighted. What you'll see is results in Google that give us solid listings for resources within the AJAX application.
Result :)
Further reading
Useful?
Assistance?
Comments
Addam says:
Philip Chalmers says:
Very informative article - I've bookmarked it!
Thomas says:
Excellent Post. I love Ajax but SEO is a huge deal too.
Gwen says:
Very clearly written. Thank you.
Sonja Duijvesteijn says:
So, this comes down to 'make sure the pages still work without javascript' and use modrewrites, but doesn't have anything to do with ajax.
It's not that I don't agree but you could've taken a lot more interesting approach to this.
Harvey says:
Rule number 3...
Use traditional links with "onclick" AJAX behaviours returning false. When a javascript enabled browser clicks an ajax link, the "return false" tells the browser not to follow the link.
With Javascript off (ie google), it will ignore the "return false" and follow the link anyway - the link is of course to the static URL mentioned in rule 1.
Juan Diego says:
Nice tips, but i wonder Ajax will be a lot better if we can change the way the title tag is displayed, i mean, changing the title tag according to the data stream... just a idea..
Leonard says:
Ed, but how do you handle SEO for pages that are dynamically generated or built? What do you do if you do not have pages sitting in a directory to be pulled in?
rich says:
Instead of using 'onclick' and 'return false' behaviors in the HTML (ugly, and soils structural mark-up ((X)HTML) with behavioral enhancements (JS and the DOM)) wouldn't it be better to direct the links to the static pages (using href="linktostaticpage") and then over-ride that with an external JS function?
This way, with JS disabled (via Google eyes or if an actual user has it disabled), the links will still work (go to the static page); the Ajax component will provide unobtrusive 'enhanced' behavior without interfering with accessibilty or SEO.
Does this make sense? Am I missing something here?
Mudgal says:
Thanks a lot for the article. Was very useful to me.
I am in a dilemma, whether to use ASP.NET 1.1 or ASP.NET 2.0 to effectively use AJAX and URL rewriting. I am using 1.1 for last two years and I don't find any reason to upgrade to 2.0 apart from AJAX and URL Rewriting. FYI, I am using both these technologies in 1.1 only but was wondering whether 2.0 will be more helpful? Any insights will be appreciated. Thanks once again.
Chuck says:
We recently posted a different technique to communicate with the SE robots by using a browser detection script and pulling the page contents from the database for the SE to index. Thoughts/comments are welcome:
Tarek says:
I am not sure you are solving the problem. If you already have a page similar to http://labs.tn38.net/ajax_seo/index.php?company=1 then this page is spidered without hassle by the search engines. You are just rewriting the url, great if you have a lot of parameters in the url but again this is not the issue with ajax. Having the content displayed inside the same page real time using JS is similar to flash pages. I am sure the search engines will work hard to be able to adjust, afterall, its them who are looking for content. The browser detection method can be a solution but I am not sure it is completely white hat as it opens the way to cloaking.
Zaur says:
I found one advantages of Ajax for seo, please have a look at this page http://www.dhtmlgoodies.com/.../ajax-dynamic-pages.html
As you see you can keep big content on one page.
Daniel Skinner says:
Have you got any solutions to the following problem:
Consider a user using your Ore Valley site (which I believe is a complete solution to using AJAX whilst considering SEO): They find the electricians page by searching google and come to .../company/1/. They then make a few AJAX content requests and decide to bookmark the Plumbers page. When the come back to the bookmark it will return them to the Electrician page (/1) since the URL that was bookmarked was .../company/1/# instead of .../company/3/
I know it is possible to use Javascript to test for the url, say you were to change the url to .../company/1/#plumbers once an AJAX request for Plumbers was made. I see two problems with this method:
1. The page that will be loaded initially will be the electrician page and then we would have to do some url testing to load the correct content (although the page source will sill be electricians) .
2. The url .../company/1 represents the elctricians page. It seems wrong to be loading Plumber content from the Electricians page as the URL is indexed in google.
Wouldn't it be nice to be able to change the URL before the hash when a AJAX content load is made without the browser actually going to that page? As far as I know this is impossible.
Any solutions and we could complete the AJAX and permanent resource location problem.
Web says:
I hear google is starting to look into javascript, flash and css files now. I wonder when they might be making their way to Ajax searchable material!
WebGeek says:
Great points! I definitely agree about the initial states containing vital content. I would go farther and say that you should have a good portion of your content available on the initial load, and then the AJAX can switch it out. That way the SE's have some food. Also a great point about unique URL's...this is absolutely essential. I've got a few more thoughts on SEO and AJAX, if you're interested.
Standard Warfare says:
I think Ajax is a great future for web-development. The only downside previously was not having good Search Engine Optimization, but thats to your guide, this is no longer an issue. Thanks alot for this article!
Tom says:
Brilliant idea Ed. btw. I really enjoyed reading all of your articles. It’s interesting to read ideas, and observations from someone else’s point of view… makes you think more. Keep up the good work. Greetings
SEO says:
Slightly veering off the Ajax side for a second, Ed you mentioned "The reason for the rewrite is due to search engines reluctance to dig deep."
Just to further on the crawling aspect, there is also the user aspect. Rewritten CRU's (Content relevant URL's) not only bring more visitors, but targeted visitors as well as a tendency to gather more backlinks.. And here's why.
Targeted: A user browsing for Tomato Seeds will be far more inclined to click a URL somewhere on the web in the form of domain.com/tomato/seeds.html then they will domain.com?id=4&ref=8&cat=5&prod=7&opt=2&value=8
More Visitors: Similar reason, users especially inexperienced PC users are far more inclined to click something that "tells" them what is at the other end. I found out my mother wouldn't click these types of URL's because she thought they may be dangerous. (yeah i know)
Also other webmasters are able to give a neat looking backlink on their page without having to bother with anchor text.
So there's a few more reasons to rewrite URL's.
Although Ed, i'm curious why you chose underscores instead of hyphens in your URL's?
SEO says:
No probs, i was always curious of this when reading your blog/site. Google treats a hyphen as a seperator, and ignores underscores so it combines the 2 keywords.
So this page for instance, Google would see as ajaxandseo.html and you loose some keywords.
Mainly i use hyphens because when a URL is hyperlinked with an underscore it looks like a space. For example between the ajax and the SEO:
Finally your back, hope you had a good holiday!