One of the things search engines are looking for today is originality. They want to serve searchers the original content and not give search results that are nothing but copy cat sites. For the most part this is a good thing, but there is one problem. Because of the way search engines see URL's they will mistake http://www.mysite.com and http://mysite.com as two different websites with the exact same content, thus giving both versions of the the URL demerits in their search score.

Here are some other examples URL's that search engines will see as duplicate content

So how do we fix this? We use Canonical URL's. The following steps ensure that the major search engines will see your site as original content.

1. Set your preferred domain in your webmasters account

You will have to manualy log into your Google and Bing webmasters account and tell them that http://www.my-seo-site.com and http://my-seo-site.com are the same.

2. Indicate the preferred URL with the rel="canonical" link element

Suppose you want http://www.my-seo-site.com/articles/cononical-urls  to be the preferred URL, even though a variety of URLs can access this content. You can indicate this to search engines as by putting the following in the of your page:

3. Use 301 redirects for URLs that are not canonical

Add the following to your '.htaccess' file in your sites root directory

RewriteEngine on
RewriteBase /
RewriteCond %{HTTP_HOST} ^my-seo-site.com [NC]
RewriteRule ^(.*)$ http://www.my-seo-site.com/$1 [L,R=301]
RewriteRule ^index.html$ http://www.my-seo-site.com/ [R=301,L]
RewriteRule ^index.php$ http://www.my-seo-site.com/ [R=301,L]

4. Use a site map to tell search engines how to crawl your site

If your CMS does not already do so there are many free site map generators out there that will make your sitemap.xml file for you. Once you have your sitemap.xml file you will put it into the root directory of your site
Then log into your Google and Bing webmasters accounts and upload your sitemap with the prefered urls for each page on your site

5. Indicate how to handle dynamic parameters

Suppose your CMS uses session id's in the URL's such as this http://www.my-seo-site.com/Articles/Canonical-urls?session=y109918ypoihf98 you can tell Google and Bing to ignore the variable session in your webmasters account. Now this link will always be seen as http://www.my-seo-site.com/Articles/Canonical-urls

6. Specify a canonical link in your HTTP header

In this case, you can use a rel="canonical" HTTP header to specify to Google the canonical URL for the PDF file, as follows:

Google currently supports these link header elements for Web Search only.