Tuesday, June 29, 2010

Video Crawability Best Practices


Posting quality videos to your website is only half the battle. They don't provide much value if search engines have trouble crawling and indexing them. When you post a new video, it is important to cover all the bases to ensure search engines can find it.

One of the most basic steps to ensure your videos can be indexed is to verify that the video urls are crawlable. Placing the url in your sitemap is not enough if that url is blocked by your robots.txt file. Robots.txt is a file in the root folder of a website that can be used to tell search engines to ignore certain urls. Before placing your video url in your sitemap, review your robots.txt file to ensure that the url of the playpage, the url of the content and player, and the url of the video thumbnail are not disallowed.

Some videos are only available in certain countries, locales, etc. Google introduced a new 'restriction' attribute video tag to let search engines where a video is eligible to be played. This tag can be used to either tell Google in what countries a video can be played or in what countries a video cannot be played. Videos that can be played anywhere do not require this tag.

Finally, if you remove a video from your website, it is important to let the search engines know that you have done so. Without alerting search engines that a video is removed, the search engine's index may not match the content of your website. Therefore, if someone clicks on a video result on a search engine, when they arrive on your site they will either be told that the video no longer exists or be served a different video. This can lead to a bad user experience. The best way to handle this situation is to return a helpful 404 page in place of the video page. From there, you may recommend related videos that may be of interest to the user.