fthead9
12-22-2005, 01:45 PM
I came across this article describing a solution for getting dynamic pages indexed. According to the article this was a better/faster solution than using rewrites. Here is the excerp:
"There is a third method, which we call proxy serving. It offers unique benefits over previously discussed approaches. Proxy serving is a style of hosted URL rewriting that marketers such as Sara Lee Direct, Northern Tool, and Ritz Camera have been experimenting with and seeing promising results from. Proxy serving relies not on suboptimal static page forms or draining scarce IT resources to reconfigure the Website. Rather, it acts as a middleman, intercepting pages being requested by search spiders and then delivering crawler-friendly versions of the pages. When a searcher finds and clicks on a listing for a particular product, the proxy server requests the corresponding page from the native Website in real-time and serves this page to the user.
This approach bridges the gap by ensuring that the indexed pages retain the integrity of your site's internal hierarchical linking structure, resulting in greater PageRank “flow.” Since these pages are not static but served in real-time as they are requested, any indexed pages are always in sync with your online catalog once they are clicked on by Google or Yahoo! searchers; as a result, those customers receive the latest pricing and inventory availability data."
The example site, Northern Tool, has over 70,000 pages indexed in Yahoo and 500,000 listed in Google. From the description it sounds to me they are serving a dynamic page to searchers but a static one for spiders, seems to be cloaking to me. Any thoughts on this technique and how they manage to avoid penalties. I have a retailer client that is redesigning their site in 2006 and we want to get all the dynamic catalog pages indexed. We've used ISAPI_rewrite on it before but I'm curious about this "new" alternative.
"There is a third method, which we call proxy serving. It offers unique benefits over previously discussed approaches. Proxy serving is a style of hosted URL rewriting that marketers such as Sara Lee Direct, Northern Tool, and Ritz Camera have been experimenting with and seeing promising results from. Proxy serving relies not on suboptimal static page forms or draining scarce IT resources to reconfigure the Website. Rather, it acts as a middleman, intercepting pages being requested by search spiders and then delivering crawler-friendly versions of the pages. When a searcher finds and clicks on a listing for a particular product, the proxy server requests the corresponding page from the native Website in real-time and serves this page to the user.
This approach bridges the gap by ensuring that the indexed pages retain the integrity of your site's internal hierarchical linking structure, resulting in greater PageRank “flow.” Since these pages are not static but served in real-time as they are requested, any indexed pages are always in sync with your online catalog once they are clicked on by Google or Yahoo! searchers; as a result, those customers receive the latest pricing and inventory availability data."
The example site, Northern Tool, has over 70,000 pages indexed in Yahoo and 500,000 listed in Google. From the description it sounds to me they are serving a dynamic page to searchers but a static one for spiders, seems to be cloaking to me. Any thoughts on this technique and how they manage to avoid penalties. I have a retailer client that is redesigning their site in 2006 and we want to get all the dynamic catalog pages indexed. We've used ISAPI_rewrite on it before but I'm curious about this "new" alternative.