[rfc-i] URIs in RFC references, was: feedback on draft-iab-styleguide-01

John Levine johnl at taugh.com
Wed Mar 26 11:15:48 PDT 2014


>> Nor will be a search engine.
>
>This is the biggest disadvantage for the default being "limit the amount 
>of text shown". (Whether it's done through cookies or login.)

It's absolutely common for web sites to look at the user agent string
and provide web spders versions of the web pages that include
everything you'd want to index.

Yes, this is a kludge, but it turns out to be a really simple and
effective one.  If you recognize Googlebot and bingbot and Slurp
(Yahoo) and maybe a few others, you'll seed all of the indexes anyone
cares about.

R's,
John


More information about the rfc-interest mailing list