[ List Archives Home ] [ Thread index for 2008 ]
[ Date index for 2008 ]
[ Author index for 2008 ]
[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
- Date: Mon, 13 Feb 2006 11:30:54 -0500
- From: "Adam Brin" <abrin at brynmawr dot edu>
- Subject: [IUG] Testing Server Missing robots.txt
Hi All,
We've been analyzing our search statistics locally and came across a
lot of anomalous data issues ... turns out, that the Testing Server is
missing a robots.txt file and Google, Yahoo, and MSN were all indexing our
III server, and throwing off our search stats. We have since added a
robots.txt file outside of the staging directory ie <server>:2082/robots.txt
with the following contents:
User-agent: *
Disallow: /
#this should tell all search engines to ignore the testing server.
Hopefully this is helpful to others,
adam
_____________________________________
Tri-Colleges Systems Coordinator
Bryn Mawr | Haverford | Swarthmore
610.526.5294
Home
Mailing list