Launchpad itself

dogfood.launchpad.net pages are showing up in Web search engines

Bug #209090 reported by Matthew Paul Thomas
6
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Unassigned

Bug Description

<http://www.google.com/search?q=site%3Adogfood.launchpad.net> and <http://siteexplorer.search.yahoo.com/search?p=http%3A%2F%2Fdogfood.launchpad.net> show dogfood.launchpad.net pages.

This should be prevented, probably using robots.txt.

(The robots.txt currently used for staging.launchpad.net isn't quite doing the job properly: <http://www.google.com/search?q=site:staging.launchpad.net&hl=en&filter=0>.)

Tags: lp-soyuz
Revision history for this message
Julian Edwards (julian-edwards) wrote :

I can fix this if you can advise on how/where to put a robots.txt.

The site crawlers are a royal PITA when you're trying to test, too.

Revision history for this message
Scott Severance (scott.severance) wrote :

Put something like the following in *.dogfood.launchpad.net/robots.txt:

User-agent: *
Disallow: /

Curtis Hovey (sinzui)
affects: launchpad-foundations → soyuz
Julian Edwards (julian-edwards)
Changed in soyuz:
status: New → Triaged
importance: Undecided → High
Julian Edwards (julian-edwards)
Changed in soyuz:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.