[Talossa.com portal]
[Citizens Toolbox]
[Immigration]
No Running elections

Parent message
  • robots.txt - - Does it work?
  • Gödafrïeu Válcadác’hJune 02, 2005 - 12:12
    yes and no...(#29), posted by Mr. Furxheir, R.M.W., [IP Hidden], June 02, 2005 - 13:45. Viewed 859 times.
    User InfoText
    Mr. Furxheir, R.M.W.
    Group: admins
    (1791 posts total)
    (last post: November 24, 2007 - 15:07)
    Citizen #20:
    Martí- Paìr Furxhéir
    > I find the following page from Google discussing how to make it so that Google spiders do not index your web page or any links on your web page.
    >
    > It can be had at http://www.google.com/intl/en/webmasters/faq.html#nocrawl
    >
    > It links to http://www.w3.org/TR/REC-html40/appendix/notes.html#h-B.4.1.2
    > where one may find a method of keeping spiders out via html.
    >
    > Are these methods reliable? Anyone have any thoughts about keeping Google away?
    >

    Yes, they work, but it doesn't remove old pages from the database.

    > What about putting code in .pdf documents to keep the spiders at bay?
    >

    That is impossible per se. You can ask, in robots.txt, the spiders not to spider your site, but if it is an old domain, it will remain online.


    > When I put Oraclâ back online, I want to make certain the site is immune to Google searches and other search engines.
    >


    Want the best solution ? Password protect the directory. Your host has a password protection system in it's Control panel which allows you to assign username and passwords ( or a common one ).

    That way, you can control who accesses your site and who doesn't.
    > Many thanks, everyone!
    >
    > GV


    This is a shorter sig. Long sigs are as annoying as fuck.
    yes and no...Mr. Furxheir, R.M.W.June 02, 2005 - 13:45
    Túischac’h Txec EreufighleuOctober 23, 2005 - 11:00
    Reply to this post | Back to the forum | Forums Overview| Activate Thread View
    Forums Overview | Login | Register | Lost your password? Cyphor (Release: 0.19, PHP 5.2.5)