  |
|
Meta Crawlers Indexrobot-name: image.kapsi.net robot-cover-url: http://image.kapsi.net/ robot-details-url: http://image.kapsi.net/index.php?page=robot robot-owner-name: Jaakko Heusala robot-owner-url: http://huoh.kapsi.net/ robot-owner-email: Jaakko.Heusala@kapsi.net robot-status: development robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: data robot-exclusion: yes robot-exclusion-useragent: image.kapsi.net robot-noindex: no robot-host: addr-212-50-142-138.suomi.net robot-from: yes robot-useragent: image.kapsi.net/1.0 robot-language: perl robot-description: The image.kapsi.net robot is used to build the database for the image.kapsi.net search service. The robot runs currently in a random times. robot-history: The Robot was build for image.kapsi.net's database in year 2001. robot-environment: hobby, research modified-date: Thu, 13 Dec 2001 23:28:23 EET modified-by: robot-name: Katipo robot-cover-url: http://www.vuw.ac.nz/~newbery/Katipo.html robot-details-url: http://www.vuw.ac.nz/~newbery/Katipo/Katipo-doc.html robot-owner-name: Michael Newbery robot-owner-url: http://www.vuw.ac.nz/~newbery robot-owner-email: Michael.Newbery@vuw.ac.nz robot-status: active robot-purpose: maintenance robot-type: standalone robot-platform: Macintosh robot-availability: binary robot-exclusion: no robot-exclusion-useragent: robot-noindex: no robot-host: * robot-from: yes robot-useragent: Katipo/1.0 robot-language: c robot-description: Watches all the pages you have previously visited and tells you when they have changed. robot-history: robot-environment: commercial (free) modified-date: Tue, 25 Jun 96 11:40:07 +1200 modified-by: Michael Newbery robot-name: KDD-Explorer robot-cover-url: http://mlc.kddvw.kcom.or.jp/CLINKS/html/clinks.html robot-details-url: not available robot-owner-name: Kazunori Matsumoto robot-owner-url: not available robot-owner-email: matsu@lab.kdd.co.jp robot-status: development (to be avtive in June 1997) robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent:KDD-Explorer robot-noindex: no robot-host: mlc.kddvw.kcom.or.jp robot-from: yes robot-useragent: KDD-Explorer/0.1 robot-language: c robot-description: KDD-Explorer is used for indexing valuable documents which will be retrieved via an experimental cross-language search engine, CLINKS. robot-history: This robot was designed in Knowledge-bases Information processing Laboratory, KDD R&D Laboratories, 1996-1997 robot-environment: research modified-date: Mon, 2 June 1997 18:00:00 JST modified-by: Kazunori Matsumoto robot-name:Kilroy robot-cover-url:http://purl.org/kilroy robot-details-url:http://purl.org/kilroy robot-owner-name:OCLC robot-owner-url:http://www.oclc.org robot-owner-email:kilroy@oclc.org robot-status:active robot-purpose:indexing,statistics robot-type:standalone robot-platform:unix,windowsNT robot-availability:none robot-exclusion:yes robot-exclusion-useragent:* robot-noindex:no robot-host:*.oclc.org robot-from:no robot-useragent:yes robot-language:java robot-description:Used to collect data for several projects. Runs constantly and visits site no faster than once every 90 seconds. robot-history:none robot-environment:research,service modified-date:Thursday, 24 Apr 1997 20:00:00 GMT modified-by:tkac robot-name: KO_Yappo_Robot robot-cover-url: http://yappo.com/info/robot.html robot-details-url: http://yappo.com/ robot-owner-name: Kazuhiro Osawa robot-owner-url: http://yappo.com/ robot-owner-email: office_KO@yappo.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: ko_yappo_robot robot-noindex: yes robot-host: yappo.com,209.25.40.1 robot-from: yes robot-useragent: KO_Yappo_Robot/1.0.4(http://yappo.com/info/robot.html) robot-language: perl robot-description: The KO_Yappo_Robot robot is used to build the database for the Yappo search service by k,osawa (part of AOL). The robot runs random day, and visits sites in a random order. robot-history: The robot is hobby of k,osawa at the Tokyo in 1997 robot-environment: hobby modified-date: Fri, 18 Jul 1996 12:34:21 GMT modified-by: KO robot-name: LabelGrabber robot-cover-url: http://www.w3.org/PICS/refcode/LabelGrabber/index.htm robot-details-url: http://www.w3.org/PICS/refcode/LabelGrabber/index.htm robot-owner-name: Kyle Jamieson robot-owner-url: http://www.w3.org/PICS/refcode/LabelGrabber/index.htm robot-owner-email: jamieson@mit.edu robot-status: active robot-purpose: Grabs PICS labels from web pages, submits them to a label bueau robot-type: standalone robot-platform: windows, windows95, windowsNT, unix robot-availability: source robot-exclusion: yes robot-exclusion-useragent: label-grabber robot-noindex: no robot-host: head.w3.org robot-from: no robot-useragent: LabelGrab/1.1 robot-language: java robot-description: The label grabber searches for PICS labels and submits them to a label bureau robot-history: N/A robot-environment: research modified-date: Wed, 28 Jan 1998 17:32:52 GMT modified-by: jamieson@mit.edu robot-name: larbin robot-cover-url: http://para.inria.fr/~ailleret/larbin/index-eng.html robot-owner-name: Sebastien Ailleret robot-owner-url: http://para.inria.fr/~ailleret/ robot-owner-email: sebastien.ailleret@inria.fr robot-status: active robot-purpose: Your imagination is the only limit robot-type: standalone robot-platform: Linux robot-availability: source (GPL), mail me for customization robot-exclusion: yes robot-exclusion-useragent: larbin robot-noindex: no robot-host: * robot-from: no robot-useragent: larbin (+mail) robot-language: c++ robot-description: Parcourir le web, telle est ma passion robot-history: french research group (INRIA Verso) robot-environment: hobby modified-date: 2000-3-28 modified-by: Sebastien Ailleret robot-name: legs robot-cover-url: http://www.MagPortal.com/ robot-details-url: robot-owner-name: Bill Dimm robot-owner-url: http://www.HotNeuron.com/ robot-owner-email: admin@magportal.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: linux robot-availability: none robot-exclusion: yes robot-exclusion-useragent: legs robot-noindex: no robot-host: robot-from: yes robot-useragent: legs robot-language: perl5 robot-description: The legs robot is used to build the magazine article database for MagPortal.com. robot-history: robot-environment: service modified-date: Wed, 22 Mar 2000 14:10:49 GMT modified-by: Bill Dimm robot-name: Link Validator robot-cover-url: robot-details-url: robot-owner-name: Thomas Gimon robot-owner-url: robot-owner-email: tgimon@mitre.org robot-status: development robot-purpose: maintenance robot-type: standalone robot-platform: unix, windows robot-availability: none robot-exclusion: yes robot-exclusion-useragent: Linkidator robot-noindex: yes robot-nofollow: yes robot-host: *.mitre.org robot-from: yes robot-useragent: Linkidator/0.93 robot-language: perl5 robot-description: Recursively checks all links on a site, looking for broken or redirected links. Checks all off-site links using HEAD requests and does not progress further. Designed to behave well and to be very configurable. robot-history: Built using WWW-Robot-0.022 perl module. Currently in beta test. Seeking approval for public release. robot-environment: internal modified-date: Fri, 20 Jan 2001 02:22:00 EST modified-by: Thomas Gimon robot-name:LinkScan robot-cover-url:http://www.elsop.com/ robot-details-url:http://www.elsop.com/linkscan/overview.html robot-owner-name:Electronic Software Publishing Corp. (Elsop) robot-owner-url:http://www.elsop.com/ robot-owner-email:sales@elsop.com robot-status:Robot actively in use robot-purpose:Link checker, SiteMapper, and HTML Validator robot-type:Standalone robot-platform:Unix, Linux, Windows 98/NT robot-availability:Program is shareware robot-exclusion:No robot-exclusion-useragent: robot-noindex:Yes robot-host:* robot-from: robot-useragent:LinkScan Server/5.5 | LinkScan Workstation/5.5 robot-language:perl5 robot-description:LinkScan checks links, validates HTML and creates site maps robot-history: First developed by Elsop in January,1997 robot-environment:Commercial modified-date:Fri, 3 September 1999 17:00:00 PDT modified-by: Kenneth R. Churilla robot-name: LinkWalker robot-cover-url: http://www.seventwentyfour.com robot-details-url: http://www.seventwentyfour.com/tech.html robot-owner-name: Roy Bryant robot-owner-url: robot-owner-email: rbryant@seventwentyfour.com robot-status: active robot-purpose: maintenance, statistics robot-type: standalone robot-platform: windowsNT robot-availability: none robot-exclusion: yes robot-exclusion-useragent: linkwalker robot-noindex: yes robot-host: *.seventwentyfour.com robot-from: yes robot-useragent: LinkWalker robot-language: c++ robot-description: LinkWalker generates a database of links. We send reports of bad ones to webmasters. robot-history: Constructed late 1997 through April 1998. In full service April 1998. robot-environment: service modified-date: Wed, 22 Apr 1998 modified-by: Roy Bryant robot-name:Lockon robot-cover-url: robot-details-url: robot-owner-name:Seiji Sasazuka & Takahiro Ohmori robot-owner-url: robot-owner-email:search@rsch.tuis.ac.jp robot-status:active robot-purpose:indexing robot-type:standalone robot-platform:UNIX robot-availability:none robot-exclusion:yes robot-exclusion-useragent:Lockon robot-noindex:yes robot-host:*.hitech.tuis.ac.jp robot-from:yes robot-useragent:Lockon/xxxxx robot-language:perl5 robot-description:This robot gathers only HTML document. robot-history:This robot was developed in the Tokyo university of information sciences in 1998. robot-environment:research modified-date:Tue. 10 Nov 1998 20:00:00 GMT modified-by:Seiji Sasazuka & Takahiro Ohmori robot-name: logo.gif Crawler robot-cover-url: http://www.inm.de/projects/logogif.html robot-details-url: robot-owner-name: Sevo Stille robot-owner-url: http://www.inm.de/people/sevo robot-owner-email: sevo@inm.de robot-status: under development robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: logo_gif_crawler robot-noindex: no robot-host: *.inm.de robot-from: yes robot-useragent: logo.gif crawler robot-language: perl robot-description: meta-indexing engine for corporate logo graphics The robot runs at irregular intervals and will only pull a start page and its associated /.*logo\.gif/i (if any). It will be terminated once a statistically significant number of samples has been collected. robot-history: logo.gif is part of the design diploma of Markus Weisbeck, and tries to analyze the abundance of the logo metaphor in WWW corporate design. The crawler and image database were written by Sevo Stille and Peter Frank of the Institut für Neue Medien, respectively. robot-environment: research, statistics modified-date: 25.5.97 modified-by: Sevo Stille robot-name: Lycos robot-cover-url: http://lycos.cs.cmu.edu/ robot-details-url: robot-owner-name: Dr. Michael L. Mauldin robot-owner-url: http://fuzine.mt.cs.cmu.edu/mlm/home.html robot-owner-email: fuzzy@cmu.edu robot-status: robot-purpose: indexing robot-type: robot-platform: robot-availability: robot-exclusion: yes robot-exclusion-useragent: robot-noindex: no robot-host: fuzine.mt.cs.cmu.edu, lycos.com robot-from: robot-useragent: Lycos/x.x robot-language: robot-description: This is a research program in providing information retrieval and discovery in the WWW, using a finite memory model of the web to guide intelligent, directed searches for specific information needs robot-history: robot-environment: modified-date: modified-by: robot-name: Mac WWWWorm robot-cover-url: robot-details-url: robot-owner-name: Sebastien Lemieux robot-owner-url: robot-owner-email: lemieuse@ERE.UMontreal.CA robot-status: robot-purpose: indexing robot-type: robot-platform: Macintosh robot-availability: none robot-exclusion: robot-exclusion-useragent: robot-noindex: no robot-host: robot-from: robot-useragent: robot-language: hypercard robot-description: a French Keyword-searching robot for the Mac The author has decided not to release this robot to the public robot-history: robot-environment: modified-date: modified-by: robot-name: Magpie robot-cover-url: robot-details-url: robot-owner-name: Keith Jones robot-owner-url: robot-owner-email: Keith.Jones@blueberry.co.uk robot-status: development robot-purpose: indexing, statistics robot-type: standalone robot-platform: unix robot-availability: robot-exclusion: no robot-exclusion-useragent: robot-noindex: no robot-host: *.blueberry.co.uk, 194.70.52.*, 193.131.167.144 robot-from: no robot-useragent: Magpie/1.0 robot-language: perl5 robot-description: Used to obtain information from a specified list of web pages for local indexing. Runs every two hours, and visits only a small number of sites. robot-history: Part of a research project. Alpha testing from 10 July 1996, Beta testing from 10 September. robot-environment: research modified-date: Wed, 10 Oct 1996 13:15:00 GMT modified-by: Keith Jones robot-name: marvin/infoseek robot-details-url: robot-cover-url: http://www.infoseek.de/ robot-owner-name: WSI Webseek Infoservice GmbH & Co KG. robot-owner-url: http://www.infoseek.de/ robot-owner-email: marvin-team@webseek.de robot-status: development robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: marvin robot-noindex: yes robot-nofollow: yes robot-host: arthur*.sda.t-online.de robot-from: yes robot-useragent: marvin/infoseek (marvin-team@webseek.de) robot-language: java robot-description: robot-history: day of birth: 4.2. 2001 - replaces Infoseek Sidewinder robot-environment: comercial modified-date: Fri, 11 May 2001 17:28:52 GMT robot-name: Mattie robot-cover-url: http://www.mcw.aarkayn.org robot-details-url: http://www.mcw.aarkayn.org/web/mattie.asp robot-owner-name: Matt robot-owner-url: http://www.mcw.aarkayn.org robot-owner-email: matt@mcw.aarkayn.org robot-status: Active robot-purpose: Procurement Spider robot-type: Standalone robot-platform: UNIX robot-availability: None robot-exclusion: Yes robot-exclusion-useragent: mattie robot-noindex: N/A robot-nofollow: Yes robot-host: mattie.mcw.aarkayn.org robot-from: Yes robot-useragent: M/3.8 robot-language: C++ robot-description: Mattie is an all-source procurement spider. robot-history: Created 2000 Mar. 03 Fri. 18:48:16 -0500 GMT (R) as an MP3 spider, Mattie was reborn 2002 Jul. 07 Sun. 03:47:29 -0500 GMT (R) as an all-source procurement spider. robot-environment: Hobby modified-date: Fri, 13 Sep 2002 00:36:13 GMT modified-by: Matt robot-name: MediaFox robot-cover-url: none robot-details-url: none robot-owner-name: Lars Eilebrecht robot-owner-url: http://www.home.unix-ag.org/sfx/ robot-owner-email: sfx@uni-media.de robot-status: development robot-purpose: indexing and maintenance robot-type: standalone robot-platform: (Java) robot-availability: none robot-exclusion: yes robot-exclusion-useragent: mediafox robot-noindex: yes robot-host: 141.99.*.* robot-from: yes robot-useragent: MediaFox/x.y robot-language: Java robot-description: The robot is used to index meta information of a specified set of documents and update a database accordingly. robot-history: Project at the University of Siegen robot-environment: research modified-date: Fri Aug 14 03:37:56 CEST 1998 modified-by: Lars Eilebrecht robot-name:MerzScope robot-cover-url:http://www.merzcom.com robot-details-url:http://www.merzcom.com robot-owner-name:(Client based robot) robot-owner-url:(Client based robot) robot-owner-email: robot-status:actively in use robot-purpose:WebMapping robot-type:standalone robot-platform: (Java Based) unix,windows95,windowsNT,os2,mac etc .. robot-availability:binary robot-exclusion: yes robot-exclusion-useragent: MerzScope robot-noindex: no robot-host:(Client Based) robot-from: robot-useragent: MerzScope robot-language: java robot-description: Robot is part of a Web-Mapping package called MerzScope, to be used mainly by consultants, and web masters to create and publish maps, on and of the World wide web. robot-history: robot-environment: modified-date: Fri, 13 March 1997 16:31:00 modified-by: Philip Lenir, MerzScope lead developper robot-name: NEC-MeshExplorer robot-cover-url: http://netplaza.biglobe.or.jp/ robot-details-url: http://netplaza.biglobe.or.jp/keyword.html robot-owner-name: web search service maintenance group robot-owner-url: http://netplaza.biglobe.or.jp/keyword.html robot-owner-email: web-dir@mxa.meshnet.or.jp robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: NEC-MeshExplorer robot-noindex: no robot-host: meshsv300.tk.mesh.ad.jp robot-from: yes robot-useragent: NEC-MeshExplorer robot-language: c robot-description: The NEC-MeshExplorer robot is used to build database for the NETPLAZA search service operated by NEC Corporation. The robot searches URLs around sites in japan(JP domain). The robot runs every day, and visits sites in a random order. robot-history: Prototype version of this robot was developed in C&C Research Laboratories, NEC Corporation. Current robot (Version 1.0) is based on the prototype and has more functions. robot-environment: research modified-date: Jan 1, 1997 modified-by: Nobuya Kubo, Hajime Takano robot-name: MindCrawler robot-cover-url: http://www.mindpass.com/_technology_faq.htm robot-details-url: robot-owner-name: Mindpass robot-owner-url: http://www.mindpass.com/ robot-owner-email: support@mindpass.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: linux robot-availability: none robot-exclusion: yes robot-exclusion-useragent: MindCrawler robot-noindex: no robot-host: * robot-from: no robot-useragent: MindCrawler robot-language: c++ robot-description: robot-history: robot-environment: modified-date: Tue Mar 28 11:30:09 CEST 2000 modified-by: robot-name: mnoGoSearch search engine software robot-cover-url: http://www.mnogosearch.org robot-details-url: http://www.mnogosearch.org/features.html robot-owner-name: Lavtech.com corp. robot-owner-url: http://www.mnogosearch.org robot-owner-email: support@mnogosearch.org robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix, windows, mac robot-availability: source robot-exclusion: yes robot-exclusion-useragent: udmsearch robot-noindex: yes robot-host: * robot-from: no robot-useragent: UdmSearch robot-language: c robot-description: mnoGoSearch search engine software (formerly known as UDMSearch) is an advanced search solution for large-scale websites and Intranet. It is based on SQL database and supports numerous features. robot-history: Formerly known as UDMSearch was developed as the search engine for the Russian republic of Udmurtia. robot-environment: commercial modified-date: Wed, 12 Sept 2001 modified-by: Dmitry Tkatchenko robot-name:moget robot-cover-url: robot-details-url: robot-owner-name:NTT-ME Infomation Xing,Inc robot-owner-url:http://www.nttx.co.jp robot-owner-email:moget@goo.ne.jp robot-status:active robot-purpose:indexing,statistics robot-type:standalone robot-platform:unix robot-availability:none robot-exclusion:yes robot-exclusion-useragent:moget robot-noindex:yes robot-host:*.goo.ne.jp robot-from:yes robot-useragent:moget/1.0 robot-language:c robot-description: This robot is used to build the database for the search service operated by goo robot-history: robot-environment:service modified-date:Thu, 30 Mar 2000 18:40:37 GMT modified-by:moget@goo.ne.jp robot-name: MOMspider robot-cover-url: http://www.ics.uci.edu/WebSoft/MOMspider/ robot-details-url: robot-owner-name: Roy T. Fielding robot-owner-url: http://www.ics.uci.edu/dir/grad/Software/fielding robot-owner-email: fielding@ics.uci.edu robot-status: active robot-purpose: maintenance, statistics robot-type: standalone robot-platform: UNIX robot-availability: source robot-exclusion: yes robot-exclusion-useragent: robot-noindex: no robot-host: * robot-from: yes robot-useragent: MOMspider/1.00 libwww-perl/0.40 robot-language: perl 4 robot-description: to validate links, and generate statistics. It's usually run from anywhere robot-history: Originated as a research project at the University of California, Irvine, in 1993. Presented at the First International WWW Conference in Geneva, 1994. robot-environment: modified-date: Sat May 6 08:11:58 1995 modified-by: fielding@ics.uci.edu robot-name: Monster robot-cover-url: http://www.neva.ru/monster.list/russian.www.html robot-details-url: robot-owner-name: Dmitry Dicky robot-owner-url: http://wild.stu.neva.ru/ robot-owner-email: diwil@wild.stu.neva.ru robot-status: active robot-purpose: maintenance, mirroring robot-type: standalone robot-platform: UNIX (Linux) robot-availability: binary robot-exclusion: yes robot-exclusion-useragent: robot-noindex: no robot-host: wild.stu.neva.ru robot-from: robot-useragent: Monster/vX.X.X -$TYPE ($OSTYPE) robot-language: C robot-description: The Monster has two parts - Web searcher and Web analyzer. Searcher is intended to perform the list of WWW sites of desired domain (for example it can perform list of all WWW sites of mit.edu, com, org, etc... domain) In the User-agent field $TYPE is set to 'Mapper' for Web searcher and 'StAlone' for Web analyzer. robot-history: Now the full (I suppose) list of ex-USSR sites is produced. robot-environment: modified-date: Tue Jun 25 10:03:36 1996 modified-by: robot-name: Motor robot-cover-url: http://www.cybercon.de/Motor/index.html robot-details-url: robot-owner-name: Mr. Oliver Runge, Mr. Michael Goeckel robot-owner-url: http://www.cybercon.de/index.html robot-owner-email: Motor@cybercon.technopark.gmd.de robot-status: developement robot-purpose: indexing robot-type: standalone robot-platform: mac robot-availability: data robot-exclusion: yes robot-exclusion-useragent: Motor robot-noindex: no robot-host: Michael.cybercon.technopark.gmd.de robot-from: yes robot-useragent: Motor/0.2 robot-language: 4th dimension robot-description: The Motor robot is used to build the database for the www.webindex.de search service operated by CyberCon. The robot ios under development - it runs in random intervals and visits site in a priority driven order (.de/.ch/.at first, root and robots.txt first) robot-history: robot-environment: service modified-date: Wed, 3 Jul 1996 15:30:00 +0100 modified-by: Michael Goeckel (Michael@cybercon.technopark.gmd.de) robot-name: MSNBot robot-cover-url: http://search.msn.com robot-details-url: http://search.msn.com/msnbot.htm robot-owner-name: Microsoft Corp. robot-owner-url: http://www.microsoft.com robot-owner-email: msnbot@microsoft.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: Windows Server 2000, Windows Server 2003 robot-availability: none robot-exclusion: yes robot-exclusion-useragent: msnbot robot-noindex: yes robot-host: <TBD> robot-from: yes robot-useragent: MSNBOT/0.1 (http://search.msn.com/msnbot.htm) robot-language: C++ robot-description: MSN Search Crawler robot-history: Developed by Microsoft Corp. robot-environment: commercial modified-date: June 23, 2003 modified-by: msnbot@microsoft.com robot-name: Muncher robot-details-url: http://www.goodlookingcooking.co.uk/info.htm robot-cover-url: http://www.goodlookingcooking.co.uk robot-owner-name: Chris Ridings robot-owner-url: http://www.goodlookingcooking.co.uk robot-owner-email: muncher@ridings.org.uk robot-status: development robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: muncher robot-noindex: yes robot-nofollow: yes robot-host: www.goodlookingcooking.co.uk robot-from: no robot-useragent: yes robot-language: perl robot-description: Used to build the index for www.goodlookingcooking.co.uk. Seeks out cooking and recipe pages. robot-history: Private project september 2001 robot-environment: hobby modified-date: Wed, 5 Sep 2001 19:21:00 GMT robot-name: Muscat Ferret robot-cover-url: http://www.muscat.co.uk/euroferret/ robot-details-url: robot-owner-name: Olly Betts robot-owner-url: http://www.muscat.co.uk/~olly/ robot-owner-email: olly@muscat.co.uk robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: MuscatFerret robot-noindex: yes robot-host: 193.114.89.*, 194.168.54.11 robot-from: yes robot-useragent: MuscatFerret/<version> robot-language: c, perl5 robot-description: Used to build the database for the EuroFerret <URL:http://www.muscat.co.uk/euroferret/> robot-history: robot-environment: service modified-date: Tue, 21 May 1997 17:11:00 GMT modified-by: olly@muscat.co.uk robot-name: Mwd.Search robot-cover-url: (none) robot-details-url: (none) robot-owner-name: Antti Westerberg robot-owner-url: (none) robot-owner-email: Antti.Westerberg@mwd.sci.fi robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix (Linux) robot-availability: none robot-exclusion: yes robot-exclusion-useragent: MwdSearch robot-noindex: yes robot-host: *.fifi.net robot-from: no robot-useragent: MwdSearch/0.1 robot-language: perl5, c robot-description: Robot for indexing finnish (toplevel domain .fi) webpages for search engine called Fifi. Visits sites in random order. robot-history: (none) robot-environment: service (+ commercial)mwd.sci.fi> modified-date: Mon, 26 May 1997 15:55:02 EEST modified-by: Antti.Westerberg@mwd.sci.fi
|
| |||||||||||||||||||||||||||||
SSL
