  |
|
Web Crawlersrobot-name: CyberSpyder Link Test robot-cover-url: http://www.cyberspyder.com/cslnkts1.html robot-details-url: http://www.cyberspyder.com/cslnkts1.html robot-owner-name: Tom Aman robot-owner-url: http://www.cyberspyder.com/ robot-owner-email: amant@cyberspyder.com robot-status: active robot-purpose: link validation, some html validation robot-type: standalone robot-platform: windows 3.1x, windows95, windowsNT robot-availability: binary robot-exclusion: user configurable robot-exclusion-useragent: cyberspyder robot-noindex: no robot-host: * robot-from: no robot-useragent: CyberSpyder/2.1 robot-language: Microsoft Visual Basic 4.0 robot-description: CyberSpyder Link Test is intended to be used as a site management tool to validate that HTTP links on a page are functional and to produce various analysis reports to assist in managing a site. robot-history: The original robot was created to fill a widely seen need for a easy to use link checking program. robot-environment: commercial modified-date: Tue, 31 Mar 1998 01:02:00 GMT modified-by: Tom Aman robot-name: Desert Realm Spider robot-cover-url: http://www.desertrealm.com robot-details-url: http://spider.desertrealm.com robot-owner-name: Brian B. robot-owner-url: http://www.desertrealm.com robot-owner-email: spider@desertrealm.com robot-status: robot actively in use robot-purpose: indexing robot-type: standalone robot-platform: cross platform robot-availability: none robot-exclusion: yes robot-exclusion-useragent: desertrealm, desert realm robot-noindex: yes robot-nofollow: yes robot-host: * robot-from: no robot-useragent: DesertRealm.com; 0.2; [J]; robot-language: java 1.3, java 1.4 robot-description: The spider indexes fantasy and science fiction sites by using a customizable keyword algorithm. Only home pages are indexed, but all pages are looked at for links. Pages are visited randomly to limit impact on any one webserver. robot-history: The spider originally was created to learn more about how search engines work. robot-environment: hobby modified-date: Fri, 19 Sep 2003 08:57:52 GMT modified-by: Brian B. robot-name: DeWeb(c) Katalog/Index robot-cover-url: http://deweb.orbit.de/ robot-details-url: robot-owner-name: Marc Mielke robot-owner-url: http://www.orbit.de/ robot-owner-email: dewebmaster@orbit.de robot-status: robot-purpose: indexing, mirroring, statistics robot-type: standalone robot-platform: robot-availability: robot-exclusion: yes robot-exclusion-useragent: robot-noindex: no robot-host: deweb.orbit.de robot-from: yes robot-useragent: Deweb/1.01 robot-language: perl 4 robot-description: Its purpose is to generate a Resource Discovery database, perform mirroring, and generate statistics. Uses combination of Informix(tm) Database and WN 1.11 serversoftware for indexing/ressource discovery, fulltext search, text excerpts. robot-history: robot-environment: modified-date: Wed Jan 10 08:23:00 1996 modified-by: robot-name: DienstSpider robot-cover-url: http://sappho.csi.forth.gr:22000/ robot-details-url: robot-owner-name: Antonis Sidiropoulos robot-owner-url: http://www.csi.forth.gr/~asidirop robot-owner-email: asidirop@csi.forth.gr robot-status: development robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: robot-exclusion-useragent: robot-noindex: robot-host: sappho.csi.forth.gr robot-from: robot-useragent: dienstspider/1.0 robot-language: C robot-description: Indexing and searching the NCSTRL(Networked Computer Science Technical Report Library) and ERCIM Collection robot-history: The version 1.0 was the developer's master thesis project robot-environment: research modified-date: Fri, 4 Dec 1998 0:0:0 GMT modified-by: asidirop@csi.forth.gr robot-name: Digger robot-cover-url: http://www.diggit.com/ robot-details-url: robot-owner-name: Benjamin Lipchak robot-owner-url: robot-owner-email: admin@bulldozersoftware.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix, windows robot-availability: none robot-exclusion: yes robot-exclusion-useragent: digger robot-noindex: yes robot-host: robot-from: yes robot-useragent: Digger/1.0 JDK/1.3.0 robot-language: java robot-description: indexing web sites for the Diggit! search engine robot-history: robot-environment: service modified-date: modified-by: robot-name: Digital Integrity Robot robot-cover-url: http://www.digital-integrity.com/robotinfo.html robot-details-url: http://www.digital-integrity.com/robotinfo.html robot-owner-name: Digital Integrity, Inc. robot-owner-url: robot-owner-email: robot@digital-integrity.com robot-status: Production robot-purpose: WWW Indexing robot-type: robot-platform: unix robot-availability: none robot-exclusion: Conforms to robots.txt convention robot-exclusion-useragent: DIIbot robot-noindex: Yes robot-host: digital-integrity.com robot-from: robot-useragent: DIIbot robot-language: Java/C robot-description: robot-history: robot-environment: modified-date: modified-by: robot-name: Direct Hit Grabber robot-cover-url: www.directhit.com robot-details-url: http://www.directhit.com/about/company/spider.html robot-status: active robot-description: Direct Hit Grabber indexes documents and collects Web statistics for the Direct Hit Search Engine (available at www.directhit.com and our partners' sites) robot-purpose: Indexing and statistics robot-type: standalone robot-platform: unix robot-language: C++ robot-owner-name: Direct Hit Technologies, Inc. robot-owner-url: www.directhit.com robot-owner-email: DirectHitGrabber@directhit.com robot-exclusion: yes robot-exclusion-useragent: grabber robot-noindex: yes robot-host: *.directhit.com robot-from: yes robot-useragent: grabber robot-environment: service modified-by: grabber@directhit.com robot-name: DNAbot robot-cover-url: http://xx.dnainc.co.jp/dnabot/ robot-details-url: http://xx.dnainc.co.jp/dnabot/ robot-owner-name: Tom Tanaka robot-owner-url: http://xx.dnainc.co.jp robot-owner-email: tomatell@xx.dnainc.co.jp robot-status: development robot-purpose: indexing robot-type: standalone robot-platform: unix, windows, windows95, windowsNT, mac robot-availability: data robot-exclusion: yes robot-exclusion-useragent: robot-noindex: no robot-host: xx.dnainc.co.jp robot-from: yes robot-useragent: DNAbot/1.0 robot-language: java robot-description: A search robot in 100 java, with its own built-in database engine and web server . Currently in Japanese. robot-history: Developed by DNA, Inc.(Niigata City, Japan) in 1998. robot-environment: commercial modified-date: Mon, 4 Jan 1999 14:30:00 GMT modified-by: Tom Tanaka robot-name: DownLoad Express robot-cover-url: http://www.jacksonville.net/~dlxpress robot-details-url: http://www.jacksonville.net/~dlxpress robot-owner-name: DownLoad Express Inc robot-owner-url: http://www.jacksonville.net/~dlxpress robot-owner-email: dlxpress@mediaone.net robot-status: active robot-purpose: graphic download robot-type: standalone robot-platform: win95/98/NT robot-availability: binary robot-exclusion: yes robot-exclusion-useragent: downloadexpress robot-noindex: no robot-host: * robot-from: no robot-useragent: robot-language: visual basic robot-description: automatically downloads graphics from the web robot-history: robot-environment: commerical modified-date: Wed, 05 May 1998 modified-by: DownLoad Express Inc robot-name: DragonBot robot-cover-url: http://www.paczone.com/ robot-details-url: robot-owner-name: Paul Law robot-owner-url: robot-owner-email: admin@paczone.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: windowsNT robot-availability: none robot-exclusion: yes robot-exclusion-useragent: DragonBot robot-noindex: no robot-host: *.paczone.com robot-from: no robot-useragent: DragonBot/1.0 libwww/5.0 robot-language: C++ robot-description: Collects web pages related to East Asia robot-history: robot-environment: service modified-date: Mon, 11 Aug 1997 00:00:00 GMT modified-by: robot-name: DWCP (Dridus' Web Cataloging Project) robot-cover-url: http://www.dridus.com/~rmm/dwcp.php3 robot-details-url: http://www.dridus.com/~rmm/dwcp.php3 robot-owner-name: Ross Mellgren (Dridus Norwind) robot-owner-url: http://www.dridus.com/~rmm robot-owner-email: rmm@dridus.com robot-status: development robot-purpose: indexing, statistics robot-type: standalone robot-platform: java robot-availability: source, binary, data robot-exclusion: yes robot-exclusion-useragent: dwcp robot-noindex: no robot-host: *.dridus.com robot-from: dridus@dridus.com robot-useragent: DWCP/2.0 robot-language: java robot-description: The DWCP robot is used to gather information for Dridus' Web Cataloging Project, which is intended to catalog domains and urls (no content). robot-history: Developed from scratch by Dridus Norwind. robot-environment: hobby modified-date: Sat, 10 Jul 1999 00:05:40 GMT modified-by: Ross Mellgren robot-name: e-collector robot-cover-url: http://www.thatrobotsite.com/agents/ecollector.htm robot-details-url: http://www.thatrobotsite.com/agents/ecollector.htm robot-owner-name: Dean Smart robot-owner-url: http://www.thatrobotsite.com robot-owner-email: smarty@thatrobotsite.com robot-status: Active robot-purpose: email collector robot-type: Collector of email addresses robot-platform: Windows 9*/NT/2000 robot-availability: Binary robot-exclusion: No robot-exclusion-useragent: ecollector robot-noindex: No robot-host: * robot-from: No robot-useragent: LWP:: robot-language: Perl5 robot-description: e-collector in the simplist terms is a e-mail address collector, thus the name e-collector. So what? Have you ever wanted to have the email addresses of as many companys that sell or supply for example "dried fruit", i personnaly don't but this is just an example. Those of you who may use this type of robot will know exactly what you can do with information, first don't spam with it, for those still not sure what this type of robot will do for you then take this for example: Your a international distributer of "dried fruit" and you boss has told you if you rise sales by 10% then he will bye you a new car (Wish i had a boss like that), well anyway there are thousands of shops distributers ect, that you could be doing business with but you don't know who they are?, because there in other countries or the nearest town but have never heard about them before. Has the penny droped yet, no well now you have the opertunity to find out who they are with an internet address and a person to contact in that company just by downloading and running e-collector. Plus it's free, you don't have to do any leg work just run the program and sit back and watch your potential customers arriving. robot-history: - robot-environment: Service modified-date: Weekly modified-by: Dean Smart robot-name:EbiNess robot-cover-url:http://sourceforge.net/projects/ebiness robot-details-url:http://ebiness.sourceforge.net/ robot-owner-name:Mike Davis robot-owner-url:http://www.carisbrook.co.uk/mike robot-owner-email:mdavis@kieser.net robot-status:Pre-Alpha robot-purpose:statistics robot-type:standalone robot-platform:unix(Linux) robot-availability:Open Source robot-exclusion:yes robot-exclusion-useragent:ebiness robot-noindex:no robot-host: robot-from:no robot-useragent:EbiNess/0.01a robot-language:c++ robot-description:Used to build a url relationship database, to be viewed in 3D robot-history:Dreamed it up over some beers robot-environment:hobby modified-date:Mon, 27 Nov 2000 12:26:00 GMT modified-by:Mike Davis robot-name: EIT Link Verifier Robot robot-cover-url: http://wsk.eit.com/wsk/dist/doc/admin/webtest/verify_links.html robot-details-url: robot-owner-name: Jim McGuire robot-owner-url: http://www.eit.com/people/mcguire.html robot-owner-email: mcguire@eit.COM robot-status: robot-purpose: maintenance robot-type: robot-platform: robot-availability: robot-exclusion: robot-exclusion-useragent: robot-noindex: no robot-host: * robot-from: robot-useragent: EIT-Link-Verifier-Robot/0.2 robot-language: robot-description: Combination of an HTML form and a CGI script that verifies links from a given starting point (with some controls to prevent it going off-site or limitless) robot-history: Announced on 12 July 1994 robot-environment: modified-date: modified-by: robot-name:ELFINBOT robot-cover-url:http://letsfinditnow.com robot-details-url:http://letsfinditnow.com/elfinbot.html robot-owner-name:Lets Find It Now Ltd robot-owner-url:http://letsfinditnow.com robot-owner-email:admin@letsfinditnow.com robot-status:Active robot-purpose:Indexing for the Lets Find It Now search Engine robot-type:Standalone robot-platform:Unix robot-availability:None robot-exclusion: yes robot-exclusion-useragent:elfinbot robot-noindex:yes robot-host:*.letsfinditnow.com robot-from:no robot-useragent:elfinbot robot-language:Perl5 robot-description:ELFIN is used to index and add data to the "Lets Find It Now Search Engine" (http://letsfinditnow.com). The robot runs every 30 days. robot-history: robot-environment: modified-date: modified-by: robot-name: Emacs-w3 Search Engine robot-cover-url: http://www.cs.indiana.edu/elisp/w3/docs.html robot-details-url: robot-owner-name: William M. Perry robot-owner-url: http://www.cs.indiana.edu/hyplan/wmperry.html robot-owner-email: wmperry@spry.com robot-status: retired robot-purpose: indexing robot-type: browser robot-platform: robot-availability: robot-exclusion: no robot-exclusion-useragent: robot-noindex: no robot-host: * robot-from: yes robot-useragent: Emacs-w3/v[0-9\.]+ robot-language: lisp robot-description: Its purpose is to generate a Resource Discovery database This code has not been looked at in a while, but will be spruced up for the Emacs-w3 2.2.0 release sometime this month. It will honor the /robots.txt file at that time. robot-history: robot-environment: modified-date: Fri May 5 16:09:18 1995 modified-by: robot-name: ananzi robot-cover-url: http://www.empirical.com/ robot-details-url: robot-owner-name: Hunter Payne robot-owner-url: http://www.psc.edu/~hpayne/ robot-owner-email: hpayne@u-media.com robot-status: robot-purpose: indexing robot-type: standalone robot-platform: robot-availability: robot-exclusion: yes robot-exclusion-useragent: robot-noindex: robot-host: bilbo.internal.empirical.com robot-from: yes robot-useragent: EMC Spider robot-language: java This spider is still in the development stages but, it will be hitting sites while I finish debugging it. robot-description: robot-history: robot-environment: modified-date: Wed May 29 14:47:01 1996. modified-by: robot-name: Esther robot-details-url: http://search.falconsoft.com/ robot-cover-url: http://search.falconsoft.com/ robot-owner-name: Tim Gustafson robot-owner-url: http://www.falconsoft.com/ robot-owner-email: tim@falconsoft.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix (FreeBSD 2.2.8) robot-availability: data robot-exclusion: yes robot-exclusion-useragent: esther robot-noindex: no robot-host: *.falconsoft.com robot-from: yes robot-useragent: esther robot-language: perl5 robot-description: This crawler is used to build the search database at http://search.falconsoft.com/ robot-history: Developed by FalconSoft. robot-environment: service modified-date: Tue, 22 Dec 1998 00:22:00 PST robot-name: Evliya Celebi robot-cover-url: http://ilker.ulak.net.tr/EvliyaCelebi robot-details-url: http://ilker.ulak.net.tr/EvliyaCelebi robot-owner-name: Ilker TEMIR robot-owner-url: http://ilker.ulak.net.tr robot-owner-email: ilker@ulak.net.tr robot-status: development robot-purpose: indexing turkish content robot-type: standalone robot-platform: unix robot-availability: source robot-exclusion: yes robot-exclusion-useragent: N/A robot-noindex: no robot-nofollow: no robot-host: 193.140.83.* robot-from: ilker@ulak.net.tr robot-useragent: Evliya Celebi v0.151 - http://ilker.ulak.net.tr robot-language: perl5 robot-history: robot-description: crawles pages under ".tr" domain or having turkish character encoding (iso-8859-9 or windows-1254) robot-environment: hobby modified-date: Fri Mar 31 15:03:12 GMT 2000 robot-name: nzexplorer robot-cover-url: http://nzexplorer.co.nz/ robot-details-url: robot-owner-name: Paul Bourke robot-owner-url: http://bourke.gen.nz/paul.html robot-owner-email: paul@bourke.gen.nz robot-status: active robot-purpose: indexing, statistics robot-type: standalone robot-platform: UNIX robot-availability: source (commercial) robot-exclusion: no robot-exclusion-useragent: robot-noindex: no robot-host: bitz.co.nz robot-from: no robot-useragent: explorersearch robot-language: c++ robot-history: Started in 1995 to provide a comprehensive index to WWW pages within New Zealand. Now also used in Malaysia and other countries. robot-environment: service modified-date: Tues, 25 Jun 1996 modified-by: Paul Bourke robot-name: FastCrawler robot-cover-url: http://www.1klik.dk/omos/ robot-details-url: http://www.1klik.dk/omos/ robot-owner-name: 1klik.dk A/S robot-owner-url: http://www.1klik.dk robot-owner-email: crawler@1klik.dk robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: Windows 2000 Adv. Server robot-availability: none robot-exclusion: yes robot-exclusion-useragent: fastcrawler robot-noindex: yes robot-host: 1klik.dk robot-from: yes robot-useragent: FastCrawler 3.0.X (crawler@1klik.dk) - http://www.1klik.dk robot-language: C++ robot-description: FastCrawler is used to build the databases for search engines used by 1klik.dk and it's partners robot-history: Robot started in April 1999 robot-environment: commercial modified-date: 05-08-2001 modified-by: Kim Gam-Jensen robot-name:Fluid Dynamics Search Engine robot robot-cover-url:http://www.xav.com/scripts/search/ robot-details-url:http://www.xav.com/scripts/search/ robot-owner-name:Zoltan Milosevic robot-owner-url:http://www.xav.com/ robot-owner-email:zoltanm@nickname.net robot-status:active robot-purpose:indexing robot-type:standalone robot-platform:unix;windows robot-availability:source;data robot-exclusion:yes robot-exclusion-useragent:FDSE robot-noindex:yes robot-host:yes robot-from:* robot-useragent:Mozilla/4.0 (compatible: FDSE robot) robot-language:perl5 robot-description:Crawls remote sites as part of a shareware search engine program robot-history:Developed in late 1998 over three pots of coffee robot-environment:commercial modified-date:Fri, 21 Jan 2000 10:15:49 GMT modified-by:Zoltan Milosevic robot-name: Felix IDE robot-cover-url: http://www.pentone.com robot-details-url: http://www.pentone.com robot-owner-name: The Pentone Group, Inc. robot-owner-url: http://www.pentone.com robot-owner-email: felix@pentone.com robot-status: active robot-purpose: indexing, statistics robot-type: standalone robot-platform: windows95, windowsNT robot-availability: binary robot-exclusion: yes robot-exclusion-useragent: FELIX IDE robot-noindex: yes robot-host: * robot-from: yes robot-useragent: FelixIDE/1.0 robot-language: visual basic robot-description: Felix IDE is a retail personal search spider sold by The Pentone Group, Inc. It supports the proprietary exclusion "Frequency: ??????????" in the robots.txt file. Question marks represent an integer indicating number of milliseconds to delay between document requests. This is called VDRF(tm) or Variable Document Retrieval Frequency. Note that users can re-define the useragent name. robot-history: This robot began as an in-house tool for the lucrative Felix IDS (Information Discovery Service) and has gone retail. robot-environment: service, commercial, research modified-date: Fri, 11 Apr 1997 19:08:02 GMT modified-by: Kerry B. Rogers robot-name: Wild Ferret Web Hopper #1, #2, #3 robot-cover-url: http://www.greenearth.com/ robot-details-url: robot-owner-name: Greg Boswell robot-owner-url: http://www.greenearth.com/ robot-owner-email: ghbos@postoffice.worldnet.att.net robot-status: robot-purpose: indexing maintenance statistics robot-type: standalone robot-platform: robot-availability: robot-exclusion: no robot-exclusion-useragent: robot-noindex: robot-host: robot-from: yes robot-useragent: Hazel's Ferret Web hopper, robot-language: C++, Visual Basic, Java robot-description: The wild ferret web hopper's are designed as specific agents to retrieve data from all available sources on the internet. They work in an onion format hopping from spot to spot one level at a time over the internet. The information is gathered into different relational databases, known as "Hazel's Horde". The information is publicly available and will be free for the browsing at www.greenearth.com. Effective date of the data posting is to be announced. robot-history: robot-environment: modified-date: Mon Feb 19 00:28:37 1996. modified-by: robot-name: FetchRover robot-cover-url: http://www.engsoftware.com/fetch.htm robot-details-url: http://www.engsoftware.com/spiders/ robot-owner-name: Dr. Kenneth R. Wadland robot-owner-url: http://www.engsoftware.com/ robot-owner-email: ken@engsoftware.com robot-status: active robot-purpose: maintenance, statistics robot-type: standalone robot-platform: Windows/NT, Windows/95, Solaris SPARC robot-availability: binary, source robot-exclusion: yes robot-exclusion-useragent: ESI robot-noindex: N/A robot-host: * robot-from: yes robot-useragent: ESIRover v1.0 robot-language: C++ robot-description: FetchRover fetches Web Pages. It is an automated page-fetching engine. FetchRover can be used stand-alone or as the front-end to a full-featured Spider. Its database can use any ODBC compliant database server, including Microsoft Access, Oracle, Sybase SQL Server, FoxPro, etc. robot-history: Used as the front-end to SmartSpider (another Spider product sold by Engineeering Software, Inc.) robot-environment: commercial, service modified-date: Thu, 03 Apr 1997 21:49:50 EST modified-by: Ken Wadland robot-name: fido robot-cover-url: http://www.planetsearch.com/ robot-details-url: http://www.planetsearch.com/info/fido.html robot-owner-name: Steve DeJarnett robot-owner-url: http://www.planetsearch.com/staff/steved.html robot-owner-email: fido@planetsearch.com robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: Unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: fido robot-noindex: no robot-host: fido.planetsearch.com, *.planetsearch.com, 206.64.113.* robot-from: yes robot-useragent: fido/0.9 Harvest/1.4.pl2 robot-language: c, perl5 robot-description: fido is used to gather documents for the search engine provided in the PlanetSearch service, which is operated by the Philips Multimedia Center. The robots runs on an ongoing basis. robot-history: fido was originally based on the Harvest Gatherer, but has since evolved into a new creature. It still uses some support code from Harvest. robot-environment: service modified-date: Sat, 2 Nov 1996 00:08:18 GMT modified-by: Steve DeJarnett robot-name: Hämähäkki robot-cover-url: http://www.fi/search.html robot-details-url: http://www.fi/www/spider.html robot-owner-name: Timo Metsälä robot-owner-url: http://www.fi/~timo/ robot-owner-email: Timo.Metsala@www.fi robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: UNIX robot-availability: no robot-exclusion: yes robot-exclusion-useragent: Hämähäkki robot-noindex: no robot-host: *.www.fi robot-from: yes robot-useragent: Hämähäkki/0.2 robot-language: C robot-description: Its purpose is to generate a Resource Discovery database from the Finnish (top-level domain .fi) www servers. The resulting database is used by the search engine at http://www.fi/search.html. robot-history: (The name Hämähäkki is just Finnish for spider.) robot-environment: modified-date: 1996-06-25 modified-by: Jaakko.Hyvatti@www.fi robot-name: KIT-Fireball robot-cover-url: http://www.fireball.de robot-details-url: http://www.fireball.de/technik.html (in German) robot-owner-name: Gruner + Jahr Electronic Media Service GmbH robot-owner-url: http://www.ems.guj.de robot-owner-email:info@fireball.de robot-status: active robot-purpose: indexing robot-type: standalone robot-platform: unix robot-availability: none robot-exclusion: yes robot-exclusion-useragent: KIT-Fireball robot-noindex: yes robot-host: *.fireball.de robot-from: yes robot-useragent: KIT-Fireball/2.0 libwww/5.0a robot-language: c robot-description: The Fireball robots gather web documents in German language for the database of the Fireball search service. robot-history: The robot was developed by Benhui Chen in a research project at the Technical University of Berlin in 1996 and was re-implemented by its developer in 1997 for the present owner. robot-environment: service modified-date: Mon Feb 23 11:26:08 1998 modified-by: Detlev Kalb robot-name: Fish search robot-cover-url: http://www.win.tue.nl/bin/fish-search robot-details-url: robot-owner-name: Paul De Bra robot-owner-url: http://www.win.tue.nl/win/cs/is/debra/ robot-owner-email: debra@win.tue.nl robot-status: robot-purpose: indexing robot-type: standalone robot-platform: robot-availability: binary robot-exclusion: no robot-exclusion-useragent: robot-noindex: no robot-host: www.win.tue.nl robot-from: no robot-useragent: Fish-Search-Robot robot-language: c robot-description: Its purpose is to discover resources on the fly a version exists that is integrated into the Tübingen Mosaic 2.4.2 browser (also written in C) robot-history: Originated as an addition to Mosaic for X robot-environment: modified-date: Mon May 8 09:31:19 1995 modified-by:
|
| |||||||||||||||||||||||||||||
SSL
