RubyGems Navigation menu

cobweb 1.2.1

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Gemfile:
=

instalar:
=

Versiones:

  1. 1.2.1 January 09, 2021 (9,28 MB)
  2. 1.2.0 February 25, 2019 (9,28 MB)
  3. 1.1.0 November 06, 2015 (9,28 MB)
  4. 1.0.29 November 05, 2015 (9,28 MB)
  5. 1.0.28 March 20, 2015 (9,28 MB)
Mostrar todas las versiones (94 total)

dependencias de Runtime (9):

addressable >= 2.3.8
haml >= 4.0.7
json >= 1.8.3
nokogiri >= 1.6.0
rake >= 0
redis >= 3.2.1
redis-namespace >= 1.6.0
sinatra >= 1.4.6
slop >= 4.2.0

dependencias de Development (7):

bundle-audit >= 0
coveralls >= 0
mock_redis >= 0
rspec >= 0
rspec-core >= 0
sidekiq >= 0
thin >= 0
Mostrar toda la cadena de dependencias

Propietarios:

Subida por:

Autores:

  • Stewart McKee

SHA 256 checksum:

=
← Versión anterior

Total de descargas 333.449

Para esta versión 2.909

Versión publicada:

Licencia:

MIT

Versión de Ruby requerida: >= 0

Enlace:

Página Documentación Descarga Revisar cambios Badge Suscribirse RSS Reportar abusos Dependencias inversas