cobweb 0.0.72

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more perofmant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

1.2.1 - January 09, 2021 (9.28 MB)
1.2.0 - February 25, 2019 (9.28 MB)
1.1.0 - November 06, 2015 (9.28 MB)
1.0.29 - November 05, 2015 (9.28 MB)
1.0.28 - March 20, 2015 (9.28 MB)
0.0.72 - September 20, 2012 (9.27 MB)

Show all versions (94 total)

Runtime Dependencies (11):

addressable >= 0

awesome_print >= 0

haml >= 0

json >= 0

namespaced_redis >= 1.0.2

nokogiri >= 0

redis >= 0

resque >= 0

rspec >= 0

sinatra >= 0

thin >= 0

Show all transitive dependencies

Owners:

Authors:

Stewart McKee

SHA 256 checksum:

eb13c0c6f8ac011f523416c29473f0b211752b19edcd3595c9154b48bd819712

cobweb 0.0.72

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

Runtime Dependencies (11):

Owners:

Authors:

SHA 256 checksum:

Total downloads 323,512

For this version 3,442

Licenses:
N/A

Required Ruby Version: None

Links:

cobweb 0.0.72

Gemfile: = Copy to clipboard Copied!

install: =

Versions:

Runtime Dependencies (11):

Owners:

Authors:

SHA 256 checksum:

Total downloads 323,512

For this version 3,442

Licenses: N/A

Required Ruby Version: None

Links:

Gemfile:
= Copy to clipboard Copied!

install:
=

Licenses:
N/A