cobweb 0.0.70

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more perofmant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

1.2.1 - January 09, 2021 (9.28 MB)
1.2.0 - February 25, 2019 (9.28 MB)
1.1.0 - November 06, 2015 (9.28 MB)
1.0.29 - November 05, 2015 (9.28 MB)
1.0.28 - March 20, 2015 (9.28 MB)
0.0.70 - September 16, 2012 (9.27 MB)

Show all versions (94 total)

Runtime Dependencies (10):

addressable >= 0

awesome_print >= 0

haml >= 0

namespaced_redis >= 1.0.2

nokogiri >= 0

redis >= 0

resque >= 0

rspec >= 0

sinatra >= 0

thin >= 0

Show all transitive dependencies

Owners:

Authors:

Stewart McKee

SHA 256 checksum:

f0c79f9e59fa91ecd14bda81043bd77f88af67a2d54badce09e8fe673fc9f1a5

cobweb 0.0.70

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

Runtime Dependencies (10):

Owners:

Authors:

SHA 256 checksum:

Total downloads 323,226

For this version 3,415

Licenses:
N/A

Required Ruby Version: None

Links:

cobweb 0.0.70

Gemfile: = Copy to clipboard Copied!

install: =

Versions:

Runtime Dependencies (10):

Owners:

Authors:

SHA 256 checksum:

Total downloads 323,226

For this version 3,415

Licenses: N/A

Required Ruby Version: None

Links:

Gemfile:
= Copy to clipboard Copied!

install:
=

Licenses:
N/A