rdaneel 0.2.2
Add robots.txt support on top of em-http-request
Gemfile:
=
复制到剪贴板
已复制!
安装:
=
Runtime 依赖 (2):
em-http-request
>= 0.2.11
robot_rules
>= 0.9.3