Web Crawler Admin Platform Crawlab v0.4.8 Release

Introduction

This release include following aspects:

1. Support installing or pre-installing multiple languages other than Python.
2. Better installation interface, can view multi-node installation status.
3. Optimized RPC, a Redis-based technique using latent queue.
4. More functionality configurations

Change Log

Features / Enhancement

• Support Installations of More Programming Languages. Now users can install or pre-install more programming languages including Java, .Net Core and PHP.
• Installation UI Optimization. Users can better view and manage installations on Node List page.
• More Git Support. Allow users to view Git Commits record, and allow checkout to corresponding commit.
• Support Hostname Node Registration Type. Users can set hostname as the node key as the unique identifier.
• RPC Support. Added RPC support to better manage node communication.
• Run On Master Switch. Users can determine whether to run tasks on master. If not, all tasks will be run only on worker nodes.
• Disabled Tutorial by Default.
• Added Related Documentation Sidebar.
• Loading Page Optimization.

Bug Fixes

• Duplicated Nodes. #391
• Duplicated Spider Upload. #603
• Failure in dependencies installation results in unusable dependency installation functionalities.. #609
• Create Tasks for Offline Nodes. #622

Reference

• Github: https://github.com/crawlab-team/crawlab
• Demo: https://crawlab.cn/demo