How to Build a Trading Bot: Step by Step
Day 3: Trading System Lambda Architecture
Today, we are going to move on to our core step: implementing a Lambda Architecture to the trading system.
I will use Kafka as the data streaming layer, probably use Mongodb or Cassandra as the batch layer, some in memory databases or spark as the speed layer. I will ignore the serving layer for now. So let’s begin today’s journey from Kafka.
- Terraform+Ansible to manage Kafka + zookeeper dockers.
First of all, you have to install the terrafor and ansible. After the installation, fill out the information in terraform.tfvars.example, and change the name to terraform.tfvars. Then:
After this, copy your kafka ips to host file under the playbooks folder, then run to get kafka node started:
ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook -i hosts start-network.yml