How To Install CSML, an Open-Source Chatbot Engine on Ubuntu 18.04
CSML is a compelling open-source language and conversational engine to create rich chatbots. Did you know that you can also install and host it by yourself for free as a standalone web server on your own server? Let’s find out how in this article!
Installation
On CSML Engine’s github repository, we compile pre-built binaries for MacOS and Linux of the CSML Engine web server, called CSML Server, at every release, under the assets section ( we also maintain docker images if you prefer). As in this example, we are going to install CSML Server on Ubuntu 18.04 (or most recent linux distributions such as Debian, CentOS, Amazon Linux 2…), we will use the linux-based 64-bit binary, called csml-server-linux-amd64
. The latest release to date is v1.4.0: let's copy the address of the corresponding asset.
To download the CSML Server executable on your own server, type the following command in your terminal:
$ wget https://github.com/CSML-by-Clevy/csml-engine/releases/download/v1.4.0/csml-server-linux-amd64
After the file is downloaded, you can simply make it executable, rename it and run it directly:
$ chmod +x csml-server-linux-amd64
$ mv csml-server-linux-amd64 csml-server
$ ./csml-server
> CSML Server listening on port 5000
Congratulations! It runs 🎉. Now go visit your server’s IP on port 5000, which is in my case http://34.241.38.167:5000/ (but it will very likely be a different IP for you).
If you can see this page, CSML is correctly installed and runs properly on your machine. But we are not there yet! We do have a bit of configuration to do. You can for now simply hit ctrl+C
to kill the process.
Configuration
CSML Server can easily be configured with environment variables. In fact, the environment variables are documented on this very landing page, so it should be easy to do!
The CSML Engine requires a database, and works either with MongoDB or Amazon DynamoDB. There are advantages and drawbacks to each of these offerings; in this example, we are going to use MongoDB, which is a popular choice.
Installing MongoDB is not covered in this article, but you can for example find instructions on how to setup MongoDB on Ubuntu on MongoDB’s documentation, or you can use one of the many SaaS offerings by some hosting providers that are ready to use. MongoDB themselves have Atlas, a managed, cloud-based, ready-to-use MongoDB service.
There are many ways to load environment variables in your shell. Let’s simply create a simple .env
file next to the server executable and copy-paste the default environment variables in this file for your setup:
ENGINE_DB_TYPE=mongodb
MONGODB_HOST=localhost
MONGODB_PORT=27017
MONGODB_DATABASE=csml
MONGODB_USERNAME=
MONGODB_PASSWORD=
ENGINE_SERVER_PORT=5000
ENCRYPTION_SECRET=some-secret-string
DISABLE_SSL_VERIFY=false
DEBUG=true
Of course, you are free to change these variables to accommodate your own setup. Among other things, you should also obviously pick your own ENCRYPTION_SECRET
and not leave the default value!
Then, you can run this command to export the variables from this file into your shell:
$ export $(cat .env | xargs)
To check that the environment is correctly set, you can use the env
command and verify that everything looks right.
Then, re-run the ./csml-server
command. The result will be the same, but now your server should be correctly connected to your database and ready to use!
Using your Self-Hosted CSML Server
To test if your CSML Server installation works properly, you can run a simple query, for example by saying “Hi!” to a sample bot:
curl -X "POST" "http://YOUR_SERVER_IP:ENGINE_PORT/run" \
-H 'content-type: application/json' \
-d $'{
"bot": {
"default_flow": "Default",
"id": "mybot",
"name": "MySuperBot"
"flows": [
{
"id": "e0a13373-2037-4590-8018-ab14e74b27a1",
"content": "start:\\n\\tsay \\"Hello from CSML Server!\\"\\ngoto end",
"commands": [
"/default"
],
"name": "Default",
"description": "Default custom flow"
}
]
},
"event": {
"metadata": {
"some": "info",
"about": "the current user"
},
"payload": {
"content": {
"text": "Hi!"
},
"content_type": "text"
},
"request_id": "67d283bd-35d5-4744-be61-2f063573022f",
"client": {
"user_id": "myuser",
"channel_id": "mychan",
"bot_id": "mybot"
}
}
}'
The server should respond with something like:
{
"messages": [
{
"payload": {
"content_type": "text",
"content": {
"text": "Hello from CSML Server!"
}
},
"interaction_order": 0,
"conversation_id": "555416aa-c033-4676-ad64-eaf5c6783617",
"direction": "SEND"
}
],
"conversation_end": true,
"request_id": "67d283bd-35d5-4744-be61-2f063573022f",
"received_at": "2021-01-22T20:58:40.154Z",
"interaction_id": "591c2ad1-b540-47b0-bd5c-9a5a3907cd0f",
"client": {
"bot_id": "mybot",
"user_id": "myuser",
"channel_id": "mychan"
}
}
You can find the full REST API documentation in OpenAPI v3 format on our Github repository.
Next Steps: Prepare For Production Use!
This article gives you an easy starting point to install the CSML Engine on your own server. The instructions are similar if you plan to use your own development machine as well. (By the way, we also provide a MacOS (intel) binary.)
However, if you need to run your own CSML Server for a production payload, there are probably a few extra steps that you should take (or just use CSML Studio, the online CSML development environment that does not need any installation). You will probably need to:
- add a reverse-proxy such as nginx and enable HTTPS for secure exchanges
- add some redundancy to make sure that it is always available
- make sure that your service is resilient and handles the loss of one or more processes gracefully
- add some auto-scalability into the mix, to make sure that you are ready to serve the right number of simultaneous conversations for your use case
Depending on how you plan to distribute your bot, there are many good ways to do it. If you are looking for architecture advice, please reach out on Slack! We are running millions of conversations securely on CSML Studio and can probably point you in the right direction.
Originally published at https://blog.csml.dev on January 23, 2021.