Wisdom & OrientDB

in a “Dockershell”

Deploy your flexible and scalable platform in a minute


In this new story, I want to show you how much it is simple today to build and deploy your own modern and scalable platform. In the following lines I will present an industrial-strength and future-proof technology stack that is based on both Wisdom Framework, OrientDB and Docker technologies.

This presentation will take the form of a pretty complete tutorial in order to install, build, develop and run an application on top of this flexible and scalable platform, on your own!

Context

Loca is an incoming SaaS product dedicated to connect people that are involved in specific international business fields. So this next-gen product will offer services across several countries. Loca’s core services will be available in all countries, whilst some specific services will be deployed in each countries.

In term of distributed architecture, a central intelligence will be located in United States serving super-services (management, authentication, data mining etc.), while dedicated intelligences located across the world (in particular in Asia, Africa and South America) will depend on the central one but also solve local concerns. These local concerns can be specific software configurations, specific legal requirements to secure/serve data etc.

Loca requires to be flexible and scalable in order to deploy both its global & local services on different sub-servers installed all around the world.

Thus, Loca will deploy in each country at least one server-side logic and one database instances both connected to the central intelligence.

Defining a flexible & scalable stack

In order to solve the requirements of flexibility and scalability, each Loca local instance will be composed by :

  • a web server running a server-side logic, exposing webservices as a RESTful API and serving a web application that the user can run on their browser. This web server will be bridged to the central intelligence one’s.
  • a database running in distributed mode, bridged to the central intelligence one’s.
Wisdom Framework: a modular and dynamic Java web services framework

First, I chose to use Wisdom Framework to run Loca’s server-side logic, as this Java framework ensures the flexibility and scalability required by Loca.

In fact, Wisdom brings modularity and dynamism to the application platform thanks to the implementation of OSGi specifications and Apache Felix iPOJO: no more framework reboot needed, all new services or service upgrades are resolved, linked and instanciated in real time.

Last but not least, Wisdom relies on Vert.x application platform that enables to bridge in simple way different server nodes, thereby fulfilling the requirements of polymorphic server logics distribution in Loca.

OrientDB: a multi-modal graph & document database

For data management, I decided to use OrientDB as Loca both deploys a social network and serves heterogeneous data about the users across the foreign countries. OrientDB offers respectively a Graph model and a Document model to handle that problematics.

The famous Docker container technology

Finally I will use the famous Docker open platform for distributed application in to simplify in a reliable manner the configuration and execution of Loca all across these different servers (and cloud providers).

Furthermore, it brings a second layer of distribution and composition in Loca architecture, in particular by enabling to package composite Wisdom & OrientDB instances.

Ready to rummmble ? Let’s start this tutorial by setting up our development environment!


Environment setup

Setup Docker on your OSX (or Windows)

As I am an OSX user, I describe in the following lines how to setup Docker on your Mac. Just jump to the next section if you are a Linux user.

Firstly, download boot2docker and install it.

Then enter the following commands:

boot2docker up
// will create a light-weight Linux image
boot2docker init
// set the permissions to shell in order to compute Docker commands
$(boot2docker shellinit)
// N.B: run previous command anytime you have that kind of error in your shell after entering a Docker command:
Post http:///var/run/docker.sock/v1.17/build?dockerfile=Dockerfile&rm=1&t=rpellerin%2Fwisdom-orientdb-sample: dial unix /var/run/docker.sock: no such file or directory

Setup Docker on your Linux distribution

The really well documented Docker’s website describes a way to setup Docker on various Linux distribution:

  • for Ubuntu it’s here
  • for Fedora here
  • for openSUSE here
  • etc :)

Bringing Wisdom Framework & OrientDB together on top of Docker

So my first idea for Loca was to setup 2 different Docker containers:

  • one for Loca webservices running on top of Wisdom
  • one for Loca data maintained in OrientDB.

This ensures to be able to deploy these containers across heterogeneous server setups and cloud providers. It will give a great flexibility to Loca infrastructure.

Let’s build and run these 2 dockers containers together!

Build Wisdom & Orientdb Docker containers

It’s time to run a Wisdom docker image and an OrientDB docker image.

In order to do this, we will use:

Later, if you want to build your own OrientDB container, just follow these steps:

  • register to DockerHub
  • create a repository “orientdb”
  • open a shell and chain the following commands:
// build your own OrientDB docker image
git clone https://github.com/orientechnologies/orientdb-docker
cd orientdb-docker
docker build -t <YOUR_DOCKER_HUB_USER>/orientdb .
docker push <YOUR_DOCKER_HUB_USER>/orientdb

Run and link Wisdom & OrientDB containers

At this point, we are going to run our Wisdom and OrientDB containers by opening the required ports and by linking the 2 containers, as Wisdom will consume OrientDB data:

// run OrientDB container
docker run --name orientdb -d -p 2480:2480 -p 2424:2424 rpellerin/orientdb2:2.0.6
// run Wisdom and link its container
docker run -d -P --name wisdom --link orientdb:db -p 9000:9000 cescoffier/wisdom-docker:0.8.0
// test the link, should return > [/orientdb:/wisdom/db]
docker inspect -f "{{ .HostConfig.Links }}" wisdom
// check if your containers are online
docker ps
Wisdom & OrientDB running into 2 distincts Docker containers

Populate your OrientDB container

As an interlude, we will populate OrientDB with some data.

First, we need to retrieve the IP of our OrientDB container in order to access the studio interface:

// retrieve your OrientDB container IP
docker inspect --format '{{ .NetworkSettings.IPAddress }}' orientdb
> YOUR_ORIENT_DB_IP

Then go with your favorite browser on this URL:

http://<YOUR_ORIENT_DB_IP>:2480/

To retrieve your OrientDB root user password do:

docker exec orientdb cat /opt/orientdb/config/orientdb-server-config.xml | grep “password=”

You can set up a new password by modifying orientdb-server-config.xml file in your orientdb-docker folder (but you will need to rebuild & rerun your container) or by doing this command manually:

docker exec orientdb sed 's/<user resources="\*".*/<user resources="\*" password="YOUR_NEW_PASSWORD" name="root"\/>/' /opt/orientdb/config/orientdb-server-config.xml

Using your browser, you are able to create a new database with your own login data:

You can now populate your database with some data via OrientDB studio in the Browse section or directly via its console like this:

docker exec orientdb ./opt/orientdb/bin/console.sh "connect remote:localhost/test root <YOUR_PASSWORD>; begin; create class TestClass; commit; return;"

Set a distributed OrientDB : AWS & local Docker nodes

As my Loca architecture will be distributed, I need now to create 2 different instances:

  • one master OrientDB running on AWS : in order to simplify this tutorial, I will use the OrientDB community preconfigured AMI image, but obviously you can run it in a Docker container too!
  • one slave OrientDB running in a local Docker container

Both of these database will replicate their own state into the other asynchronously. For Loca this enables to bring data close to the users by running on a local hosting provider.

Let’s start to setup our AWS OrientDB instance in distributed mode:

// get and run a ready-to-use OrientDB AMI
// then open the port 5701 on your server (for AWS, AWS console > security groups)
// ssh / sftp your AWS instance in order to ...
// copy hazelcast.xml in /opt/orientdb/config
// copy updateip-aws.sh in /opt/orientdb/
// copy default-distributed-db-config.json in /opt/orientdb/config and set chmod 0644
// configure Hazelcast by setting your public & private IPs
// you should have to run this script twice as ifconfig.me is slow
sh /opt/orientdb/updateip-aws.sh
// finally launch (in sudo) OrientDB in distributed mode
sudo sh /opt/orientdb/bin/server.sh -Ddistributed=true -Xmx1024m

Then, let’s launch our local OrientDB in distributed mode too:

// run OrientDB Docker image while opening port 5701
docker run --name orientdb -d -p 2480:2480 -p 2424:2424 -p 5701:5701 rpellerin/orientdb:2.0.6
// configure Hazelcast with container's public & private IP
docker exec -i -t orientdb sh /opt/orientdb/updateip.sh `docker inspect --format '{{ .NetworkSettings.IPAddress }}' orientdb` `curl ifconfig.me/ip`
// run OrientDB in distributed mode
docker exec -i -t orientdb ./opt/orientdb/bin/shutdown.sh
docker exec -i -t orientdb  ./opt/orientdb/bin/server.sh -Ddistributed=true -Xmx1024m

Hence, these two OrientDB instances will be automatically linked thanks to hazelcast and will replicate asynchronously each data modification:

OrientDB in distributed mode with 2 different members (nodes)

Run your first app on “Dockerized “ Wisdom Framework & OrientDB

It’s time now, to run our first app in Wisdom using OrientDB to store its data!

I will use here a fork of Jonathan Bardin [@barjo], serial contributor to Wisdom project codebase, wisdom-orientdb/sample that I have refactored in order to make it runnable in a standalone manner.

This application provides a simple Todo management webapp running on top of Wisdom that stores its data in an OrientDB database named todolist.

Build & run Wisdom OrientDB sample app

// build an OrientDB Wisdom sample app docker image
git clone https://github.com/romainPellerin/wisdom-orientdb-sample
cd wisdom-orientdb-sample
docker build -t rpellerin/wisdom-orientdb-sample .
// kill your previous Wisdom if it is still running
docker stop wisdom
docker rm wisdom
// run the sample on top of Wisdom docker container
docker run -d -P --name wisdom-sample --link orientdb:db -p 9000:9000 rpellerin/wisdom-orientdb-sample
// create a todolist database
docker exec orientdb ./opt/orientdb/bin/console.sh "create database remote:localhost/todolist root <YOUR_PASSWORD> plocal"

Your application is up, you can now browse it:

// retrieve your OrientDB container IP
docker inspect --format '{{ .NetworkSettings.IPAddress }}' wisdom
> YOUR_WISDOM_IP
// and browse
http://<YOUR_WISDOM_IP>:9000/
Sample app running on top of “Dockerized” Wisdom & OrientDB

In addition you can monitor your Wisdom Framework status at this url:

// log in with "admin" as login & pwd (by default)
http://<YOUR_WISDOM_IP>:9000/monitor/dashboard
Wisdom Framework Dashboard: monitor performance, manage your bundles, shell & cie

This monitor offers you plenty of functionalities in order to track your application performance, to check what REST services (routes) are currently running (exposed), to install/update/remove bundles (features), to access a web shell implemented on top of Shelbie etc.

Wisdom Framework REST webservices (active routes) monitoring

Pretty cool!

Run your Dockerized Wisdom app in watch-mode!

This is a must-have for each developer who doesn’t want to repeat bothering steps each time he needs to test his new source code version: Wisdom Framework’s watch-mode!

In this mode, Wisdom is continuously watching your sources & resources, and builds a new version of your project each time he detects a modification. No more manual and repetitive make or mvn clean install commands.

@cescoffier explains on wisdom-docker repository how to launch Wisdom in watch-mode while deploying a new version of your app on your Wisdom Docker container.

In this tutorial context, we will run this watch-mode on our wisdom-orientdb-sample Docker image, just by entering these 4 commands:

cd wisdom-orientdb-sample/
// install Maven then do
mvn clean install
// run Wisdom on sample app target folders
docker run -d -P --name wisdom --link orientdb:db -p 9000:9000 -v `pwd`/target/wisdom/logs -v `pwd`/target/wisdom/conf:/wisdom/conf -v `pwd`/target/wisdom/application:/wisdom/application rpellerin/wisdom-orientdb-sample:0.8.0
// launch a local Wisdom instance in watch-mode that will update in real time your target folders
mvn wisdom:run -DwisdomDirectory=target/wisdom

Now each modification of wisdom-orientdb-sample will be taken in account while updating your Docker application container with the last source code. Furthermore, the watch-mode exposes your container’s logs output into the shell!

Wisdom Framework watch-mode: real-time detection of resources modifications and Docker image updating

Really useful, isn’t it? ☺

Let’s modify your first Dockerized Wisdom app!

We will now learn how to develop a web service in Wisdom by updating the wisdom-orientdb-sample application code.

Wisdom project structure

I don’t want to duplicate here the great Wisdom Framework documentation, but here is the structure of a Wisdom application project:

  • src/main/java contains the Java sources of your application
  • src/main/resources contains the assets (javascript libs, css files, images etc.) and templates included in your application package
  • src/main/configuration contains the configuration file (application, runtime, logger…​)
  • src/main/assets contains assets that are copied to Wisdom but not included in application package
  • src/main/templates contains assets that are copied to Wisdom but not included in application package

Warmup your environment

In order to clean your dev environment, you should now:

  • clean your previous todolist table in OrientDB by dropping all tables:
docker exec orientdb ./opt/orientdb/bin/console.sh "connect remote:localhost/todolist root root; begin; drop class TodoList; drop class Todo; commit; return;"
  • reinit your Wisdom docker container
docker stop wisdom
docker rm wisdom
// then rerun your instance in watch-mode as described in the previous section

OrientDB data model modification

Into this Todolist app, we will add a User model to existing TodoList and Todo models in src/main/java/todolist/model folder, in order to bind the Todo with the User who created it (if he is registered in the database).

Here is the new User.java model source code:

package todolist.model;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
@JsonIgnoreProperties("handler")
public class User {
@Id
private String id; // unique identifier from java persistence
private String name;
    public void setName(String name) {
this.name = name;
}
    public String getName() {
return name;
}
}

We will now update the Todo.java model to add a reference to the user who owns it:

...
public class Todo {
@Id
private String id;

private User owner; // add a reference to the owner

private Date date;
    ...
    // add Owner getter & setter
    public User getOwner() {
return owner;
}

public void setOwner(User owner) {
this.owner = owner;
}

}

Modify the TodoList controller

As this sample app populates the database with a first todo, we have to update the todo with an owner in TodoListController.java class located in src/main/java/todolist/controller:

@Validate
private void start(){
//Populate the db with some default value
if(!listCrud.findAll().iterator().hasNext()){

User user = new User();
user.setName("@rom1_ubidreams");
// my twitter ID ;)

Todo todo = new Todo();
todo.setOwner(user);
todo.setContent("Check out this awesome todo demo!");
todo.setDone(true);
}
}

Now it’s time to add a UserController.java that will expose a service on /name route in order to verify that a specific User exists is registered in the OrientDB database. We will implement this with a search by name.

// add headers on your own, it's way more readable without ;)
...
@Controller
@Path
("/user")
public class UserController extends DefaultController{
static {Class workaround = Proxy.class;}

@Model(value = User.class)
private OrientDbCrud<User,String> userCrud;

/**
* Retrieve a user by name.
*
*
@response.mime text/json
*
@return user if exists, else 404.
*/
@Route(method = GET,uri = "/name/{name}")
public Result getUserByName(final @Parameter("name")
String name){
Iterator<User> iter = userCrud.findAll().iterator();

// search for a user with this name
User user = null;
while (iter.hasNext()){
user = iter.next();
if (user.getName().equals(name)) break;
user = null;
}

return (user == null ? notFound().render("unknown") :
ok(user).json());
}

}

As an aside you, can see in the previous code how much it is simple in Wisdom to declare a GET route to a webservice on a specific URI, here /user/name/{name}.

If you have followed correctly all this tutorial steps ☺, you should have these data retrieved from OrientDB by the User controller:

http://{YOUR_WISDOM_IP}:9000/user/name/@rom1_ubidreams
// displays user info
{"id":"#14:0","name":"@rom1_ubidreams"}

User Interface template modification

We are going to do a really simple UI modification by adding a new text area before the todo one’s.

This sample UI is generated with Ractive.js a useful and powerful JavaScript template engine. So we will add this text area in list.ract located in src/main/resources/assets/app/views folder:

<div class="col-md-4 col-md-push-4" id="newTodo">
<textarea id="owner" placeholder="Enter your owner id"/>
<textarea on-enter="newTodo" placeholder="Press Enter to add a
TODO" autofocus/>
</div>

If you want to use Ractive.js in your own Wisdom project, you just have to add the wisdom-ractivejs plugin in your pom.xml file like this:

<plugin>
<groupId>org.wisdom-framework</groupId>
<artifactId>wisdom-ractivejs-maven-plugin</artifactId>
<version>0.1.4</version>
<executions>
<execution>
<id>compile-ractivejs-files</id>
<phase>compile</phase>
<goals>
<goal>compile-ractivejs</goal>
</goals>
</execution>
</executions>
</plugin>

User experience upgrading

We will finish now by modifying the client-side logic in order to validate the user name before pushing the todo to the server. We will modify the existing newTodo method of TodoListController.js JavaScript file located in src/main/resources/assets/app/ folder:

function newTodo(event) {
    // highlight in red the owner text area if it is empty
var ownerId = $("#owner");
if (ownerId.val() == "") {
ownerId.addClass("error");
return;
}
else ownerId.removeClass("error");
    // create a user
var user = {
name: ownerId.val()
}
    // add the user reference to the previous todo
var newtodo = {
owner: user,
content: event.node.value,
done: "false"
};
    // if our user controller returns valid User data push the todo
// and highlight in green the owner field else in red
$.ajax({
type: "GET",
contentType: "application/json; charset=UTF-8",
url: "/user/name/"+user.name
}).done(function(data) {
ownerId.addClass("valid");
newTodoValidUser(newtodo);
}).fail(function(){
ownerId.addClass("error");
});
}

// push the todo to the server
function newTodoValidUser(todo){
$.ajax({
type: "PUT",
contentType: "application/json; charset=UTF-8",
url: _model.url,
data: JSON.stringify(todo)
}).done(function(data) {
_model.todos.push(data);
});
}

We will finish this tutorial by running our app and entering a first todo with a valid user name:

// run your app by browsing
http://{YOUR_WISDOM_IP}:9000
// enter rom1_ubidreams as owner
// enter "finish my Medium story" as a todo
// press [ enter ]

You should get this result:

As you can see there is only one difference with your result and mine. My todolist indicates that I finished this Medium story! ☺

Conclusion

In this story, I presented a set of technologies that we used in order to build a scalable and flexible SaaS platform: Wisdom as a modular & dynamic webserver, OrientDB as a distributed Graph/Document database and Docker as a container for all these instances.

As an aside, I strongly recommand you to adopt Wisdom as a core technologies for your next web services based applications.

Have a look to all existing cool features of Wisdom Framework on Github and feel free to join its community!

In fact, despite the fact @cescoffier & @barjo are doing an amazing job on Wisdom (800k lines of code produced in one year and a half o_O), they are seeking for talented open source contributors!

Wisdom Framework project recruits talented contributors.

You are in ? Cool! Just pick up a task here!

Finally, I sincerely hope this article was interesting and useful for you.

If yes, share it and follow me on Twitter, I’ll keep you informed about my next shot ;)

Otherwise, you can recommend this article on Medium below (yesss … it’s that little heart icon at the bottom of this screen ;] ).

If not, feel free to skip it! :p

That’s all folks!