Focus on Solutions, Not on Tools

Published in

METRO SYSTEMS Romania

3 min readJun 28, 2019

“There is nothing permanent except change” and this holds true especially for the information technology. Digital transformation is where the focus is for many organizations today. One of the challenges is to innovate and come with a simple solution, while still interconnected with legacy systems.

In this article, I address a practical example of how to produce effective solutions with the technology that you already have at your disposal.

Let’s say you need to know how the load varies each day on the components in a Siebel enterprise for example. How would you solve this?
If you have it available, Oracle Enterprise Manager Cloud Control is one good way to perform Siebel monitoring and multiple other tasks. Else, you may look into ELK stack or Grafana for displaying graphical reports. Or just have a pivot table in an Excel sheet, why not?

My solution approach for automating this task was to use web2py (already deployed on a server for a different project) and a bit of JavaScript.

As mentioned on the project page, web2py is a free and open source full-stack framework for rapid development of secure database-driven web-based applications, written and programmable in Python. It also:

has a small footprint, requires no installation or configuration and exposes most of its functionality (including an IDE with debugger and database admin modules) via a web interface.
provides a plethora of built-in features: database abstraction layer (DAL), CRUD (Create, Retrieve, Update, Delete) functionality, forms system, role-based access control, a scheduler, a built-in wiki, various protocols such as RSS, CSV, JSON, XML and many more.
its structure encourages the developers to separate data representation (the model), data presentation (the view) and the application workflow (the controller) — this is the MVC pattern. Under the hood, an /x/y/z URL maps into a call to application x, controller y.py and function z in that controller.
requires minimal effort to build quite powerful data-driven web applications. Equally important, it greatly reduces the learning curve (originally was developed as a teaching tool).
See here the video course from Prof. Massimo Di Pierro, the creator of web2py.

After building the application scaffolding with the help of web2py, next step was to create the data visualization based on the text source (a CSV report generated by some custom shell scripts). For this I made use of the following JavaScript libraries: papaParse.js, pivot.js and pivotTable.js.

Papa Parse is a fast in-browser JavaScript CSV parser.
Pivot.js is a lightweight open-source JavaScript Pivot Table that knows how to summarize large data sets.
PivotTable.js adds drag and drop support to pivot.js. In addition to the built-in table, heat-map, and table-bar chart renderers, PivotTable.js also allows to create C3 Charts, D3 Visualizations, or Google Charts. The main page of the project shows the basic parameters description and source code demonstration.

Below is the representation for my envMon/webcon/stats/enterprise/date URL (application/controller/function/argument1/argument2):

In web2py/applications/envMon/controllers/webcon.py controller file:

from ssh import *def stats():
    response.subtitle = “Resources Usage for Siebel components”
    fetch_csv(str(request.args[0]), ’webcon’, str(request.args[1]), str(request.args[2]))
    return dict()

In web2py/applications/envMon/modules/ssh.py module file:

# Run the shell script to generate the CSV output def fetch_csv(env,comp,m,d):
    return subprocess.call(['applications/envMon/static/sh/fetch_csv.sh', env, comp, m, d])

In web2py/applications/envMon/views/webcon/stats.html view:

{{extend 'layout.html'}}
<link href="{{=URL('static', 'css/pivot.css')}}" rel="stylesheet" type="text/css">
<link href="{{=URL('static', 'css/c3.css')}}" rel="stylesheet" type="text/css">
<script src="{{=URL('static','js/papaparse.min.js')}}"></script>
<script src="{{=URL('static','js/pivot.js')}}"></script>
<script src="{{=URL('static','js/c3_renders.js')}}"></script>
{{include 'webcon/cpu.html'}}
{{include 'webcon/mem.html'}}
{{include 'webcon/tasks.html'}}

And in web2py/applications/envMon/views/webcon/tasks.html:

<div id="tskTable">tasks</div>
<script type="text/javascript">
    $(function(c) {
     Papa.parse("{{=URL('static/csv'," % s_webcon_ % s % s.csv " %(request.args[0],request.args[1],request.args[2]))}}", {
      download: true,
      skipEmptyLines: true,
      complete: function(parsed) {
       $("#tskTable").pivot(parsed.data, {
        rows: ["Component"],
        cols: ["Time"],
        aggregator: $.pivotUtilities.aggregatorTemplates.sum()(["Tasks"]),
        aggregatorName: "Number of Tasks",
        renderer: $.pivotUtilities.c3_renderers["Stacked Bar Chart"],
        rendererOptions: {
         c3: {axis: {x: {type: 'timeseries',tick: {format: '%d', culling: {max: 11}, rotate: false} },
     y: {label: "count"} },
   grid: { y: {show: true}}, zoom: {enabled: true},
   bindto: "#tskTable",
   size: { height: 400} } } }); } }); });
 </script>

Conclusion

Always strive to improve your ability to see smarter ways to perform your tasks. See how to become better at determining the needs and then proactively deliver new value-rich solutions. Focus on providing solutions, regardless of technologies used.

Focus on Solutions, Not on Tools

Written by Paul Robu