CortexDB: From context management to an application

The agile development of applications requires an also agile architecture. The CortexDB and its tools provide developers with the flexibility needed for agile implementation. This partly requires a rethinking in development.

Cortex AG
Cortex AG
Feb 4, 2020 · 11 min read

When using classic project methods, the contents are already identifiable in advance, the costs can be calculated and risks can be identified. At the beginning of the project, a definable final state must be assumed, which can be described and planned in detail. These methods are therefore considered a monolithic approach.

A time buffer is added for unexpected events or new findings during the project phase. Thus, there is a fuzziness in the planning, which increases during the ongoing project and has to be compensated later by an increased use of resources or capital. This often has negative effects on costs, time requirements and the quality of the result.

In contrast, agile development follows a product vision, which is maintained in a collection of all previously known components (so-called backlog). This is subject to constant adaptation and is based on the expectation that after each development cycle a potentially usable result is available. The success is thus based on close teamwork, fast feedback, high reactivity and continuous process improvement. The members of the team therefore carry a high level of responsibility for the success of the project.

Database driven applications in a dynamic environment

Within companies, database specialists and administrators are often assigned to operations and not to development. In addition, a distinction is also often made between database administrators and developers. Administrators perform general administrative tasks and have little knowledge of the applications; database developers work closely with application developers and have a deeper knowledge of the applications. Users, developers, database developers, and database administrators also have different views of the development and deployment process. In addition, databases are considered a sensitive component of an application. Especially in relational databases with a grown structure and a large number of dependencies, a deep understanding is necessary in order to be able to implement upcoming changes as risk-free as possible. This circumstance has led to the fact that in many companies databases have now become a bottleneck in the otherwise agile development process.

Figure 1: Silo systems for different requirements (S.A.R.L. Martin, 2014)

As a solution for this circumstance, there is no need to extend the databases from a certain complexity or for further requirements. From this point on new databases are created (a new data silo). This creates a silo landscape that distinguishes between operational (application) databases and analytical databases. Interfaces are then to be integrated between the individual silos in order to transfer, synchronize, aggregate or archive data. A zoo of different, self-contained systems is created.

The agile development process for database requirements

In a relational database model, the objects of the department are divided into different tables. Simple changes from the perspective of the user department can therefore result in complicated processes from the perspective of the database developers.

On the one hand, object-oriented mapping is mandatory for simplification (object-relational mapping), on the other hand, there is no generally accepted model for object-oriented data modeling within relational databases. Due to this complexity, the usage of agile project methods for (relational) database changes is limited. It requires the assignment of database administrators and developers to the software development and furthermore longer processing cycles (“sprints”) for the changes. This circumstance can be countered with a schematic database.

The CortexPlatform as a solution strategy

Figure 2: Elements of the CortexPlatform

The usage as a schema-less database with the integrated functions of the CortexDB is possible via the corresponding API just like with other products. Here the content of the database is accessed directly to create, read, update or delete objects. Every change to an individual object is transaction-safe. Figure 2 shows in the middle part the database level with the API for the CortexDB. This enables direct communication with the database without the need for additional tools; only the basic functions of the CortexDB are available. This includes, for example, a complete index of all contents, which can be used universally without administration. This simplification for developers is supplemented by additional functions when accessed via the API functions of the CortexUniplex.

The CortexPlatform in the database environment

Searches within the indexed fields can therefore only be performed in the combination as represented by the index. Searching using other fields and their combinations results in additional indexes. As a consequence, each additional index requires more memory and has to be adjusted.

Schemaless databases are generally assigned to the NoSQL area. Records are not subject to a superordinate schema and are considered as so-called “containers”. This allows the necessary flexibility when storing objects. However, as with SQL databases, an index is required for selection. This index must be configured by administrators and developers and demands the same consequences as in a relational environment.

The Concept of the CortexPlatform

The CortexDB is to be assigned to the schema-less databases. Records correspond to containers without a predefined structure. This also applies to individual field contents whose type is not required for the CortexDB. Only when using an application it is necessary to interpret and display the contents accordingly.

In addition, the CortexDB indexes all contents of all fields of all containers in an universal index. Manual adjustments are no longer necessary. It is irrelevant which field type actually exists. Texts, dates and numerical values, as well as graphs (links) to other objects are subject to automatic indexing and can therefore be selected in any combination. The CortexUniplex application offers functions based on this and its own API.

The CortexUniplex as an entry into development based on the CortexDB

The configuration of the CortexUniplex means that the UniplexAPI can access and use the configuration. Preconfigured selections and output lists (also pivot and formatting of the contents), rights and roles and other settings are available to a developer. Correspondingly, the configuration of automatisms and their execution takes place within the CortexDB with the CortexUniplex. The results are also available via the API; the development effort is reduced to the interactive application and the definition of processes in advance.

The UniplexAPI is accessed, which in turn uses the configurations of the CortexUniplex to access the data via the API of the CortexDB. The functions of the database are thus easily available.

The internal structure of the CortexDB

The separate encapsulation and integration of the JavaScript implementation “V8” (provided by Google under BSD license) as an additional server function allows the execution of JavaScript code stored in the database. The execution of this code and the return of results is done on the server side. The use is thus possible for three separate automatisms:

  • for transaction-dependent automatisms at the time a container is changed,
  • for subsequent automatisms after changing containers (queue),
  • for the time-controlled execution (to the minute) of code by the database server.

The combination of these three automatisms is possible and allows the mutual influence and usage of the generated results. The use of the second automatism (downstream in a queue) also allows the use of system resources outside the database server. Furthermore, the usage of JavaScript code is possible when using CortexUniplex lists. In this case, the contents of a list are accessed, which are processed on the server side and the result is returned in a field.

Using stored code

The subsequent automatism (so-called “agent”) is used to process changes across data records and can also access other resources of the operating system (for example, to write a file, connect to another database, and so on). Here, the initiator of the execution is again a transaction.

Figure 5: View of a record with code and variables

The time-controlled execution of code (the so-called “scheduler” and most interesting of the three) can be used free of transactions and runs exclusively within the database. Any content (container, values or code) is accessible. Variables can be stored in code, in fields of the code container or in other containers. The configuration and modification of the stored code is done within the containers. These can be edited - in addition to API usage - with the help of CortexUniplex.

Usage of the CortexUniplex

Figure 6: Record view of a Containers in the CortexUniplex

For the representation and access, corresponding rights and roles can be defined. A user therefore only gets access to the information for which access rights exist; this UniplexAPI also observes these access rights.

In addition to this, the configuration of selection options (so-called portals) can be carried out. This enables the repeated use of selections for different use cases. For example, the selection of data records with a certain status, certain contents in different fields or also according to processing statuses. The mapping of work processes is thus possible and the configurations of these selections are also available via the UniplexAPI.

List representations are mandatory for output within CortexUniplex. These allow the presentation of single or many containers with selected contents. Please note that the line display of a container can contain a large number of sub-lines (so-called sub-lists), for example, to display and use information from other data sets via graphs (links). This makes it possible, for example, to display a hierarchical list of parts of a master data set or a networked structure. As with the functions mentioned above, these lists are also available via the UniplexAPI.

Figure 7: Simplified JSON of a Uniplex list with one record

Each output via the UniplexAPI returns a so-called JSON object. The handling is therefore possible in any programming language.

This JSON object contains the configuration of the displayed contents (field configuration), as well as the contents themselves. Further processing in another application or user interface is thus easily possible.

The lists as output of many data records (e.g. as a selection result), as well as the search configurations of the portals, the data record configurations, rights and roles and also pivot lists for aggregated information are available in this way. The logic of the application itself, the data model for the objects and automatically running processes are thus available via the CortexPlatform in a configurable way. These functions are used via an individual application.

Individual applications

Figure 8: Individual form in a separate app

From the point of view of software development, the separation is clearly visible here in the sense of the paradigm “model - view - controller”. Since the Model (database) and Controller (logic) areas are configured on the server side of the CortexPlatform, only the View (the application) needs to be developed.

For the implementation of automatisms and other code, the database is also suitable as a place for storage and execution. Without any further data flow, the information within the CortexDB can be selected, read and modified to write new findings into existing or new data record containers.

Summary and perspective view

With the UniplexAPI the use of all configurations is possible. The implementation of individual applications based on the UniplexAPI is therefore quickly possible.

The structure of the CortexDB allows the use and selection of all information by the integrated index. This eliminates any adaptation to index structures. With the data sets (containers) and with the complete data model, you can work just as flexibly and agilely as in the rest of the project.

Furthermore, the CortexDB requires only low hardware requirements (low footprint). The operation in small environments (e.g. single board computers or microcontroller boards like Raspberry Pi etc.) is therefore possible.

Additional information

Dr. Barry Devlin

“Making Data Agile for Digital Business” - http://bit.ly/CtxDB-9sight-1902

“Managing Data on Behalf of Different Actors” - http://bit.ly/CtxDB-9sight-1903

“Distilling Deeper Truths from Dirty Data” - http://bit.ly/CtxDB-9sight-1904

“CortexDB Drives Agile Digital Transformation” - http://bit.ly/CtxDB-9sight_1905

CortexDB

News and tutorials, tips & tricks about CortexDB…

CortexDB

News and tutorials, tips & tricks about CortexDB, CortexPlatform, showcases, sucess stories and much more.

Cortex AG

Written by

Cortex AG

Cortex AG provides the #cognitive #database CortexDB with automatically 6th normalform as content index. CortexDB can be used for app development and #BI

CortexDB

News and tutorials, tips & tricks about CortexDB, CortexPlatform, showcases, sucess stories and much more.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store