Star Gazers
Published in

Star Gazers

How to develop software better than a robot

Software development will become harder with time. Don’t stop developing yourself.

Photo by Possessed Photography on Unsplash

Making software is essentially formalizing real world knowledge using formal systems. Real world includes everything we are surrounded with. Software developers write code to solve real world problems. For this, they are acquiring domain knowledge, organizing it, and expressing it in a rigorous way so computers can algorithmically process the knowledge, apply it to input data and produce output data.

One good example to illustrate the above is the concept of a city map. Map depicts a part of the real world using geometrical shapes and signs. Anyone with a map can solve problems related to navigating in the city: Finding places, making routes from one point to another, estimating distances and areas, and many others. Both humans and computers can do it, provided the map is in a clear, “understandable” format. For humans the best format is probably an image, for computers — vector graphics.

When we intend to map some territory, we inevitably abstract ourselves from many details. Each map is an example of abstraction. There are also different maps for different purposes, helping to solve different problems.

It is important to note, that there is a core knowledge about maps (like projections and coordinate systems, vector algebra operations, ways to represent features and points of interest), and then there is a specific knowledge, which is relevant to a subproblem at hand (for example, coordinates of bird nests).

Software is about knowledge

Our software is only as useful as is it’s knowledgeable about problem and solution domains. The ideal situation is that would be a low-code. For example, if we are building a bird observation application, we can just take Google Maps, provide it with coordinates and submission form — pretty much doable as a mashup. This is possible, because our application can leverage established (and standardized) domains, and we just need to make a small abstraction step from seeing a bird at some site to turning it into a point with coordinates. Here it’s quite easy, because we understand that a bird is too small for an area — so it is just a point on a map. For tracing bird migrations we would probably need lines, and as our appetite grows we would probably need areas to aggregate bird sightings. All of those are still very well supported by basic geo-server technologies.

The above demonstrates that a software developer needs to build understanding of the problem domain, find subsystems, which are standard enough for ready solutions to exist, connect subsystems together, and create software for the unique domains. Of course, software can be written in many different ways and using any universal tools, which are out there. This question can facilitate the choice: Which tool allows me to focus on the knowledge representation and processing as opposed to solving technical subproblems?

The resulting system will contain knowledge of a special domain as a sum of knowledge about standard domains and specific knowledge. If specific knowledge is not required, there is no need to write a new software product.

So far all this is quite top-level, and it’s a pity people, many developers included, are not that good at applying general principles to the down-to-Earth everyday tasks. Let’s see more practical hints.

Ontology first

We are at the beginning of a new project and we already know that this specific domain is not covered by other software makers. Books on software design suggest making an analysis of the domain, together with domain experts.

At first, it’s good to write down what is there, or make an ontology. Here we have another possibility to get the knowledge easier — there are a lot of ontologies created for different areas of knowledge. There are probably several ways to describe the domain, and we can get a feeling that we found some new concepts, which do not have corresponding names in our language. Or maybe unsupervised machine learning discovered some topics, which are optimal for our domain, but they are so outlandish, that it is hard to come up with names for them.

One way or another, we have found some thing — entities, we assign identifiers to them, and maybe we even have an idea of their life cycles. This is our first, very approximate abstraction.

Next step is to connect the dots. This is where our ontology will get it’s is-a, has-a, use-a, part-of connections. Here we also separate entities and value objects, because value objects do not have a life of their own. We enriched our ontology at this point, and we have a better abstraction.

At the next level we can go deeper into the attributes of entities, or we can decide that we will not hard-code them at all. This is totally good, and in big enterprise software even entities are not hardcoded at times.

Now we have an ontology. It can be expressed as an RDF/OWL triples, as a database schema, as an object-oriented design, as a collection of data types, as a bunch of logical declarations, S-expression, or maybe a grammar of domain-specific language. This is the basis of our theory, hopefully, informed by real world knowledge.

Do I already hear “overkill”, “overdesign”, “just write it in JavaScript and have a life” from the trenches? Please, have enough time to reflect, and with some experience the understanding comes. Thank you for reading so far: There is hope for better software for all of us.

Ideal software is also cheaper

If ontology has been expressed in some declarative way (that is, it’s data, which can be processed), we can start finding solutions our clients are waiting for. From all the above, perhaps, object-oriented code is the least convenient one, as it does not easily lend itself to experiment with, to “kick the tires” of our ontology, and transform the ontology to a better one step at a time. We can even do it using metaprogramming.

When we are satisfied with our ontology, and it is possible to answer all the questions for the problem at hand, we can be more confident, that many future problems can also be solved. This is because highly abstract representations usually have nice properties like making omissions easy to spot. For example, summing two knowledge graphs is just pouring them together, if we were careful enough not to reinvent ontologies. It is also easier to check our domain logically when factual knowledge and business rules (or code in a computationally universal language) are separate.

If you still do not believe it, look at the big companies. They started developing their knowledge graphs a long time ago, while the rest thought it’s “overkill” or exotic hobby of European Commission, sponsoring Semantic Web research.

For some problem domains software development can utilize the knowledge directly. Business rules engines can operate directly on some formats of knowledge representation, so user interface and/or API is all that needed. Other domains may require optimizations. I claim here that optimizations do not necessitate hardcoding ontology in machine language. Recall that knowledge in the form of data can be easily manipulated, so with the right skills a developer will be able to produce an optimized solution, which can directly follow from the knowledge base. This is more like optimizing compilers, which use many different techniques to produce optimized low-level code given abstract syntax trees.

There is a problem though

Rare reader, who is still with me here, can express doubts this will work. I must admit, I have doubts too. It’s not easy to find developers, who are comfortable with thinking in terms of abstractions and knowledge in the code, thinking in systems of systems. There are of course those, who deal with higher-level abstractions in some enterprise level systems, but I am talking about developers in a more usual sense, crafting their solutions with lower-level tools and frameworks.

And here is the main point: It is possible to learn your way to handle complex abstractions! When one learns that, specific programming languages and frameworks will not overwhelm any more (one can even discard some of them as inconvenient). What is left is the universal (meta)knowledge body (I do not want to call it patterns, maybe — heuristics?) and then of course the domain knowledge. The secret here is that good tools and frameworks already incorporate most of the elements, composability, and perhaps are optimized enough to get the job done. The best is when domain knowledge is quite compact in the code, and there is no need to manually maintain the same things in many places.

Mathematics (what mathematicians study, not necessarily elementary school math) is a good example of the power of abstraction. Mathematical concepts can help with finding abstractions, especially when doing functional programming. Good developers without math education rediscover their own math. Also physicists are very good at math, because it is the language to build their real world abstractions also called models.

Conclusion

The secret is simple. Learned heuristics, tactics and patterns allow one to think about the problem and solution economically. Most of us do not use physical rubber ducks to calculate 2 + 3, so why many developers still do it in software design?

Do not understand me wrong, I do not suggest that software developers should always discover Maxwell equations for the problem domain, but a habit of finding a minimal theory, which describes the problem and enables a solution will lead to faster development over time. To use a metaphor, robots will replace those, who still use a candle to light their work, instead of modern convenient light sources, which would be impossible without knowing Maxwell equations. Unfortunately, most of the technical discussions are around which candlestick is better. If you are not indifferent to your craft, start mastering universal skills today. It will not happen overnight, building a habit to see software development in a holistic, semantic and systematic way requires a lot of determination, practice and theoretical studies. The prize is also high: quality software done right first time.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store