I read ThoughtWorks Technology Radar Vol.18 so you don’t need to

Statement for beginning: I’m reading Technology Radar every quoter for years, and I love it.

FYI: New edition can be found here. Enjoy!

And now we can start.

For those who don’t know, ThoughtWorks is Consulting Company — the one which has Martin Fowler and Rebecca Parson in their team. Due to type of their business, they have opportunity to work with different clients, from different industries, with different problems. Variety of use cases allows them to evaluate different techniques, platform, tools, framework and languages, suggesting what is really worth considering and which are overhyped and problematic in long run.

Due to that, they are able to create Technology Radar. It’s their quarterly report that is highly opinionated view on what happening in digital business. From my perspective, they are Zeitgeist of our industry — sweet spot between not-yet-worth-your-time and everybody-already-bored-with-the-topic looking. Unfortunately, while being full of knowledge, it’s also a bit overwhelming due to it’s structure. Blockchains, Clouds and mobile development are mixed together so it’s really hard to retrieve knowledge specific to the category we interested in.

That’s why I decided to do an analysis of current edition and present you conclusion that emerge from it. I also allowed myself to add a bit of my own analysis over it. I hope it will be an interesting lecture both for those who already read it whole, as well as those who never heard about it.

Have a good time.

Blockchains and Distributed Ledgers

What I consider really valuable in Technology Radar is a fact that it’s not hype-driven — editors are rather restrained from putting every new-and-shiny thing that snoops through Hacker News headlines. That’s why they hesitated a lot before adding Distributed Ledgers to the report. Even when ICOs were booming around a world, there were still just minor mentions of Blockchains. Now, when a dust settled and we live in the “post-revolutionary” world, battle-tested technologies can be found in the Radar. By browsing it we can easily distinguish two main players on the market.

The First is Hyperledger, a “permissioned” distributed ledger. It is targeted especially for enterprises and backed by big companies like IBM and Intel. Hyperledger graduated from assessing to trial phase, which means it can be considered as the right tool whenever applicable business case looms out. It happened mainly due to Composer, a framework that brings really pleasurable developer experience for the platform.

The second one is the Ethereum platform. It’s heavily represented in the report with Solidity (Smart Contract Language used by Ethereum), Truffle (Application Development Framework) and Open Zeppeline (layer over Solidity for writing secure smart contracts). Those two platforms are becoming major players for anybody who wants to integrate Distributed Ledgers into their company processes.

Editors from ThoughtWorks also see the importance of interoperability between blockchains, showcasing Interledger protocol already used by Ripple. We can additionally find a mention of Corda ledger, distributed alternative for Blockchain, specifically centered for Business usages. While stability problems of this rather young technology are still a problem, a variety of possible usages puts that technology in an interesting position between competition.

Zeitgeist: a Blockchain is no longer a hackish tool, and currently a lot of tools are going mainstream. To “sell-out” to businesses, they’re making developers experience far more pleasurable while providing a variety of covered business scenarios. Permissioned alternatives to typical Blockchains, like Hyperledger and Corda, are gaining interest. However, Ethereum is still the king of decentralized use cases, and with the richness of ecosystem around, it will be hard to dethrone.

Mobile

Not a lot of novelties in that section, apart of a set of proofs that Mobile market definitely matured.

Google is in the process of Android development standardization, making it more pleasurable and efficient. Projects like Flutter (Dart based framework similar to React Native) and Android Architectural Components try to define how applications should be created on Google Platform. The former is especially interesting, bringing the possibility of decoupling applications from Android itself, which can be valuable as Google is investing in projects like Chrome OS or Fushia. Meanwhile Ribs from Uber is the opponent of the latter, proposing a different approach to mobile architecture. They’re targeting both iOS and Android applications, they are gaining a lot of community support. It’d be interesting to observe results of this competition.

Though Android developers have been using Reactive Programming for a long time, on iOS it’s recently gaining popularity. It’s reflected by the introduction of SwiftNIO in the new Radar edition. Similar to Netty, Event Loop based framework is getting traction, and being a project from Apple itself definitely helps there. While not reactive by itself, SwiftNIO is already important building block for emerging toolset.

What’s more, we can also find interesting utilities for resolving a problem which frightens current mobile market — users are downloading fewer applications, which enforce packing more functionalities into the existing ones. To not go insane while working with those bad monolithic codebases we can use tools like Atlas and Beehive, which are a pragmatic solution for that problem from Alibaba for respectively Android and iOS.

Lastly, new Report tries to help you with testing mobile applications — both on a low level, promoting techniques like embeddable mocks that decouples your application from external dependencies, as well as presenting powerful platform like Appium, allowing distributed automated tests on multiple devices.

Chatbots

Not much to see here. Which is also an interesting message.

The most notable thing we can see in this edition is that there is no mention of lock-in platforms like DialogFlow or Amazon Alexa Stack anymore. In their place, ThoughtWorks specialists promote Rasa — open source alternative that allows you to create your own conversational API without sharing your data with external vendors. Even with all current buzz about them, it clearly shows reality where there are not much real business use cases that can justify investment in that exciting (from the tech enthusiasts point of view) technology. Maybe Google Duplex will change that.

While chatbots and VA seem to be in hiatus on West markets, when we take a look at East ones, there is a clearly different story to tell. WeChat is becoming a de facto main platform that can be considered “another internet” in China. During progression to that market, you definitely cannot miss integration with it. While many different platforms try to replicate that success (I’m pointing at you, Facebook Messenger and Slack), there is definitely long way ahead to achieve such level of market penetration.

Frontend

It’s good to see that Angular vs React is no longer interesting topic. The community already split and there is no worth in discussing usage one of the main player over the other. The only “framework” that is mentioned (but only in assesment phase) is hyperapp, mainly due it’s minimalistic approach and growing community. We probably can consider the topic of JavaScript frameworks closed for now, at least from ThoughtWorks point of view.

That doesn’t mean nothing interesting is happening on front-end. Microservices also penetrated this environment and Microfrontends are emerging as viable solutions for products maintained by multiple teams. What’s more, tooling is evolving — with alternative packed managers (like yarn to nom), alternative build tools (Parcel to Webpack) and static analysis tools (like Flow to TypeScript). That allows to quickly test new ideas and (like it was with Yarn and NPM) coerced faster evolution of already established tools — something that always bring value to the ecosystem.

GraphQL doesn’t lose its pace — Apollo is mentioned as a tool that should be at least tried, and with the suggestion to also use GraphQL for backend aggregation, we probably can start to carefully consider GraphQL as not a fad, but rather technology to stay with us for a longer period of time.

WebAssembly is a future — it still needs a lot of polishing to be considered a production ready, but if somebody wants to jump on its bandwagon, that’s the best moment — maybe not yet this year, but probably soon we will surely hear about interesting use cases of this huge revolution coming to the web.

Last but not least, CSS world is also evolving — new standards/approaches, like CSS Grid and CSS Modules, prove that community started to get together how to solve problems of probably the messiest part of a frontend stack. About time.

Backend

I remember that when .NET Core was announced, it was viewed as “THE END” of .NET technology, the swan song of Microsoft stack. Now, we can consider it as one of the most important Platforms on the market, gaining popularity on multiple Cloud Platform (not only Azure) and with the ever-expanding ecosystem (like also mentioned Swashbuckle). Futurist of our industry often misses with their predictions heavily.

For JVM Platform, Kotlin is getting a lot of popularity and overshadowing previous Poster Childs, like Scala, Groovy or Clojure. The latter is still discussed in the Report as a language that powers Clara Rules — promising rule engine with great DSL in Clojure. Meanwhile, mentioning of Reactor is cementing the fact that with Reactive Stream, added to JVM Platform, reactive systems become more and more natural way to write Java Applications.

Security

Security always was, and always will be, the crucial part of developers work — especially among corporate entities, which are the biggest clients of Thoughworks. That’s why every edition of the report contains a lot of tooling that can be used to preserve invaluable corporate secrets and customers data.

Starting with techniques, we have Sidecar for endpoint security. Sidecars are a promising architectural pattern that helps preserve core logic of application clean, while delegating necessary complexity, like security, to external, proxy process. It seems to be a great addition to other architectural trends.

Different security auditing tools can be also found in the report — Scout2 and cfn_nag for AWS, nsp for Node Applications. What’s more, Infrastructure configuration scanner itself can be found beyond techniques that are worth assessing. When we add dependabot, which automatically check the freshness of your dependencies, to the mix, we are receiving a powerful combination of protection layers.

But not every attack vector can be predicted upfront. That’s why Security Chaos Engineering is a powerful technique that allows you to always be prepared by continuously self-attack your system. While hard to be implemented properly, it cannot be dismissed easily due to its much potential.

There is also a special place in heart of report editors which store tools that help you with authentication of users. Hosted Identity as a service is really promising technique, while Keycloak is an interesting open-source platform that helps you with authorization and authentication in Microservice world.

No system is safe if credentials are not protected properly. GoPass is the example of a tool that helps with distribution of credentials in a safe way. Still, we need to remember that no tool can help you when you don’t have a good culture of working with passwords in your company. If your infrastructure is automated properly, implementation of three Rs of enterprise security (rotate, repair, repave) is a good way to make attack layer very thin.

With all the talk about security, we cannot forget that it cannot paralyze your organisation. There is one thing that connects all developers — everybody hates VPN. That’s why in the report we can find BeyondCorp — Research Paper created by Google, describing good practices, that after implementation allow employees to connect with secure services from the open web, without proxies or VPNs. I definitely want to live in the world where every company implemented those techniques.

Testing & Monitoring

That’s category is surely significant part of Technology Radar.

Between standard testing frameworks (like Enzyme for React and AssertJ for Java) and well-known techniques (like Chaos Engineering), many more unusual but interesting solutions are promoted. It’s easy to spot that it was the important theme during the creation of this edition.

There is a tool for load testing (Flood.io), the reader is pushed into the usage of Jupyter Notebook for testing automation. We are also able to see original ideas like… Unit Test for architecture. Additionally, we have WireMock, which is a great tool that can be used as a mock server which allows you to replace your dependencies under testing harness. Everybody should find something for himself.

Moreover, definitely the king there is coping with UI in E2E testing suits. A diversity of tools is astonishing. We have Headless versions of both Chrome and Firefox, we also discover BackstopJS, that use them for performing regression testings. What’s more, if you want to fully register the behaviour of your application under such a test suite from CI environment, you can generate such a screencast using Cypress. Last but not least, for Mobile Developers, there is Appium Test Distribution, that allows you to execute tests on multiple devices simultaneously. UI regressions were always the hardest and most expensive part of the test suite, so it’s great to see a lot of tooling emerging to help you fight off that category of bugs.

Monitoring

While very little attention to monitoring tools was paid, those items we can find in the report are impressive. People from the monitoring world definitely like to do abbreviation, and to the widely used ELK (Elastic Search, Logstash, Kibana) stack we can add now TICK stack — Telegraf, InfluxDB, Chronograph and Kapacitor. Influx has a lot of positive buzz in the community, so it’s good to see ecosystem build around it is growing.

Another interesting tool we can find in the Radar is Sentry. In the world where JavaScript errors are often undetected, Sentry allows developers to react very quickly, providing great customer satisfaction level.

Wrapping up, while not being a ReallyBigThing(tm), Log Level per request is a technique that should be both used by developers, as well as supported by logging tools. Probably anybody who has ever dig through unnecessary large production logs with a lot of noise will agree with me in a spot.

Containers

Ok, Kubernetes is a winner. We can go to next section.

Nobody is discussing Docker Swarm or Docker Enterprise Edition (which itself supports Kubernetes out-of-the-box currently) as viable alternative anymore. Every important cloud player has it’s own Kubernetes tooling (Google Kubernetes Engine, AWS Fargate powered by Elastic Container Service (and EKS in near future), Azure Service Fabric powered by Azure Kubernetes Service) and ecosystem of tooling around this technology is emerging with unprecedented speed — Technology Radar mentions not only Package Manager (Helm) and operation toolbox (kops) created by Kubernetes team itself, but also monitoring (Sonobuoy) and service mesh implementation (Conduit), which are tools created by external businesses that want to monetise on Kubernetes popularity.

The more players invest in technology, the greater is chance for it to stay. Having all big cloud providers and portfolio of external companies is a sign of further consolidation for Kubernetes Stack. They are quickly proceeding antifragile path.

Honorary mention there are Windows Containers — Microsoft invested a lot in that technology and they currently can be admitted production ready. With “native” Docker support for MacOs, containers cannot be considered as just Linux domain anymore.

What is intriguing is fact as little coverage Docker had in this report. That isn’t a good sight for the company which pioneered containerization trends. We will probably see in the near future if they still have any great idea to monetise their industry-breaking technology.

Machine Learning

While describing Contenerization trends, I wrote that Kubernetes is a clear winner. We have the similar situation with Machine Learning. Highlighting PyTorch as easy to grasp and comfortable to use don’t change global scope: TensorFlow is the main platform for those interested in Deep Learning, and its advantage grows every day with the whole ecosystem rising around it.

That’s why I’d like to focus on less obvious trends and tools. First of them is Jupyter Notebook — I see a lot of enthusiasm for this open-source, more hackish replacement for old-good Mathematica (fond memory from lovely University years) and with the variety of usages (analytics, testing) it’s starting to emerge as a very versatile tool for anybody interested in Data Science. I tested it myself recently and I’m surely going to use it more in future.

It’s good to see that, in a world of fancy frameworks, people from Thoughworks don’t forget about old, good mathematics that is the basis of all Computer Science stuff and they’re highlighting CVXPY. It’s great to see that libraries which resolve mathematic problems in an elegant way are still able to get the attention of both community and consulting companies.

Cloud

If I would need to show my favorite, most impactful suggestion flourishing from current Technology Radar, I would choose what ThoughtWorks wrote about the usage of Cloud in business.

They promote usage of different clouds, always choosing best tool for the job. What’s even more interesting, they strongly oppose using only generic solutions from cloud providers and argue that by only taking bare VMs or storage we’re missing competitive advantage brought by their most valuable, lock-in solutions. It’s always a hard decision, but we cannot forget that fearsome vendor lock-in is just kind of risk which should be weighted against gain we can achieve using highly specialized elements of cloud providers toolchain.

In a market where even market leaders have competitions that are right behind their backs, creating your homemade solutions for problems which are already solved by huge teams of developer hired by big cloud players is always a balance between time, control, and cost. Relying your business heavily on specific provider can definitely shot you in the back, what history is showing us (Parse, we’ll remember!), but sometimes that is risk worth taking, especially if cloud competence is just a mean to a goal, not your strategic advantage.

Databases

High Availability is a theme which connects all the products from this category.

Proprietary, Globally Distributed Databases are on the Spotlight. Both Microsoft’s Cosmos DB and Google Cloud Spanner are the best of the kind, approaching the problem in different sides, still solving very important customers problems — an ability to store data in the distributed, but consistent and high available way. What’s more, they are solutions that provide you as much throughput as your wallet can handle. While vendor lock-in is a possible issue there, ThoughtWorks suggest that they are an interesting way of solving a variety of challenges.

However, if you still want to produce your own high available database, the third position — Patroni by Zalando — should interest you. It’s template for creating your own HA PostgreSQL, with battery (monitoring, orchestration) included. It is described as a battle-tested solution which resolves many different edge cases, especially those you don’t even think about until starting implementation of your own DB cluster. What’s more, their support Kubernetes (there is a lot of Kubernetes in that article).

Architecture

Current architecture trends are always well represented in ThoughtWorks Technology Radar.

Firstly, as in previous editions, there is the promotion of Lightweight Architectural Record usage. I’ve got opportunity recently to join a team who was executing that technique for over a year and I’ve seen personally how great it is for new people who join your organization.

Architecture Fitness Functions are a concept that got a lot of traction in 2017, being represented by two loud (while not universally well appreciated) books: “Clean Architecture” from Uncle Bob, and “Building Evolutionary Architectures”, which was co-authored by Rebecca Parson, CTO of ThoughtWorks. While it’s still too early to say if that’s theoretical concept will take off, I’ll be observing it’s future influence on the industry.

Service Mesh is also represented in the report, both as the technique itself as well as Kubernetes implementation named Conduit. In the world where microservices are not “hip” anymore, Service Mesh emerges as an architectural trend that will be a theme of conference speakers throughout next years.

There is also a lot of interest in covering techniques for inter-service communication in the report. We have several event-connected concepts: how strong we should rely on them in the truly event-based systems (EventSourcing as a source of truth), or how consumers should drive their design (Domain Scoped Events).

Moreover, the suggestion of GraphQL usage between services, not only for the frontend applications definitely attracted the eye. I’m also very happy to see explicitly pinpointing that your event queue (and Kafka specifically) shouldn’t be used in the same way old Enterprise Service Bus was. I already have seen trends going in that direction so it’s time to be more vocal about how bad that idea is. We cannot allow history to come full circle.

Serverless

Serverless platforms definitely went mainstream. They are so mainstream that now our industry starts asking hard questions — about the orchestration of many small functions in the way that don’t make our teams insane. One of the possible solutions is a usage of API Gateways to group a lot of smaller functions together. ThoughtWorks is reluctant to this — on one hand, they’re promoting Kong API Gateway as an interesting idea, while warning about overusing API Gateways in a way that make developers life miserable. Golden mean is needed in every context.

From the tool side, in place of the previously mentioned Serverless framework, in this edition, we can find it’s the biggest competitor, Apex. While not the most flexible on market (supports only AWS Lambda, while Serverless Framework works well also with Google Cloud Function, Azure Functions, and IBM’s open whiz), Apex is less complex and provide nicer developer experience for those who don’t need the support of many platforms simultaneously. Especially as Generic Cloud Usage was already mentioned as a problem of our industry.

CI/CD

It’s interesting to see how little coverage Continous Integration/Deployment has in comparison to previous editions. We received only two tools (modern, quite similar to themselves CI/CD Systems) and one technique for using Pipelines also to handle our “infrastructure as a code”.

While the synergy of IaaC and continuous deployment seems like a really exciting idea, underrepresentation of CI/CD is a sign of that we should start considering it the solved problem, especially in comparison to the situation from a few years ago. It’s fabulous to see that our industry matured in that respect and we can focus on other, more burning problems.

Culture

To better understand this section, we need to remember who’s a major client of ThoughWorks — big corporations. Such organizations have their well established internal structure and a lot of stabile (sometimes called legacy) products. That brings a plenty of problems — I suppose the majority of you reading it is working in such companies, so you can imagine).

There is this cult-classic book called “Innovator’s Dilemma” which describe in convincing (and frightening) way all the forces that prohibit change (and innovation) in such an organization. Domain Driven Design brings a solution for the problem of cohabitation with legacy services on an architectural level — anti-corruption layer. In new Technology Radar, it escalated from technological to the team level. Whatever we call it Autonomous Bubble Pattern (a wonderful name that I started using in a spot) or intrapreneurship, I’m really happy that ThoughtWorks try to codify such approach, cause when you have a catchy name for it, you can start mentioning it’s necessity on the meetings :)

Autonomous Bubble Pattern, from my perspective, is a necessity if we want to use of the second promoted technique — Applying product management to internal platforms. I suppose everybody was working in the past on the internal product, one developed without concept and idea, pushed around by clients. If you treat your internal service as a product, you are gaining a power — the power to choose a direction of your development efforts. You are responsible for your own success — it’s hard to find something more motivating.

IoT

In IOT space, LoRaWan is the talk of a community — next generation layer of wireless communication between devices with low consumption of power. While not being enormously popular standard yet — a process of the widespread testing started and possible potential is currently evaluated.

For those of you interested in robotics projects, two positions from the current edition of radar will be interesting. Firstly, we have GoBot, brother of popular Cylon.js. Go is the language that found a lot of interest of people from IoT community, especially due to it’s easy to grasp concurrency model (still I see a lot of investment into Rust, so we’ll see who will be a winner of this battle).

As the Arduino is still the major player for hobbits robots constructors, Moongose OS that major focus is on commercial implementations — use cases where Arduino is too high level, while Native SDK is too low level. Support from Google Cloud Platforms and AWS IoT is definitely a big pro for the platform.

Wrapping up, from my perspective the biggest thing from the report is growing support for WebBluetooth standard. Lack of support for Bluetooth, the major protocol of smart devices, was a big gap between web and native applications. Great to see this rift is smaller and smaller every year.

Other

There were also few things from “platforms” category that didn’t fit into any of my sections.

Godot is competition to Unity, specially fitted for VR experience. Contentful is API-First CMS for developers, and if you need an even simpler tool, Netlify seems as good way to deploy static page very quickly.

Still, my pick from that section (while being probably the least useful from my own perspective) is Language Server Protocol — tooling that Microsoft uses in their Visual Studio Code editor for smart syntax suggestion. Great to see such things as open-sourced.

Conclusion

Uff, that was long. If you got there, thank you. I hope my “reverse index” Technology Radar with a bit of comment made it a bit easier to grasp. I appreciate all the feedback — I hope that with every quarter, I’ll be able to produce such as wrap up better and better :)