FOSS — The foundation of Capillary Tech

Why is FOSS so important to the community ?

• Startups would not have been possible without FOSS, since it’s very time and cost effective to get started with openly available free, reusable software components. This has a direct and immeasurable impact on all the innovation coming through across the world in various fields.
• We can all learn from open source code as there is no differentiation based on geographies, race, gender etc.
• We all use it all day everyday directly and/or indirectly.

How is Capillary dependent on FOSS

Capillary Technologies, like almost all other companies, started off completely on FOSS projects. Right from the operating system to the programming languages, databases, caches, web servers, queuing systems, front end components, web sites and so on, FOSS is almost always the first choice. The free and openly available set of libraries, services etc has exploded in the last couple of decades giving us all, a lot of options to choose from to suit the problem at hand. FOSS kicks in at each and every stage of handling a transaction at Capillary.

FOSS powering Front End at Capillary

Though we use a lot more libraries to power our front end, we would like to highlight a few critical FOSS components and its usages.

• X6 JavaScript Diagramming Library — https://x6.antv.vision/en

X6 helps us create a canvas on which our users can easily drag and drop components to form flow charts. Since this library supports custom components of other JS libraries(eg: React components), it saved us a lot of dev effort as we could reuse existing components.

• C3 / D3 JS — https://c3js.org

Provides the base for charts and visualisations on the UI. Easy to use, and very well documented with a huge developer base, has support for most things out of the box and highly customisable as well.

• Antd / Antd-mobile — https://ant.design

Forms the base design library for all the UI apps to be used with react. We use many components from Antd directly and have also written custom wrappers on top of a few for better flexibility.

• react-intl — http://formatjs.io/docs/react-intl

It helps to set up internationalisation in all our front end projects. It deals with localised messages, date and number formatting and parsing, bi-directional text, and other internationalisation issues.

• CK Editor — https://ckeditor.com
  CK editor used in our product to configure and compose email templates. It is a rich text editor and provides a lot of useful features out of the box simplifying composition of template content.
• ExcelJS — github.com/exceljs/exceljs#readme

Used for downloading reports from UI in excel format and adding custom formatting to it. It helps us put org related analytics/performance data in excel and share them via engagement channels.

• JEST — https://jestjs.io

A comprehensive JavaScript testing solution which works out of the box for most of our JavaScript projects. It helps us build unit testing structure on React and Node applications. We are grateful to the community for providing such a useful library by contributing bug fixes and improvements regularly.

FOSS powering Backend at Capillary

We would like to highlight a few of our critical components built completely on FOSS.

• Twitter Finagle — https://github.com/twitter/finagle

Finagle is an extensible RPC system for JVM written in scala. We use it to build high concurrency application servers

• Apache Spark — https://spark.apache.org/

Used for performing ETL, processing, querying and analysing customer interactions and data throughout the customer lifecycle. We use Apache Hadoop Yarn for resource management and job scheduling, Apache Hadoop HDFS for file systems and Apache Hive for data warehousing.

• Grafana — https://grafana.com

Grafana dashboards bring data together in a way that is both efficient and organised. It allows us to better understand the metrics of our backend applications performance data through queries, informative visualisations and alerts.

• Prometheus — https://prometheus.io/

We use Prometheus as our metrics server to capture a lot of system and business metrics. Lot of alerts, monitoring dashboards are powered by prometheus. It also allows for anomaly detection alerting us of some unexpected behaviour.

• Apache Livy — https://livy.apache.org/

We use Livy to submit jobs to Apache Spark and it help us interact over REST.

• Maxwell by Zendesk — https://github.com/zendesk/maxwell

Maxwell powers our Change Data Capture mechanisms for transforming MySQL bin logs to JSON messages and pushed to RabbitMQ brokers for async processing.

Conclusion

• We accept we have taken a lot more than we have given back. We have a few ongoing and upcoming initiatives to increase our contribution to FOSS
• The company gives back to the open-source community by reporting bugs and patches to projects like Apache Livy, Apache Spark, Maxwell, and Azkaban.
• Sponsoring FOSS Hackathon 2021 is our small way to help the wonderful community.
• We will share a much more exhaustive set of FOSS technology at play in solving specific problems. This, we feel, will help everyone become aware of the available tools which scales reasonably well. Would also love to hear back regarding other tools which are better at its job compared to what we are using.