👍 For news and articles from Skills Matter, subscribe to our newsletter here.
As major tech conference programmes are built, themes can often emerge and become central to the discussions that occur. This has never been more obvious than for this year’s Scala eXchange London.
As our incredible programme committee has worked to build an expert-led programme for this year (including having the creator of Scala, Martin Odersky, give a keynote speech), five key themes have emerged that are set to drive the growth and direction of so many scala-based projects in 2019.
Matt Roberts from Disney Streaming Services is speaking at this year’s Scala eXchange, focusing on type-driven development as an approach to writing programs.
“We’re constantly looking for ways to improve the way that we write programs, and putting types (and type safety) first is something that has really helped us do that. I also think that focusing on types and the type system leads to a broader discussion than if we just discuss functional programming or object orientated programming in isolation.”
The core advantage to type-driven development lies in its ability to help developers maintain and test their code, which is a reflection of functional programming’s biggest appeal today: its relationship with building concise code.
In relation to deep learning research, Xiayun Sun (Joy) sees a major advantage in using types in Scala and is basing her talk at Scala eXchange on her observations of academia.
“I spend lots of personal time following deep learning research. Despite recent exciting developments, the engineering side of things hasn’t really caught up. This all makes me think if there’s a better, more robust way to do things. As I use Scala in my day job, and I’ve experienced the benefit of using types to help model programs to be less bug-prone, I wonder whether we can apply the same philosophy to research.”
Scala eXchange London is Europe’s largest Scala conference. Check out our interview with The Guardian’s Maria-Livia Chiorean on using Scala and this year’s programme at Scala eXchange.
4. Apache Kafka
There is no real surprise that Apache Kafka is such a discussed tool at this year’s Scala eXchange. The demands of big data streaming and processing continue to grow as the IoT becomes a reality and more sensory points are added to the tiniest of daily interactions. Then there are major data requirements, such as in geospatial technologies or fraud detection, which require tools that can handle real-time data feeds.
Some data naturally comes as a never-ending stream of events. Batch processing requires you to store it, halt the data collection to process the data before you move onto the next batch and try to aggretate across these multiple batches. Streaming this data is an obvious solution.
Using tools like Apache Kafka enables you to use the data that you need, and can help when you are looking at giant data sets that may be even too large to store.
Deliveroo are one such company that have adopted Kafka for their data requirements. Lindsey Dew and Omnia Ibrahim work with the tool at the food delivery company and have come across a number of challenges and solutions when using Scala with Kafka. Their talk at Scala eXchange is highly anticipated for the revelations it promises to bring for those developers who are striving to build data-driven projects.
3. Apache Spark
A data processing engine, Spark can be used for streaming but also offers libraries for SQL, machine learning and graph computation.
Spark is run on four languages (Scala, Python, R, and Java) but Scala’s affinity with Spark is that: it is compiled, JVM-based, less verbose and concise (features of a functional programming language), is scalable (due to its modularity) and has OOPS support. Python and R are not JVM based and have less scalability, while Java is more verbose.
Holden Karau, open source developer advocate at Google, is delivering a talk at Scala eXchange on the potential for Spark to be applied to validation, to avoid headaches post deployment and even automatically detect issues through validation in a real-time system. You can check out Holden’s talk on Spark from last year’s Scala eXchange for free here.
Meanwhile, Anna Bladzich and Adam Davidson from Elsevier are talking about how Spark can help developers and data scientists collaborate through the CF (Collaborative Filter) algorithm on Spark, futher supporting this growing idenfitication of the need to merge functional programming with data processing tools to shift both approaches and coding philosophies.
2. Tagless Final
2017 was said to be the year of Tagless Final by some in the community. A relatively new addition to functional programming, Tagless Final is a pattern that essentially helps developers build a subset of the host language that is safe (invalid states can’t be expressed), allowing them to declare domain-specific languages generically, without specifying a monad.
The Tagless Final approach is applicable to many scenarios. Speaking at Scala eXchange, Viktor Lövgren has built the Typelevel Ciris library to make it easy, safe, and secure to write configurations as code.
“Ciris has a Cats Effect integration and works really well with a tagless final approach. It’s a great way to write functional programs while dealing with side effects.”
A strong discussion point for this year’s Scala eXchange in London, the Cats library allows you (according to Matt Roberts) to “steal, define, refine, and abstract away types so that code is easier to maintain and to test.”
Developed by Typelevel as a spiritual successor to Scalaz, Cats is meant to act as a foundation for an ecosystem of other libraries, taking a modular approach (which sets itself apart from Scalaz and mirrors the inherent strengths of functional programming).
According to a 2017 Stack Overflow survey, Scala is the second highest-paying programming language in the US, which is no doubt because of the boom in both functional programming as it meets the needs of big data processing, as well as Scala’s advantages over other languages (being scalable and JVM-based).
Richard Dallaway, a member of the Scala eXchange Programme Committee, sees great potential for Scala, which is supported by the growing numbers attending functional programming conferences such as Scala eXchange.
“I suspect a large proportion of Scala code is deployed in enterprise-ish places. With Scala Native on the way (and with Scala.js already out), there’s potential for Scala to appear in more places.”