A singularity is a point in time or configuration space in which a system becomes unpredictable or misbehaved in some way. For example, in mathematics, the function f(x) = 1/x is singular at x=0 because it “explodes”; in general relativity, Stephen Hawking showed that the Big Bang has infinite density (i.e., space is point-like) and Nobel Prize winner Roger Penrose argued that black holes necessarily have a singularity; maybe most famously, the technological singularity is a hypothetical point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization (Wikipedia).

Image for post
Image for post

In this blog post…


Last week I found myself giving the same counterintuitive advice to regatta sailors in my local sail club and to one of my team members at Palantir: “Sometimes you have to slow down to go faster.” Let me explain.

Image for post
Image for post

In software engineering

We’ll begin with a software engineering anecdote from my job at Palantir. [Of course, if you’re really just interested in the sailing material, feel free to skip ahead to the second section below!] A new colleague recently joined my team and I began working with him on a new project in the context of Foundry. …


Monotonicity is an important concept in relational databases and, as I will demonstrate in this blog post, also for APIs in service-oriented architectures.

Image for post
Image for post

Intuitively, a database or an API is monotonic if information “does not disappear”. Monotonic database queries can be more efficiently analyzed and optimized than non-monotonic queries, and monotonic APIs can help developers avoid race conditions and undefined state. At Palantir, we obviously care about fast database queries and safe, well-behaving APIs, and I use the framework of monotonicity regularly when discussing database or API architecture with my teams. So let’s dig in!

Relational databases

I suspect that most readers…


Update (15.06.2020): Thankfully, the tracing infrastructure proposed by Google and Apple does indeed protect from the simplistic attack described in this article. However, the tension between privacy, utility, and security remains a tough challenge, as demonstrated by recent research: for example, it seems that a wormhole attack against DP-3T is indeed feasible and can have effects very similar to the ones imagined below.

With new technology come new opportunities for hackers. Instead of discussing attacks on the tracing infrastructure itself (e.g., …


Palantir’s customers use our Foundry and Gotham software platforms to solve their hardest data problems. Many of our customers work in highly-regulated environments, which introduces additional constraints on how we can operate our software. This article presents the complexities of running SaaS platforms in heterogeneous environments and sketches the solutions we have come up with to manage our hundreds of dedicated installations across the fleet.

Image for post
Image for post

One of the most common misconceptions about our software is that “Palantir sucks up all data into its database”, implying that we maintain some form of omniscient data vacuum cleaner that gobbles up data from…


Image for post
Image for post

Being quarantined in my Munich apartment as my Palantir colleagues around the world spring into action to help our customers respond to the COVID-19 pandemic has been an opportunity to reflect on the lessons we have learned as a company over a decade of supporting both crisis response and public health. Palantir has provided our data integration and analysis platforms for urgent public health and disaster relief work through many of our partnerships, whether on long-term cancer research or in-the-moment responses to natural disasters. We have identified a number of cultural and technology practices that are especially valuable in a…


Editor’s note: This is the first of four posts in our blog series “How we do Agile. Part 1 (this post), Part 2, Part 3, Part 4.

Image for post
Image for post
I suspect that these ballet dancers are more agile than any of my teams … ?

Coined in 2001 by a group of developers who had long been rethinking the standard approach to building software, the term “Agile” has become a common buzzword in corporate boardrooms. A recurring theme in my conversations with our customers’ CTOs and other leaders is: How does Palantir do Agile? The question reflects their curiosity regarding the success and failure modes of Palantir’s product development model, and often also the desire to adopt some…


I recently wrote a blog post about the foundational data engineering features in Palantir Foundry: co-versioning of data and code and branching. While writing the post I reflected on my own experience of building data pipelines before Foundry; the lack of strong collaboration tooling in standard big data systems meant that I continually re-invented simple mechanisms for keeping track of data flow through my data pipelines: which dataset was produced by which script, when was it last run, in what order do I need to run the different scripts, is the dataset up-to-date with the latest version of the script…


Image for post
Image for post

If you have visited Palantir’s website recently, you have learned that our platforms let organizations “manage data like software engineers manage code”. Collaboration tools like Git play a major role in the software engineering tool chain: Git commits establish code versions and Git branches isolate developers and features into their own sandboxes. In this blog post, we are going to explore why dataset versioning and sandboxing are critical to collaborative data engineering, in full analogy the importance of Git workflows for software engineers.


According to neuroscientist Michael Merzenich, our brain learns languages in two different modes, depending on when we learn them: Native tongues, i.e., languages learned as a child, are all handled by the same region in the brain, while “second” languages, i.e., languages we learn as an adult, map to other regions in the brain, distinct from the region responsible for our native tongues. In other words, native tongues share computational resources (i.e., neurons), while second languages require additional brain space (literally). Let’s draw an entirely unscientific analogy to formal languages. A language is regular if and only if it corresponds…

Robert Fink

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store