This is the second part of a cycle, where we continue to demonstrate our horizon detection workflow. Crucial concepts of the task, its history, and importance for seismic exploration, as well as its quality assessment, are covered in the previous part of the publication; it is highly recommended to read it before proceeding. This time, we will look at:
- data generation in seismic exploration tasks
- neural network architectures
- techniques to enhance detected surfaces
- infrastructure for model deployment
Once more, let’s dig in!
Data generation
Seismic volumes are generated by sending a (usually acoustic) signal into the ground and registering its response: this way, we can get an idea of the subterranean formation. Obviously, the raw signal goes through a lot of pre-processing procedures before it can be used to actually locate potential reservoirs and structures; nevertheless, during the seismic interpretation stage, one can think of the data as huge 3-dimensional arrays (also called seismic cubes and volumes).
As the entire cube is too big to fit our memory constraints, models are trained on either 2D slices from it or small 3D sub-volumes. To train even a single neural network, one needs to generate pairs of seismic data and segmentation masks (more on them later) time and time again; thus, the speed of data retrieval is crucial. The inference stage is just as demanding: we must iterate over the entire cube to stitch the horizon from small patches.
To speed up both prototyping and actual production-ready models, we’ve developed a custom storage for seismic volumes. It stores the data in a hierarchical way, accelerating loading operation by up to 30 times, compared to regular SEG-Y storage, which is simply not meant to be quick.
Equally important is our choice of locations from where to generate data: we want our training pairs to be close to the horizon surface; it makes little to no sense to sample crops with no training labels on them. To this end, we’ve developed a flexible sampler that can be used to customize the locations of the train and test parts.
These features (and many more) are a key part of our framework for seismic interpretation seismiQB: besides, it has a rich library of augmentations (both mathematical and geological) and tools for processing. You can learn more about it on our GitHub; we are also planning on a separate article to describe it in detail.
Encoder-Decoder conception
From the machine learning perspective, the task of horizon picking can be seen as either binary segmentation, multiclass segmentation, or regression. Each one of them has its pros and cons:
- binary segmentation directly models the horizon surface, which is convenient and easy to work with. Yet, it outputs a mask that must be converted into a depth-map
- multiclass segmentation splits the whole volume into two parts: above the tracked reflection and below it; it has the same problems with mask conversion, as the binary segmentation
- regression allows us to directly predict depths of the horizon at each point; unfortunately, it is hard to train such models as they are very unstable and introduce a lot of jitter into predictions
In this article, I will focus on the first approach, binary segmentation. Our research indicates that it gives the best results, compared to the other two, while the additional complexity of converting the mask to a depth map is negligible. This approach also allows us to utilize a wealth of papers on semantic segmentation and edge detection, ranging from simple UNet to very sophisticated combinations of DeepLab, CPD, and HRNet-OCR.
The rest of this section dives deep into technical details (and even some code). Skip to the next chapter if you are not ready for it!
The vast majority of the modern segmentation networks employ the encoder-decoder architecture: it is also known as the sandglass architecture (not to be mistaken with the sandglass block) and originates from the autoencoders, widely studied in the previous decades. When talking about this model design, it is impossible not to mention the UNet paper, which used it to great success in the field of biomedicine.
UNet is a great example of a simple, yet effective and very popular neural network. On a higher level of abstraction it consists of:
- encoder, which passes the tensor over multiple blocks and sequentially reduces its spatial dimensionality. It also saves outputs from its stages for later use
- embedding, that thoroughly processes the output of encoder: that is the place of various pyramid-like modules
- decoder, that restores the spatial shape of the tensor, applies all the more operations like Conv-BN-ReLU, and makes use of stored tensors from the encoder
On this level of abstraction, one can construct hundreds of completely different neural networks. By changing operational blocks from, say, vanilla Conv-BN-ReLU to a block with residual connections, or with dense connections, or even both, we can modify the behavior of our model to suit current needs. The same is also true for downsampling and upsampling strategies as well: it is easy to seamlessly swap, say, bilinear interpolation to a transposed convolution. This design also includes all of the DeepLab variations, allowing us to effortlessly implement recent papers in our research.
As with a lot of other patterns in programming, our (arguably more abstract) view on the encoder-decoder concept allows for clean and concise implementation. Our library, BatchFlow, allows us to define such models with a clear and straightforward dictionary:
By swapping the DefaultBlock (which implements exactly Conv-BN-ReLU) in the code above with another module, for example, with ResBlock, one can drastically change the model’s performance. This method of model configuration lets us effortlessly explore a huge number of architectures to get the best one; by using it we’ve obtained the best model for horizon detection. We will shortly release a separate article on our BatchFlow library, that extensively explains all of its features for model creation, research, and data generation. Stay tuned!
Horizon detection with neural networks
We took our time to thoroughly revisit the necessary concepts: metrics, data generation, model creation, and research. That said, we are yet to discuss the exact formulation of the horizon detection task from an ML standpoint; specifically, what should be used as train and test datasets. The following options are available:
- train on multiple cubes, inference on completely unseen data
- train on part of the cube, inference on the rest of it
- train on a few slices of data, distributed evenly on the cube, inference on entire seismic cube
Let’s discuss the positive and negative sides of each of them. The first one is looking like the most obvious and most promising: yet, it is very hard to apply in real life. By training on a separate set of cubes, we will create a model that detects some of the horizons on the unseen data. But in reality, geology experts need to track one very specific reflection, not the random ones (usually, the easiest) that are given by the model. Thus, this model can be utilized to retrieve a huge number of labeled surfaces, some of which are useful, but it can’t be the sole model for horizon detection.
The other two options are very similar; the difference is, in the last approach our data to learn from consists of just a few slides — less than 0.5% of the total area. Using the information from the same cube for train and test significantly boosts the quality; it also gives our model a prior on which precisely horizons to track, eliminating the ambiguity of the previous approach. To understand which slides to label, we’ve developed an algorithm for assessing the geological hardness of the field: in short, we want to get more human labeling in the hard places, whereas more trivial locations don’t require that much effort and involvement from the specialists.
That is exactly how we propose to detect horizons; in short, the whole pipeline looks like this:
That’s it! The only step of the list we are yet to cover is the assemble of predictions: this is a procedure of combining individual patches into the horizon, and it is also a part of our seismiQB library.
This short pipeline allows us to track reflection surfaces with high quality and fidelity and takes less than an hour even on the largest fields.
Note that it requires only a few slides to start the process: by carefully choosing the task setting, we avoided (or at least mitigated) the data dependency and use only a handful of labels to guide the model on the entire field.
Enhancing the result
As we can see from the previous image, the produced horizon covers a lot of the cube area…but not all. There are holes in the predicted plane, and our neural network design does not guarantee their absence. Such holes are unacceptable in the resulting horizon and must be somehow stitched.
To do so, we apply one more neural network. The difference to the previous model is:
- it uses the already-labeled surface as the training dataset, not only the carcass
- its final layer is constrained to always output something (via softmax along the depth axis)
This model is looking very like and in the spirit of currently trending (for the last few years) student-teacher neural networks and allows us to create horizons of not only high quality and fidelity, but with 99%+ coverage. Training an additional model and using it for inference adds an overhead to the detection pipeline, but we can mitigate it by carefully choosing places to apply the extension model (only where it is actually needed), resulting in less than 30 minutes for postprocessing overall.
Testing
To make sure that our technology is bullet-proof, we applied it to an enormous dataset of just shy of a hundred horizons on twenty-something fields, developed in the last 20 years. Roughly, for each of them we did the following:
- create a carcass from a seismic cube: its total area is less than 0.5% of field area
- keep only the carcass from a hand-labeled horizon
- train detection neural network on the carcass, inference on the whole cube
- train postprocessing neural network on the prediction from the previous step, inference to stitch all the holes
- compare the original (hand-labeled) surface to the one produced by our models
Each pair of cube and horizon is, essentially, a separate dataset, and when performing research on a huge amount of datasets it is hard to analyze the results. Reporting average values of key metrics is of little use, as it hides the specificity of each of the surfaces; on the other hand, manually inspecting them is practically impossible.
We stand on the middle ground of choosing a pre-defined set of complex horizons and assess their quality in detail. Each of the model-generated surfaces is compared to the hand-labeled one with a number of metrics, and in the vast majority of cases, it is easy to notice the improvements in the detected horizons. This conclusion is confirmed by our geology experts, that carefully examined each slice of the ML tracking.
Infrastructure
Compared to the previous horizon detection methods, our technology is sophisticated and, more importantly, requires specific hardware: GPUs or other accelerators. Deploying neural networks in a large company is not easy, and it is essential to have an effective and well-oiled way of distributing our product; otherwise, the overheads of data transfer would eat all the speedups. In short, to use our algorithm, one must:
- run a docker container with our image from DockerHub (10 minutes)
- pull the seismiQB repository with our code and scripts (1 minute)
- execute a script to convert your cube into much faster storage (5 minutes)
- run a script to apply a set of neural networks onto your cubes and horizon carcasses: it parallelizes each of them into a separate accelerator and keeps a detailed log of each stage (up to 2 hours)
That is it! To keep our data and results of research, we store it in an internal DVC server. It allows us to get the archive projects to run a lot of experiments on a completely new server in a matter of minutes.
Currently, a lot of our efforts are made to testing this pipeline on as many cloud computing services as possible: our goal is to seamlessly run the proposed seismic exploration workflow on any platform with reasonable hardware. To this end, we work tightly with NVidia to improve our usage of accelerators, computing as much of our processing steps on GPU.
Summary
Prior to our work, horizons were picked manually by a team of trained experts, taking months to interpret even a single cube. Results were highly subjective, as well as their assessment: the only quality control was performed by other geologists, with no digitized metric to guarantee correct detection.
In this article, we briefly overviewed the entire pipeline of our technology for horizon detection and its implementation in one of the world’s largest Oil & Gas companies. Our developed method is successfully tested against numerous archival surveys and is currently applied in fields with ongoing exploration, reducing the time to create the structural model by a few months.
We also came up with a number of explicable metrics: they provide critical insight into the quality of a detected surface and can be computed instantaneously. The necessity for them and a more detailed description can be found in the previous part of this article.
What next
In the near future, we plan on investigating even more complex designs of neural networks, as well as changing the task formulation entirely: the carcass interpolation is just one possible way to detect horizons, and there are other ways to do so.
Furthermore, there are also a lot of other structures to be located on the seismic data: faults and reefs, estuaries and facies. By making relatively small changes in our codebase, we are already able to adequately detect them, slowly but surely approaching a fully automated seismic interpretation. Deep investigation of methods to find those objects is our top priority, and you can expect separate publications of our results in a little while.
Over the course of this publication series, we briefly touched on a lot of concepts. Yet, some of them deserve more attention and further elaboration:
- the metrics we came up with are described in the previous part of the article, yet the exact formulae and implementation details are still missing
- description of our libraries: one for general machine learning and one for convenient seismic interpretation
- details of the enhancement procedure that stitches up the holes in the horizons
We are eager to share all of these details about the horizon detection process in separate publications, which will come after this article. At the same time, there are a lot of avenues to explore: various architectures, training procedures, fully unsupervised models, and other formations to detect, that should be covered appropriately.
All in all, we are in for a busy year. Expect hearing from us soon!
