We have a requirement to transfer Parquet data using Azure Blob Storage as medium between two Spark Applications. To achieve low latency, the read side should read data as soon as the write complete, so here I dig into the process of the Azure Blob…

最近遇到一些人想轉職軟體工程師，想說分享一些個人去年的面試經驗，讓大家了解一下軟體公司針對工程師是怎麼面試的，希望對大家有幫助。

To accessing different ADL storage:

1. Get Client ID, Client Secret and Token Endpoint for all your ADLs, you can refer this step by step guide,
2. If you have only…

The story begins with comparing throughput of a service using multiple processes or multiple threads. In multi-processes test, to measure throughput precisely, I want to make sure all the threads among different processes start at the same time, so I use a…