Apache

Author | Ouyang Tao, Big Data Development Engineer Apache DolphinScheduler (hereinafter referred to as DS) is a distributed and easily scalable visual DAG workflow task scheduling system, dedicated to solving the intricate dependencies in the data processing, thus making it be used out of the box in the data processing…

A reference guide. — Data Engineering is vast and rapidly evolving. It could be confusing to choose tools from the plethora of options. Therefore, in this blog, I have consolidated 120+ tools/solutions of a modern data engineering lifecycle. Agenda — Typical Programming Languages in DE Data Exchange and Serialisation Distributed File System (DFS) List of 20+…

What I learned while dealing with vulnerabilities Background BHI Security 팀에서 Internal VAPT 수행 시, 내가 관리하고 있는 smokeping* 서버에 TRACE Method가 열려있다는 문제가 발견되었고 이를 수정하기를 요청하였다. * Smokeping은 네트워 …

For this project, we will be utlizing the CLI to create an Apache web server on an EC2 instance. Prerequisites AWS Free tier account AWS CLI installed and configured Vim is installed on your CLI AWS IAM non-root user with administrative permissions Objectives Create a free tier t2.mirco …

Spark: A Data Engineer’s Best Friend — Data engineering, as a separate category of expertise in the world of data science, did not occur in a vacuum. The role of the data engineer originated and evolved as the number of data sources and data products ballooned over the years. Therefore, the role and function of a data…

What should be completed prior to creating the EC2 instance from the CLI: Access to your AWS account AWS CLI installed on your OS Follow the link below to setup if not already completed: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html Getting a new key pair and setting up a new security group Step 1: At the CLI enter the following command to create a key pair: aws…

How to choose the right data scheduling tool? This is a question for all data engineers. Xu Haihui, a software development engineer at China Mobile (Suzhou) Software Technology, said: “As the old saying goes, ‘If you want to do something well, you must first make use of the tools’. If…

For this project, I will be showing you how to create an EC2 instance with an Apache server using the AWS command-line interface. The AWS CLI allows you to interact with AWS services using commands in your shell. Prerequisites You will need the latest version of AWS CLI, I will be…

公司開發的 APP 每年都需要通過資安認證，所以 APP 中有提供連結的網站也需要進行弱點偵測，由於沒有專門的網頁資安人員，這部分是委託資安公司進行網頁弱點掃描。 整個滲透測試流程會先取得網路及網頁掃描結果，並進一步滲透攻擊， 將相關測試結果撰寫檢測報告並提供修正建議，所以不用擔心不知道如何修正弱點，不過當然程式還是要自己寫啦。 Burp Suite 資安公 …