Job: Data Engineer @ NYC Planning [FILLED 4/4/2019]

Join a modern tech team building the future of city planning

Application deadline: March 18, 2019 or until filled

In a nutshell

We’re hiring a Data Engineer to grow NYC Planning’s data engineering team. We’re looking for someone highly-motivated and innovative, ready to modernize dataset development by automating processes, and ensuring procedures are reproducible and transparent.

You will be involved in the design and implementation of entire data pipelines, from capturing and storing disparate data sources to processing that data and making it available to DCP staff and digital applications. You should understand data, how to improve its quality and accessibility, and be excited about advancing the adoption of new data technologies in New York City government. If you’re interested in shaping the future of city planning and building smarter government, you should definitely apply.

Already interested? Apply here.

About NYC Planning’s data engineering team

The Enterprise Data Management team maintains core city datasets such as street centerlines and building addresses. It ensures that DCP Planners have access to quality data to inform decisions that ultimately shape NYC. Data engineering, a unit within Enterprise Data Management (EDM), develops new data products and reengineers existing ones, such as PLUTO, using modern open source technologies. These datasets feed a myriad of systems used by City Planning, other city agencies, and the public as well as supporting a variety of apps and analyses that inform policy decisions.

We intend to make our mark by automating the production and QA/QC of high quality datasets, with full transparency. We’re a small, growing team, and you’ll have a chance to make an immediate impact. We’re going to trust your opinion and expect you to help us make the right decisions. You’ll be empowered to dive into our processes, test new tools, and give your thoughts on how we can do better.

We are only a small part of NYC Planning, New York City’s planning agency, one of the largest city planning agencies in the world. You’ll have a chance to work with brilliant colleagues, many of them renowned experts in their fields. You’ll learn first hand how New York City runs.

What will you be working on?

You’ll have the chance to work on EDM’s growing portfolio of initiatives. Some of the products and projects you’ll get to work on include:

• Developing and maintaining data pipelines, with a focus on writing scalable, clean, and fault-tolerant code to handle disparate data sources
• Automating quality assurance, creating reports and visualizations to debug data processes, check input data, and provide data users with insight into data issues
• Working with data owners, city agencies, and the public to improve the quality of existing data products, create new data products, and improve data production and management practices across New York City.
• Documenting data idiosyncrasies, bringing order to chaotic upstream data workflows, guaranteeing seamless use in downstream applications.
• Sharing your work and lessons learned via blog posts with colleagues and the world
• Cross functional projects with NYC Planning Labs

What tools will you be working with?

The primary technologies we use but are certainly not limited to are:

• PostgreSQL/PostGIS
• SQL, Python, Bash
• Spark
• Airflow
• Geosupport
• Git/Github

Ready? Apply here.

You should apply if:

• You have 2 or more years of data engineering experience (or relevant education) building data pipelines with SQL and Python (we use a combination of SQL, bash, Python scripts, and Jupyter notebooks to generate, QA/QC, and manage our data. We’ll be looking to you to help improve these processes.)
• You are very organized and thorough, with a proven ability to bring order and logic to heterogeneous data environments.
• You have worked on modern data engineering teams and understand the importance of collaboration, teamwork, and writing clear code for your colleagues.
• You are comfortable communicating about data with colleagues of varying levels of proficiency.
• You are familiar with spatial data formats and have experience working with spatial data in PostGIS and/or Carto.

What you get

• Salary: $60,000 — $74,000, commensurate with experience
• New York City health benefits, including full dental coverage
• 15 paid vacation days a year, with an additional 11 paid Federal holidays
• Eligibility for a host of NYC retirement plans
• Agency-issued MacBook Pro and dual-monitor workstation
• Modern office in downtown Manhattan
• Strong work/life balance

Apply here. We look forward to hearing from you!

We’re also hiring for:

• Full Stack Developer (Planning Labs)
• Interaction Designer (Planning Labs)

Know anyone?