Hello, World! We’re the Data Engineering team at NYC Planning
Data Engineering is a new data development team within the NYC Department of City Planning’s Enterprise Data Management (EDM) group. We are passionate about data, and everything associated with it, from how it’s created and documented to how it’s shared and used. We believe that data are foundational and we’re excited to bring data to the forefront at NYC Planning.
What do we do?
As data engineers we develop data products. We’re responsible for creating new databases and modernizing the creation of existing datasets, such as NYC’s definitive tax lot dataset (aka PLUTO), for NYC Planning. The data we produce are used by planners and civic technologists alike in analyses, which help inform decisions that ultimately shape NYC. Our data products also power downstream applications, such as NYC’s Zoning and Land Use Map, used by planners, policy makers, and the public. Therefore, it’s imperative that we publish data and supporting documentation of the highest quality.
We’re not just excited about the data products we build, we’re equally passionate about how we build them. It’s important to us that our development processes are open, transparent, and reproducible. To ensure this we use modern and open source technologies, such as PostgreSQL and PostGIS, as well as open data sources, and publish our code in the open on GitHub. We want a data user to easily understand what a dataset is and what each field represents via accompanying documentation, but enable them to dive into the code used to create the dataset to see how we calculated a value or even generate the dataset themselves.
Working with and for users
We don’t publish datasets just for fun. Rather, we want to make sure that the data products we publish satisfy end user needs. As a result, we solicit, prioritize, and incorporate feedback from data users like yourself. We recognize that we are not the subject matter experts or end users of the data products we develop; therefore, we partner with business owners, stakeholders, and end users to ensure that the data we create are accurate and valuable, and we’re looking forward to growing these partnerships.
Lastly, it’s part of our mission to educate users about our data products. We recognize that metadata is important, but it sometimes can be overwhelming so data users don’t always read it or easily understand the information it contains. Therefore, in addition to standardized metadata we’ll be publishing associated ReadMe files and blogging about our datasets to make information about a dataset and how to use it more accessible.
As we grow and take on more projects we look forward to growing our data user community via data users groups to get direct feedback from you on the data products we build. Let’s do this!
