Data Engineering Culture @ FC
Working as a Data Engineer at Funding Circle is a great opportunity for those who have a passion for Data & Technology. Data is the backbone of many organisations today and is the driver behind our platform. In this blog, we will delve into the culture within the Data Engineering team, and the exciting challenges and opportunities that come with it, here at Funding Circle.
How does the team approach problem solving?
Deepti: Having joined DataOps, which is a new team here at Funding Circle, we first brainstorm and try to resolve the problem ourselves. Sometimes, we take support from our wider platform team, who is also very helpful and supportive. So far, it is working for us, and we continue to have this approach.
Henry: Working for the Pink Pandas who look after Data Ingestion, we approach problem solving with an open-minded and collaborative approach. We begin by clearly defining the problem and gathering as much information as possible about the current system and the desired outcome (value delivery). From there, we brainstorm potential solutions and evaluate the benefits and risks of each option.
We make sure to test our solutions thoroughly before deploying them to production. This includes unit testing, integration testing and performance testing to ensure that the solution is robust, efficient, and scalable.
Audrey: Open culture so everyone is comfortable to share their ideas with the team.
What is the most interesting thing you have learned this year?
Deepti: This is my fifth month at Funding Circle, and the learning has never stopped! Every day is exciting and new. I come from a GCP background and am now learning every day and loving learning AWS.
Henry: Making use of GitHub Autopilot to increase my efficiency by more than 20%.
Audrey: Creating streaming pipelines — Kafka, Hudi, Spark.
With our hybrid working model how do you maintain collaboration? (Stand Together)
Audrey: Paired programming. An open culture means you can reach out to anyone, and they’ll be willing to take some time to answer any questions. When we do go into the office, we often have team lunches, drinks after work, and the occasional karaoke/Data Team night out, as well as 3 day hackathons.
Deepti: It is important for our small team to work together. Most of us are new and still learning FC’s methods used in the technologies, so it helps the whole team to have more discussions while working. And scrum ceremonies also help build that relationship, we try to have a bit of chit-chat once the work calls are done. Soon, we are going to start team socials as well.
How would you describe the culture in your team?
Henry: Our culture is one of collaboration and continuous learning. We believe that working together as a team is essential for the success of any project, and that clear and open communication is key to achieving our goals.
Audrey: We share knowledge with each other and encourage each other’s development, e.g. through Lunch & Learn sessions.
Deepti: The data platform is a diverse team, and I am glad that I am part of such diversity where everybody is valued and their opinions matter.
What’s the engineering culture like at Funding Circle?
Henry: Engineers are encouraged to think outside the box and come up with new and creative solutions to problems. This allows us to approach problems from different angles and come up with unique solutions that might not have been considered otherwise.
Audrey: We’re very agile and not tied to technologies, which allows us to come up with and implement novel solutions. This also keeps us motivated to keep tabs on new technologies and best practices in the industry.
What type of projects have you been exposed to in your time with us?
Audrey:
- Getting data from a range of sources e.g. SFTP and applications
- Ingesting it into our Data Lake in S3 and making it queryable so analysts can generate reports.
- Included batch and streaming data as well as monitoring and alerts using PagerDuty.
Henry:
- Near real-time streaming data pipelines
- Monitoring and Observability solutions
- Incremental batch processing on AWS Glue
Which value (Stand Together, Make It Happen, Think Smart, Be Open, Live The Adventure), Obsess Over The Customer) resonates most with your team?
Deepti: It would most definitely be “Be Open”. Since we are a cross-functional team, we have to collaborate within the wider data platform team and with Data Analysts and Data Scientists teams. We make sure to understand and be open about the feedback we get from each side.
Audrey: I believe our team (Pink Pandas) embodies all these values to varying degrees. We excel at Standing Together when it comes to brainstorming ideas and presenting the final solution in a cohesive voice to our stakeholders, always providing support to each other.
What’s your favourite part of being a Data Engineer at Funding Circle?
Deepti: Exposure to creative technologies and the amazing challenges that require debugging and finding the root cause.
Henry: Being part of a close community of brilliant minds with an insatiable hunger for innovation.
Can you share what’s the most exciting project in the works for the new year?
Audrey: Launching our streaming ingestion tooling.
We’ve already done some Beta releases and now we’re almost production ready.
Deepti: Building pipelines for model inference. I have never executed a Machine Learning Model before, but now we are building pipelines to schedule those models. Also, we are recommissioning a repo which involved a lot of understanding of tech used. This is challenging but something new for me.
Henry: Delivering near real-time data streaming capabilities to Funding Circle, empowering teams to make data-driven decisions and innovate at pace.
Thank you for reading! Make sure to follow us on Medium and our new engineering page on Linkedin.
