The experiences of Product Software Engineers in Customer Success

Article written by Sheng Wang, Miguel Cruz and Henrique Ferrer

As a software engineer, how do you think a multinational tech company works? At Feedzai (a FinTech company that fights financial crime using AI and machine learning), there are two major departments: Product and Customer Success (CS). To explain what they are, we, software engineers from the Feedzai Product Team, had the opportunity to experience the CS team’s routine to better understand both departments and get more certainty on how to describe each of them.

1. Product: where the Feedzai’s products are developed.
2. CS: where Feedzai’s products are actually requested and provided to clients.

However, the real purpose of this blog is to share our experiences with the CS team as Product engineers. The following sections outline the experiences of three of us — Sheng, Miguel, and Henrique — who joined Feedzai as Product software engineers and had the opportunity to also work as CS engineers.

Product vs CS

The main difference between the Product team and the CS team is their objectives. While the key objective of the Product team is to improve the product, by developing new features, testing, and fixing bugs, etc. The CS team works to tailor the product to the customers’ needs. This means that the CS team deals with all the problems that might be client visible, deploys new versions of the product, and adds custom features that clients want.

Sheng

In September 2019, I had the opportunity to join Feedzai as a junior Software Engineer.
My journey as a Feedzaian with Product as part of the Machine Learning team to focus on the heroic mission: Fight against Fraud.

After two months working on the Machine Learning team, I received a proposal to temporarily join the CS team. Whenever something occurs that you don’t expect, the first reaction would be to ask, “Is there something that I did wrong?” It turns out that it is a normal thing to happen when the company wants their employees to learn new experiences. In the CS team we are working directly with clients and see what their requirements are and what our product actually does, i.e., a big picture of Feedzai’s product. I realized quickly that it is a great opportunity to learn directly from users’ real-world experiences.

I considered it a win-win proposal. As a software engineer, we rarely know what a client really wants. This is a great opportunity to gain some real-world insights.

As my journey began in CS, I met my product manager. The first thing that he emphasized was that the client doesn’t know what they want, so we need to build a Minimum Viable Product (MVP). In other words, we don’t build a car if the client just asked for only four wheels. Even if (they think) they want a car.

These were very different procedures from the Product team where all members worked 100% for the same project. In contrast, on the CS team, members can work on three different projects from different clients during a single day of work.

At the beginning I thought, isn’t that inefficient? Because based on the book, Scrum: The Art of Doing Twice the Work in Half the Time, engineers lose a great amount of time to context switching. But the projects from the same team are supposed to work like that because most of them are similar to each other.

I worked on the integration of our product with a third-party score classifier. It was very challenging, because I wasn’t working on something on the scratch. Instead, I was working on an existing project. I managed to work with several tools in a real project, I faced several multithreading problems, I found bugs that were left because at the time that part was done in a rush, and I managed to write readable documentation for engineers who will eventually work on those parts in the future.

I also experienced the blue-green deployment. This has a huge impact on our client because we are deploying to the production environment, i.e., clients’ working environment, thus we need to ensure that everything is functional. Basically blue-green deployment is a process that creates a new independent application and after its initialization we can swap the old application with this new one. After deployment we used to save the old application for 24 hours as backup if the new one fails for any reason. I have learned no to deploy applications on Friday and don’t deploy at all if there is something that doesn’t feel right, because by Murphy’s law, it won’t work, and it will only bring more cumbersome work to our support team.

To sum up my experience in the CS team, I really enjoyed working with them. Although it isn’t the job that I see myself doing in the long-term, it was indeed a good experience for me, and I wouldn’t mind repeating it again for another short term.

Miguel

It was the end of April when my manager and his manager scheduled a meeting with me. I wasn’t sure what I should expect from that meeting. Once they explained to me that I would need to help the CS team for a while I must admit I was not happy, but I was ready to start this challenge.

I was changing from the guy that delivers and fixes the product features to the guy that deploys the product and has to deal with all the problems that might be client-visible and find a way to fix them. This was all I knew about what was waiting for me “on the other side”.

I was allocated to the Anti-Money Laundering solution. This means that Feedzai already had a big part of the work done, since it’s one of the solutions offered from the product.

The work had nothing to do with what I was doing in my previous team. Coding was almost out of my weekly tasks. I had to upgrade the solution version, migrate changes across environments and fix problems with cloud deployments.

While I wasn’t 100% happy with the arrangement, it helped me build a broader vision of the product and its different use cases. It also put me in the user’s shoes, since I had to work with the product in real production environments and not a simple development environment. I also had the opportunity to interact with some areas of the product outside of my team’s responsibility that I had little knowledge about.

Since soft skills are something that everyone likes to talk about, this experience made me improve mine, most likely more than my technical skills.

By April, the pandemic had hit Portugal. I was working from home a little over a month, still adapting to a remote life, when I had to shift to a team that was US-based. The other two engineers of the team, the project manager, the client, everyone except for a fraud analyst were US-based. I never had onboarded onto a team from a different country, let alone while working from home.

I had to understand all of my responsibilities and how to perform them with a team with a five hour time zone difference. It made me improve my capacity of adapting to different environments and circumstances in little time.

One of the engineers in the new team was a new hire, and I ended up mentoring him in his first months. Not so much on the CS work side, because I was also a newbie on that part, but more on the product side. This challenged me, not only to improve my mentoring skills, but also to improve what I knew about the product, since I had to consider the new hire’s questions and sometimes question my own knowledge about some particular cases.

Even though I was not completely happy about this temporary change at the beginning, looking back, I think it was a positive experience. I didn’t improve my programming skills while I was there, but I believe it made me a more experienced engineer, with knowledge in broader areas of the job, the product, and even the company which will improve the quality of the work I deliver from now on.

Having the opportunity to work with people from a different department that have a different vision of things, also helped me see things from a different angle. I would say it was the closest I can get to working for a different company without actually leaving Feedzai.

Overall, I’d say it’s interesting to experience a different environment within the same organization as long as you feel supported throughout your stay.

Henrique

COVID-19 brought plenty of uncertainty to all of us. For Feedzai, it brought strategy changes. For me, in addition to some of my colleagues, it brought one big unexpected career path shift.

It was April when it was told to me for the first time that I would join the CS team and leave my software engineer role within the Machine Learning team. To be honest I didn’t really understand what that meant for my future but I gladly accepted as I saw it as a big opportunity to learn more, improve my skills, and get some good understanding on how Pulse (Feedzai’s main product) is being used by the end user.

May 1 was my first day with the new team. I was introduced to the clients that my team was taking care of and what I was going to be my first task: implementing a 3rd party product integration using an API and testing it. This was a perfectly good example of how my daily job was going to change. From fixing, making tests, and build new features for the product to customizing, maintaining, and deploying tailored versions of the product to meet the clients’ needs. My job went from mostly coding to toggling configurations and finding the best ways to meet the client’s high expectations.

To be honest, since I am in the CS team I had a bitter-sweet experience. I learned a lot, mostly on the soft skill part of things because I needed to communicate better, needed to have great teamwork since we were all working from home and had to be very adaptable since I had not done anything remotely similar to this. Technically, I also got the opportunity to learn much more about Docker, AWS, and some other 3rd party tools for CI/CD. My team was always there for me, were friendly and helpful, which was super nice. In the end, it’s been a great opportunity to learn and discover more about the product and how people actually use it which is very important for someone who wants to develop new features and to improve the overall product the client is using.

Looking back, I see this was a good decision by Feedzai. But personally, what I disliked the most was the timing of this change. Since all the employees were working remotely (because of COVID-19), I did not have the chance to meet anyone personally on my new team. Actually I still haven’t had the chance at the time of this writing. This made the onboarding process a little bit more cumbersome and the integration a little bit trickier and with a few more hiccups. This being said, since I was replacing a former Machine Learning team member that was returning to the Product team (Sheng) I had some help getting started.

Last but not least, I think this experience was quite nice and that I am improving in almost every way. This took me by surprise because in the beginning I was not completely sure if I would like it even though I was 100% on board with the challenge.

I see this now as a great opportunity to see how things work on the “other side” and learn how Cloud OPS work and how everything is connected from the development of the product to their deployments and even their usage.

Conclusion

We all agree that CS is not the perfect world, but these experiences really helped us gain a broader vision of the company. Furthermore, it helped us to understand better how we are supposed to deliver our products to the client and what are the difficulties we usually face delivering it. From a personal interest perspective, it provided us with an opportunity to improve our soft skills by speaking with clients and how we should address their difficult requests.