Clear signals: Enhancing communication within a data team

“I thought they were going to do it..”

If you hear that often, your team is suffering from conversational misalignment. Clear communication is the lifeblood of successful data engineering projects, yet it’s often overlooked in favor of technical skills. The consequences can be severe: breaking changes left unreported, critical commits miscommunicated, and team morale plummeting as a result.

As communicators, we bear the responsibility of ensuring our message is understood. But this is easier said than done. Go too deep, and you risk losing your audience in technical jargon. Stay too broad, and you may end up stating the obvious or being too vague to be useful.

In this blog, I’ll share insights from my experiences on overcoming these day-to-day communication challenges within data engineering teams.

Relevant XKCD comic

Small talk, big impact. Everyday communication for engineers

Hey, do you have time for a quick call?

2 hours later…

“Quick calls” often turn into long discussions, revealing hidden issues between coworkers. These talks can drag on as people try to find the real problem behind the question. This often turns seemingly innocent questions to lengthy problem-solving sessions.

Effective communication within teams can be as challenging as the technical problems we face. A lesson I learned the hard way was the importance of not jumping straight to solutions without properly framing the underlying problem. A colleague once pointed out during a discussion, “Hey, don’t express problems as solutions.” This simple advice has stuck with me ever since.

There’s plenty written about communicating with clients and management. Presenting a slide-deck that you have thoroughly prepared is one thing, but it’s an entirely different thing to communicate well with your peers on a day-to-day basis.

When frontend meets backend — a communication breakdown

During a recent project to improve our data pipeline’s reliability in our testing environment, a miscommunication led to unexpected issues for our front-end team. They weren’t aware of our testing schedule, and made manual backups of their work data. This data included breaking changes, and had not been tested yet. They did not know that, and it disrupted their work. This highlighted two problems: poor communication about ongoing tests and the ineffectiveness of using casual slack messages for important updates.

Initially, I started working on a complex solution that allowed us to load new data on another testing environment. However, before implementing it, I took a step back to reconsider what I really wanted to achieve. Instead of focusing on the immediate technical problem, I thought about our main goal: ensuring our front-end team could work efficiently with reliable data.

By shifting my perspective to this end-goal, I realized I was heading down the wrong path. In a discussion with my team lead, we developed a simpler, more effective solution. We decided to automate the backup process, ensuring it would happen at the right time and with the most up-to-date, stable version of our work.

So what?

By focusing on the ultimate goal rather than the technical specifics, we saved quite a lot of time. This experience reinforced the value of starting conversations at the beginning of the issue, rather than with the proposed solutions.

In any communication, it’s crucial to express the ‘so what’ of a situation. This ‘so what’ varies depending on your audience. When talking to clients or stakeholders, it’s the synthesis of your findings. When communicating with colleagues or management, it’s the essence of what you’re trying to achieve. By providing this context upfront, you ensure a more comprehensive understanding. This approach often leads to better outcomes, as it allows everyone involved to grasp not just the solution, but the entire journey that led to it.

A framework for clear and productive team discussions

Drawing from past miscommunications, I’ve developed stronger, more effective communication strategies. When engaging with peers, I approach communication with a problem-solving mindset. Here are the key steps I follow to align clearly with my conversation partner:

1. Identify the Root Cause: Before proposing solutions, thoroughly understand what needs fixing or improvement.
2. Provide Context: Describe the problem in detail, ensuring everyone understands the situation. Don’t shy away from stating the obvious. For example: “The goal of this conversation is to find a way to mitigate X. I’m raising this point because it impacts Y.” While these points may seem clear to you, remember that others don’t share your perspective.
3. Deep Dive: Delve into the issue. Tailor your communication to your audience — technical details for technical peers, and higher-level summaries for non-technical ones.
4. Enter Problem-Solving Mode: Transition to brainstorming solutions. Present your ideas along with their pros and cons. Be open to counter-arguments and avoid becoming too attached to your own plan. Remember, the goal isn’t to be right — it’s to solve the problem.
5. Summarize: At the end of the discussion, summarize the key points and agreed-upon actions. This helps ensure everyone leaves with the same understanding and clear next steps.

Navigating Feedback

They say feedback is a gift. But sometimes it can be a little bit hard to unwrap it. Knowing when to detach personally from the feedback and when to take it to heart is crucial for your professional growth. Giving feedback can be even more intimidating, especially when addressing superiors or colleagues.

Receiving Feedback

Detaching Emotionally: Not all feedback should be taken personally. For instance, if your code doesn’t meet team standards, remember that it’s not a reflection of your self-worth. Code is a tool — a means to a business end — and not a personal expression. Learn to detach: delete without remorse and refactor without guilt. This mindset helps you focus on the objective quality of your work rather than seeing it as a personal critique.

Taking it Personally: Sometimes, feedback is personal and should be treated as such. When comments pertain to the quality of your work, your productivity, or how you communicate, it’s crucial to engage in self-reflection. Look in the mirror and ask yourself why you behave in certain ways. Recognizing your ego and avoiding the victim mentality are vital steps in using feedback constructively. Willingness to receive tough feedback can distinguish you positively in the eyes of your colleagues.

Giving Feedback

When preparing to give feedback, consider the following steps:

1. Choosing Your Battles: Knowing when to speak up and when to hold back is a skill honed through experience. If you feel strongly about an issue, it’s often a sign you should address it.
2. Preparing Your Approach: Schedule a one-on-one meeting and plan your feedback carefully. Frame your comments constructively by:
   - Clarifying why you’re bringing up the issue
   - Discussing how it impacts you and the team
   - Offering specific examples and potential solutions
3. Delivering with Empathy: Use “I” statements to express your perspective without sounding accusatory. For example, “I noticed that…” instead of “You always…”
4. Encouraging Dialogue: Make the feedback session a two-way conversation. Ask for the recipient’s perspective and be open to their input.

Effective Communication During the Development Process

How does this communication strategy fit into the development process? Effective development is all about teamwork, continuous feedback, and making small, incremental changes rather than solitary deep dives.

1. Start Small: Focus on small code changes that fix specific issues instead of huge ones that change everything. This makes reviews easier and progress smoother. Regularly update your team on progress, challenges, and next steps.
2. Iterate and Validate: Implement one feature at a time, validate it, and then move on to the next. This step-by-step approach helps catch issues early and keeps the process flexible. Use these iterations as opportunities to practice giving and receiving feedback effectively.
3. Keep Reviews Simple: Make your code changes clear and easy to understand. A format I like is:
   - Context: ‘The goal of this change is…’
   - What changed: Function A was refactored to take X into account
   - Notes: Code can be improved but a quick fix was needed
4. Practice Problem-Solving Communication in Meetings: Use planning meetings and retrospectives to practice problem-solving communication.
5. Continuously Improve Communication: Make a habit of seeking feedback and working on your communication skills, just as you would with any other aspect of your work. Regularly evaluate and refine your communication approach to ensure that you’re effectively collaborating with your team and contributing to the project’s success.

Conclusion

In this blog post, I’ve highlighted the importance of clear communication within data engineering teams and provided strategies to enhance it. These include:

• Focusing on the root problem (so what?)
• Using a structured framework for productive discussions
• Learning to give and receive feedback effectively
• How these communication strategies can be adapted to fit seamlessly into the development process.

A lot of this might sound obvious in retrospect, but many of us learn these communication skills the hard way, through trial and error. Clear, effective communication is as crucial to successful data engineering as technical expertise, yet it’s often overlooked or undervalued. By focusing on problem-solving communication and learning to give and receive feedback gracefully, we can significantly improve our team’s efficiency and success rate.