Embracing decision making
Did you ever notice that “What are we having for lunch?” is sometimes difficult to answer? What about “What should I wear today?” or “Should I use the train or bike to work today?” or “What should I do after work?” or “Should I have a pint or half-a-pint?”, etc.
Throughout the day, we face multiple decisions both major and minor. In this article, we will go through some tips that help me navigate the easy and difficult decisions that I deal with daily.
Software architects face different situations where they make decisions, weigh options, accept circumstances, and deal with consequences. During my journey as an architect for the past 2.5 years, I learned some techniques to help me navigate these decisions.
Avoid impulsiveness
Sorry to disappoint many of you, but a perfect solution does not exist.
Impulsiveness can lead to hasty decisions, negative consequences, or even risky situations down the road.
Instead of looking for the perfect solution or promoting the first solution that comes to mind, I recommend investigating further and identifying different solutions. Take time to think, research, and list as many solutions as possible, even if it doesn’t seem obvious or efficient at first.
Embrace imperfection
wabi-sabi (侘寂) is a Japanese concept that aims to embrace imperfection and transience while appreciating its beauty. Mistakes are inevitable but are a good indicator of learning and growth.
Try your best to ask the right questions, understand the context, accept feedback, and highlight strengths and weaknesses so you feel empowered to make the decision. Moreover, you can put in place guidelines to limit the impact of these mistakes.
Change is inevitable
The only constant is change. — Heraclitus
Products evolve, requirements change, expectations may vary, and assumptions might be challenged. Thus, any decision may or may not be the right one given the constraints known at the time. Be careful of context changes that would make or break your decision.
Think of a solution as a function that changes with time, adapt accordingly, be patient, focus on the process and not the end goal, and think of it as a marathon instead of a sprint. Also, be ready to review your decision as new information emerge.
Don’t be afraid to delegate or ask for help
No one thinks I am right more than me! After all, we should support ourselves and trust our abilities and decision-making. However, we always have too much on our plate, which could become daunting, so we tend to look for: shortcuts, workarounds, quick wins, or work solo assuming we know it all, etc.
I recommend involving the right people at the right time, trusting their experience, delegating the choice (if it falls within their expertise), and collecting feedback whenever possible.
Be objective
Personal preference is not an indicator of the effectiveness of an option. Prior to decision-making, each option must be analysed. Start by gathering enough information to assess each option, keeping the analysis factual, and presenting data to back up your facts. A template I use: for any decision, list the workflow complexity, implementation cost, data transfer cost, operability cost, impact on design/quality, etc. By employing this approach, I can quickly identify the advantages and disadvantages which avoid making biased or subjective decisions.
Should I doubt myself? 🤷♂
Before making the final decision, assessing the risks helps avoid falling into the self-doubt trap. Among the existing frameworks, I mostly use the Worst Case Consequence Analysis framework or WCCA.
The framework consists of asking 3 (maybe not so simple? 🤔) questions:
- What is the worst-case consequence for this option?
- What is the probability of this outcome?
- Is this an acceptable risk?
Putting it all together
Let us consider a scenario where we design a reporting system for a standard 3-tier application architecture that uses a relational database.
The first solution that comes to mind is to use the same relational database. To avoid impulsiveness, we should investigate further the reporting requirements. For example:
- Identify the data sources (tables associated with each report)
- Analyze the transformations for each data source (aggregation, filtering, indices to apply)
- Identify the volume of the data (10 rows, 100 rows, 1M rows, etc.)
- Determine the access frequency (daily, hourly, etc.)
- Check the freshness of the transformed data sources (real-time, daily, etc.)
Once the requirements and constraints are defined, we can suggest alternative solutions. For example:
Disclaimer, a simple analysis was done to explain the process.
- Tables and Views on the primary database
Complexity: Easy, no additional implementation to support a different connection string or data store.
Advantages: Simple implementation, no additional inferred cost, no data duplication, ideal for small reporting purposes, perfect for low transformations, no latency.
Disadvantages: not suitable for complex data transformations, not appropriate for frequent access, performance impacts on multiple reads, performance impacts on large data sets, performance impacts on reading data in write-lock mode.
WCCA — Consequences: impact on production, slowness, costly solution.
WCCA — Probability: High.
WCCA — Acceptable: No. - Tables and Views on the read-only replica
Complexity: Medium - set up the cluster for the relational database and include an additional connection string for reporting requests.
Advantages: Simple implementation, no data duplication (excluding mirrored data), additional inferred cost, ideal for small and large reporting purposes, minimal latency, suitable for low transformations, applicable for frequent access, performance impacts on multiple reads.
Disadvantages: not suitable for complex data transformations, performance impacts on large data sets.
WCCA — Consequences: additional cost, performance issues with complex transformations and large data sets, inconsistent data from sync latency.
WCCA — Probability: Low-Medium (depends on write frequency).
WCCA — Acceptable: Yes. - New reporting tables on the read-only replica
Complexity: Medium/Hard - set up the cluster for the relational database, add reporting tables, update the data in the new table (events, triggers, schedule, etc.), and include an additional connection string for reporting requests.
Advantages: additional inferred cost, additional implementation cost, ideal for small and large reporting purposes, ideal for low transformations, minimal latency, suitable for frequent access, no performance impacts on multiple reads, appropriate for complex data transformations, performance optimizations (indices) simple to manage without impacting write performance, minimal or no performance impacts on large data sets.
Disadvantages: medium/complex implementation, data duplication (saving data under a different format), additional data workflows, managing extra schemas and codebase.
WCCA — Consequences: additional cost, additional data workflows.
WCCA — Probability: Low.
WCCA — Acceptable: Yes. - New tables on a reporting/read-optimized data store (ElasticSearch, TimeSeries DB, Data Warehouse, Cassandra DB, generated files)
Complexity: Medium/Hard - set up the additional resource(s), set up data migration pipelines, and manage data workflows (events, triggers, schedule, etc.).
Advantages: additional inferred cost, additional implementation cost, ideal for small and large reporting purposes, ideal for low transformations, minimal latency, suitable for frequent access, no performance impacts on multiple reads, appropriate for complex data transformations, performance optimizations (indices) simple to manage without impacting write performance, minimal or no performance impacts on large data sets.
Disadvantages: medium/complex implementation, data duplication (saving data under a different format), additional data workflows, managing extra resources and workflows.
WCCA — Consequences: additional cost, additional infrastructure.
WCCA — Probability: Low.
WCCA — Acceptable: Yes.
Then, we can identify the impactful constraints such as data volume, aggregation, data access frequency, and data freshness (schedule or real-time). The target is to select the option that matches the long-term plan but also fits the short-term objectives.
Finally, we can present the solutions to developers, database experts, architects, ops engineers, and other relevant team members. They will help you assess the maintenance, implementation, and operability costs and challenge the proposed options.
Decision-making can be challenging, and as your level of responsibility increases, so do the choices that you have to make. I use these tips to help me navigate decisions at work. They help me better describe the problem, rationale, and reasoning behind the choices.
I am curious to know your thoughts on these tips. Did you find them helpful? What approaches do you use when making a decision?
Do you have additional tips?
