The Agile Way To Engineering Decisions

|
Moving quickly vs moving slowly

At Samsung Ads we love moving fast to stay competitive. However, balancing speed with risk is crucial. While moving quickly can give us a competitive edge, it’s important to weigh that speed against potential risks like getting locked into a vendor, exposing sensitive data, or facing unpredicted costs caused by a lack of cost analysis. So, strive for a balance.

The first step is to assess how risky a decision might be. Ask yourself:
Is this a high-stakes decision?
Are there any vulnerabilities?
Is this decision reversible?
How much will it cost to reverse this decision?

If a decision is low-risk or easy to reverse, then it’s a good candidate for quick decision-making. This way, you avoid getting bogged down in too much analysis and can adapt more easily to change. On the other hand, if a decision is high risk, it’s worth taking the time for a thorough evaluation. By initially assessing the risk, you can better decide how fast to move, helping you be both agile and cautious.

An example of a decision that is easily reversible: Picking an open-source relational database that supports SQL
Rationale: Check the most popular SQL databases, see which ones are supported by your cloud provider and you can narrow it down to popular candidates such as MySQL, MariaDB and PostgreSQL. Picking either one is easily reversible as long as teams are sticking to standard SQL features. For the most part, it would be an easy migration if you ever decide to revert the choice to another SQL-compliant database.

An example of a decision that is not easily reversible: Picking a cloud provider
Rationale: Say you’re weighing the options between AWS, Google, and Azure. There are a lot of dependencies, many stakeholders, huge security implications, long-term contracts, a lot of vendor lock-in, etc. Reversing this decision would be expensive.

When Options Analysis Goes Wrong

Regardless of the type of decision you’re working on, sometimes we notice that the execution of the options analysis process is either moving too fast or too slowly.

Moving too fast 

Couple of signs you’re moving too fast:

  • Decisions are made without collecting sufficient information or understanding of the impact. Don’t let your opinions get in the way of real analysis. You can certainly include new tools in the analysis, but stay objective and document your thinking process.
  • Stakeholder exclusion – Not enough time is given for consulting key stakeholders, leading to a lack of buy-in or unforeseen objections later on. In these cases ensure that all the right stakeholders are consulted. Explore if other teams have done or are doing similar work to yours.
  • You haven’t done any hands-on exploration of options or you are relying on other people’s experiences with it.

To correct these, allocate time for comprehensive evaluation and stakeholder feedback. Also, re-evaluate the levels of risks associated with your decisions. Try things out.

Moving too slow

When you’re moving too slow it’s typically for these reasons:

  • Analysis paralysis – an indecisiveness caused by an abundance of data leading you to believe that there’s more research required to identify the best option and that the winning option will become obvious
  • Feature and scope creep – spending too much time comparing features which aren’t even required. Vendors are good at providing an abundance of details. Without a quick analysis and hands-on experience, it’s hard to eliminate or narrow down the options
  • You have no fixed timelines

To find the balance between speed and quality of research, accelerate the process, time box your work and define hard deadlines for each step of your process.

By being aware of these signs and corrective actions, you can better navigate between the pitfalls of hasty decisions and full indecision. Adopting a framework that encourages both moving quickly and covering all the bases will enable you to make decisions that are timely yet well-considered. 

Options Analysis Process 

These are the steps of the Options Analysis process: 

Step 1. Decision Classification

Determine if the decision is high-stakes and irreversible or if it is reversible and less consequential. Allocate time and resources accordingly. Refer to the beginning of this page for the importance of this step.

Step 2. Scope and Time Definition

Define the scope and requirements of your decision. Avoid being influenced by feature creep especially when analyzing multiple vendor offerings. Stick to what is required at first and define the realistic set of minimum requirements. Ensure you’ve defined clear timelines for each of the steps in the process to ensure you don’t get trapped in analysis paralysis.

Step 3. Criteria Definition

Establish the parameters or criteria that will be used to evaluate each option. These can be both quantitative (cost, time, ROI) and qualitative (user experience, strategic fit). 

Popular criteria include:

  • Functionality – Does the software perform the tasks and processes that are defined by the scope?
  • Scalability – Can the software support growth?
  • Security – Has Security reviewed it?
  • Reliability – What is the uptime, and how does the vendor handle outages?
  • Reviews – What are real users saying about it on Gartner, G2, TrustPilot, etc.?
  • Total cost of ownership (TCO) – Consider both the upfront costs and ongoing ones as well as costs to maintain by our staff
  • Support – Is it supported? By which company? What level of support is offered?
  • Reputation & Popularity – How likely is the software going to be supported in the future?
  • Ease of use – Is it user-friendly?
  • Learning Curve – Is it easy to start using it without training?
  • Customizability – Can it be modified to meet our specific needs?
  • Legal and Compliance – Does the software meet relevant legal and compliance standards? How easy is it to change contract terms?

Step 4. Data Analysis

Conduct a preliminary analysis of both data and user impact within the clearly defined scope

Step 5. Hands-On Evaluation

Rather than relying solely on hypothetical data analysis, conduct proof-of-concept evaluations or pilots to practically assess the software’s capabilities, limitations, and alignment with project requirements and scope. Capture your experience, learning curve, ease of installation, support required and received from vendors or OSS community, etc

Step 6. Stakeholder communication

Inform all the stakeholders of your progress for feedback and validation of concerns

Step 7. Decision and Documentation

Make an informed decision about the recommended option based on your data, hands-on experience, and stakeholder discussions. Document your process by using the options analysis template showing the story of how the process of elimination happened and why the selected option makes the most sense at this time. Don’t forget to indicate risks and options to reverse decisions if this is the case.

Step 8. Sign up and present to your lovely neighbourhood Architecture Review Board (ARB)

Now that you’re done, get your work approved. The best practice is to sign up for ARB as soon as you think you’ll be done within the next three weeks. By signing up in advance you’d lock in the slot and you’ll give all architects a heads-up that your presentation is coming. Ask your lead architect for a review ahead of the ARB date.

Too long, didn’t read

Options analysis is a decision-making process in which different choices are evaluated to determine the best fit for specified needs. It starts by determining whether it is an easily reversible decision or not, which impacts the level of detail you’ll need to achieve in your work. Some decisions are not (easily) reversible and they need deeper analysis.

The options analysis process also includes setting evaluation criteria, picking the right options, collecting relevant data, trying them out, and then assessing each option based on these criteria to make an informed decision. The goal is to create an objective assessment of pros and cons as well as to mitigate risks while aligning with specific objectives and sticking to a predefined scope.