Iterative Process in Agile: Optimizing Software Development

Since the first iPhone version was released back in 2007, thirty-four iPhone models have been created (as of mid-2023). They’ve come a long way from what now seems a primitive piece of hardware (no offense meant to Apple fans!) to the potent, high-performing, intelligent gadgets we use today. All this time, Apple has continued to make changes in every new version, adding features, altering the design, enhancing performance and security – and always welcoming feedback from its customers.

Well, that's a shining example of how iterative development works – which is the topic of today’s post.

What is an iterative process?

An iterative process, generally speaking, is an approach to solving problems in a trial-and-error style. It involves creating or starting something (a project, product, initiative, etc.), testing it, identifying areas for improvement, implementing changes – and repeating the process over and over until the final goal or result is achieved.

Trial and error be like… Source: theycantalk.com

Such an approach can be applied in various areas of business and even personal life. For example, within a company, the R&D department would continuously test multiple ideas to come up with a new product (be it a new ice cream flavor or a flu vaccine), the marketing department can use it to create the best-performing campaign, the sales team can experiment with pricing strategies to get maximum revenue, and the list goes on. In one’s personal life, it can relate, say, to trying different ingredients to devise the best salad ever.

But for many people, when they hear about the iterative process, the first thing that comes to mind is software development. This approach lies at the core of the Agile philosophy and all the related frameworks (Scrum, Kanban, Lean, Extreme Programming, etc.). To better understand the idea, let’s look at an iterative approach as opposed to a non-iterative or traditional one.

Iterative vs traditional development

Agile is commonly contrasted with a traditional approach known as the Waterfall model – a step-by-step process during which a project is executed in a linear fashion, going through consecutive stages.

Within the Waterfall approach, strong emphasis is placed on planning and conceptualization, locking the requirements and resources within these initial phases (typically, in detailed documentation). Then, little to no changes are made along the way. Every next stage starts only when the previous one is completely finished.

Even though the Waterfall model works fine for some projects (e.g., landing pages design), in many cases, it’s not viable because of the many surprises and disruptions that always happen in real life regardless of what you’ve planned ahead. Since it’s so inflexible, with no adjustments or improvements made, at the end of the day, the result often turns out to be no good for the ever-changing market.

In construction and manufacturing, it’s better to go with Waterfall. Source: MonkeyUser.com

On the contrary, the iterative approach assumes that business requirements might change during the project so the work is organized in short iterations or sprints. It implies building an MVP early in the project, getting constant feedback (from the client, users, and/or other stakeholders), and implementing changes and improvements gradually until the desired result is achieved.

Iterative vs incremental development

The terms iterative and incremental are often used together or even interchangeably when describing Agile development. However, there’s a subtle difference between them, even though they usually do come together.

Incremental development is about breaking the project into blocks and then working on them one by one, delivering one increment at a time. It involves going through a number of iterations when new features are added gradually, improving the product until it’s finished. It sometimes happens with incremental development that feedback is only received at the final stage since there’s nothing to be reviewed at the end of each iteration.

Meanwhile, iterative development implies promptly rolling out the potentially shippable deliverable and then progressively refining it based on feedback and other inputs, iterating through versions.

Iterative vs incremental. Source: EmbracingAgility

Summing it up, it’s not uncommon for these two approaches to be used together. There’s an iteration component in incremental development because a number of stages have to be gone through to improve the product and achieve the result. Meanwhile, an iterative process includes increments as new functionality is added and new versions are released.

Okay, now that we’re done with the terminology, let’s look at the most common stages of iterative development.

Iterative design steps

Regardless of its scope and management approach, any product development process consists of more or less the same key stages. What’s crucial to realize is that regardless of the framework or methodology that you implement, there will always be the same initial, predevelopment (sometimes called discovery) stages when you

• discuss the project with the client,
• frame out main objectives,
• develop key requirements, and
• estimate the project scope, timeline, and resources.

Eugene Ovdiyuk, a project manager at AltexSoft, shares, “Typically, after the presale phase, comes the elaboration stage, when we go into the detail of how the project will be organized. We work on all the requirements, plan on the information architecture, devise the user flow, develop a test strategy, and so on.”

We’ve mentioned that the Agile iterative process doesn’t imply spending a lot of time on planning and documentation, but. Agile and flexible shouldn’t mean chaotic and unprepared, so planning ahead allows for organizing the process comfortably. Besides, documentation is the best way to have everyone on the same page and share important project-related information, especially in big teams. So you’ll still have to do some paperwork, and believe us, that will save you time, stress, and effort during the course of the project.

Rick and Morty can teach you the importance of documentation and proper planning

Once you have everything agreed upon, have buy-in from all stakeholders, and have your upper-level strategy created, you start with the actual iterations or sprints. In Scrum, sprints typically last 14 days, but Agile is about being flexible, right? So you can choose the optimal duration based on the project, team preferences, and your experience (the common standard is 1 to 4 weeks).

The main stages are planning, development, testing, and evaluation. Sometimes design and deployment are also singled out as separate steps. “Scrum is the small Waterfall,” Eugene chuckles, meaning that the phases are very similar to the traditional approach; however, they come in very short cycles, and a lot of work is done in parallel.

And that’s another super important aspect to understand: When we talk about the development stage, it doesn’t mean that designers or QA folks are lolling around. Instead, while engineers buckle down to that code, they work on other tasks. And that’s the main difference between iterative and traditional development that allows for workflow optimization.

A common iterative cycle. Source: coolblueweb, Medium

So let’s look at the common stages of the iterative process in software development.

Planning: estimating and distributing tasks for the next sprint

Commonly, the planning stage is very short: It’s basically a team meeting where the project manager (PM) or product owner (PO) discusses the backlog items with the team and sets the goals for the next sprint. It’s sometimes also called the backlog refinement session. During the meeting, team members create the sprint backlog, estimate user stories (e.g., with the help of story points), and distribute tasks for the upcoming sprint.

Note that despite a widely-held belief, tasks can be added to the sprint backlog during the sprint if, for example, the team works more efficiently than planned or it turns out that some essential feature is missing.

Building: developing the code

That’s the stage when the code is essentially created, as the team develops the projected functionality and the design that meets specifications. As we said, the goal is to create the prototype or an MVP as early as possible – that you can review, test, and get insights for further work.

Remember that you don’t develop the entire product at once but only work on limited functionality scope, proceeding in small batches from iteration to iteration.

Testing: checking for bugs and defects

Here, you do various quality control activities to assess the product's performance, usability, security, etc. As we said, it’s important to discover bugs and inefficiencies early before they snowball into intangible problems. So testing is an essential part of every iteration as it provides valuable feedback and allows you to discover defects and spot opportunities for improvement.

Watch our video explainer about testing in software development

As the project goes on, the approach to testing will probably change. During initial iterations, you’ll focus more on unit and integration testing, while later on, system testing and user acceptance testing must be conducted as well.

Read more about various QA types, methods, and approaches in our dedicated article.

Review: evaluating the sprint outcome

At the end of each iteration, you have to evaluate what’s done, how it aligns with the project objectives, and what has to be changed, added, or adjusted. Besides, that’s when you present the sprint results to the client and get their feedback. Analyzing that feedback, the PM or PO does backlog grooming and outlines further actions.

Also, if the increment is ready for delivery and it makes sense to release it, it’s deployed in production as the final step of the iteration.

If you practice Scrum, you might also do a retrospective meeting for performance evaluation at the end of each sprint.

Now what should you do next? Simple: Go back to step one and repeat everything all over again – until the customer is happy with the result.

Benefits of iterative development

So now we know what the iterative approach is about. Let’s see what its main benefits are.

Flexibility. Agile is all about being adaptive. The iterative approach allows for easy and timely adjustments according to the changing requirements or market conditions – even if they weren’t planned.

Increased efficiency and speed. Often the iterative approach implies working on several features simultaneously – which increases efficiency and speed of delivery. Besides, little time is spent on planning and documentation in contrast with the Waterfall model.

Enhanced collaboration and transparency. Since feedback is an integral part of the iterative process, collaboration between team members and other stakeholders (clients, management, end users, etc.) is much greater than in the traditional approaches. That allows everyone to be on the same page, have a common understanding of the project goals, and communicate ideas.

Continuous improvement. Continuous feedback flow delivers valuable insights, and as the team incorporates lessons learned from previous cycles, every new version produced in every new iteration is better than the previous one.

Reduced risk. Continuously addressing low-level issues during the project lifecycle reduces the risk of having big problems at the end (or overall project failure).

Cost-effectiveness. Since weaknesses are often noticed at the early stages of the project and changes are implemented, it can save money on fixing big issues later on.

Innovation potential. As the project goes on and feedback is received, there are often fresh ideas arriving on how to improve the product. The iterative process encourages teams to be creative and leaves space for the implementation of new ideas that can become a real breakthrough.

Iterative development challenges (and ideas on how to cope with them)

All that being said, iterative development isn’t a silver bullet and, just like anything else in the world, has its downsides.

High stakeholder involvement. It might not be an actual challenge but rather an inconvenience for the client. Not everyone is eager to be that heavily involved in the process.

Possible solution: Explain the importance of frequent communication and feedback. Agree on the cooperation schedule and scope in advance.

Scope creep and technical debt. Scope creep refers to a situation when the project develops in an unexpected way (due to new ideas that appear along the way), so it requires more time and resources than initially planned. It’s harmless if minor, but significant scope creep can take you away from initial objectives.

Technical debt happens when undone tasks pile up, often due to new tasks coming up or because higher-priority tasks take longer than expected.

Possible solution: Maintain a list of priorities and milestones. Carefully analyze the suggested scope changes, align them with your objectives, and don’t accept them if they’re getting the project too far off course. If changes are necessary, communicate them clearly to stakeholders and explain the need for additional resources.

Unrealistic expectations. The MVP is usually far from the expected project outcome, which can be disappointing for some customers.

Possible solution: Communication is the key. Explain to your clients what they can expect during the project.

Challenging time and cost estimation. Since you don’t have a plan set in stone at the beginning of the project, the timeline and costs are usually hard to estimate. In addition to stakeholders who push on the team to produce early estimates, we humans tend to be over-optimistic with our effort assessment, you know. That’s why estimating backlog items is an essential part of every iteration – which allows for adjusting the overall project estimate.

Possible solution: Write down all the tasks that must be performed and apply the Rough Order of Magnitude techniques to estimate the required resources, cost, and duration. It entails looking at other projects with similar tasks and scope (analogous estimates), statistically adjusting historic scopes to the current ones (parametric estimates), and weighing the optimistic/most likely/pessimistic scenarios with three-point estimating (PERT). Consider any outlying factors that can impact duration (e.g., whether you have all the resources or you’ll have to wait/pay for access to tools or data). Inform stakeholders that more accurate estimates will be available after the first 2-3 sprints, when you know the average team performance, its velocity.

Enjoy the company of Frodo and his gang as you learn all about estimations in software development

The lack of rigid planning and the trial-and-error nature of the iterative model sometimes lead to significant changes throughout the project. Make sure all stakeholders are aware of that and are willing to be flexible rather than stick to a strict schedule or scope – otherwise, the whole point of iterative development is dead on arrival.

If the project has a tight timeline and/or budget – consider choosing an alternative development approach. It’s also not a good fit when there’s absolutely no room for error in the project.

How to start with the iterative approach

If you’ve decided to integrate the iterative approach into your project management routine, we’ll sum up a few tips that might help you get the best out of it.

Embrace change. Remember that changes are inevitable, and that’s basically why iterative design is implemented. Develop a mindset that will allow you to stay flexible and allow changes (new ideas, innovation, etc.) to the project.

Communicate often. We’ve already mentioned the importance of extensive and transparent communication with stakeholders. It’s crucial as it allows you to solve multiple obstacles on the way and keep everyone informed on the progress. Make sure everyone understands the project objectives and ways to achieve them.

Encourage feedback. Communication is important not only with customers but also with team members and other stakeholders. Build a healthy feedback culture where new ideas are welcome and constructive criticism is accepted.

Don’t neglect documentation. While it shouldn’t be overly extensive, it’s still essential to any software development project.

Monitor KPIs. Again, iterative shouldn’t mean unorganized, so remember to monitor performance and make sure the project is on track.

Iterative development is a great way to promote continuous improvement and encourage innovation. But remember that there’s no perfect methodology that suits all teams and all projects. If you feel something isn’t right, you can always try a different approach and find what works best for you.