Many of us who have worked with agile development methodology understand its strengths and disadvantages in different scenarios with clients and development teams. Agile is a paradigm that prioritizes adaptability to change and cyclical learning, perhaps for this reason it is one of the most used work methodologies of the past 20 years in the technology industry and the main one utilized in ArkusNexus.
To understand its disadvantages and how we have overcome them it's important to know something about its history first: Agile, unlike its predecessor (Waterfall), comes mainly from the concept of reality, where the requirements of a software project change constantly and it’s not possible to stabilize them; at least, that was the consensus of the 17 engineers who started it in Utah, US. back in 2001.
Influenced by the Lean Production industrial model formulated by Toyota in 1977, they introduced it as a methodology where change must be accepted as a constant for continuous improvement, and where any internal process or function that doesn’t bring benefit to end users is stipulated as a loss, because of this principle, we prioritize tangible early results in cycles or what we call Sprints and do internal retrospectives with each cycle.
The project management, on the other hand, lies in progressively discovering the requirements while we learn more about the market; work is divided into independent and measurable units that engineers can set and identify as milestones, seeking adaptive planning, evolutionary development, continuous delivery, and flexibility to the constant change of requirements.
More importantly, and perhaps widely known by developers, when the agile philosophy was created, 12 principles were defined, giving birth to the agile manifesto, which to date is still being promoted and applied; however, like many "ancient" writings, some parts of it have been misinterpreted, and sometimes the whole manifesto itself, resulting in unpleasant situations such as team micromanagement, normalized burnouts due to unforeseen events, lack of planning or technical documentation that helps to synchronize team efforts, or even to the total collapse of large investment projects.
However, we do not affirm that the agile principles are perfect from their conception; we have learned through years of experience that these, more than absolute rules, are guides for 90% of the situations that we seek to have where we can have better control; but there will always be a 10% where some will have to be ignored to reach our desired result; this in principle is what differentiates the internal agile process of ArkusNexus many times. From the 12 principles of agile, the original 4 are usually the most popular and taught to developers when they are introduced to agile, these are:
1. Individuals and interactions over processes and tools.
2. Working software over comprehensive documentation.
3. Customer collaboration over contract negotiation.
4. Responding to change over following a plan.
Perhaps one of the main causes for the bad perception of the agile model, according to its creators, is the interpretation and simplification of these 4 most popular principles, where the word “over” is understood as a “not necessarily”, for example, starting with the first principle: “Individuals and interactions not necessarily processes and tools”, makes us understand that establishing processes and mechanisms based on tools should not be the priority of the team; when in reality they simply wanted to say that the result that should matter to us the most is the individuals and how they interact with our product and that the processes and tools that we define along the way allow us to precisely reach the result; but not that a documented process should not be established as many teams come to interpret.
Every principle may fall into opposite frames in the day-to-day reality of the teams, continuing with the first principle, a common extreme in the absence of established processes and tools happens when we see very long meetings without setting a time box limit, development tickets without clear specifications or acceptance criteria that denote the expected result, adding the lack of mechanisms for prior ticket verification such as DOR (Definition of Ready), often results in last-minute adjustments that translate into overtime for developers, or, considerable delays in team’s deliveries.
However, on the contrary, sticking strictly to rules of syntax, nomenclature, and ticket allocation is not something we should focus on too much since as the principle indicates, a critical point (that 10% of the time) can reach where these processes will no longer bring a benefit to the end-user and their interactions with our products when we focus more on the formality of the process than on the agility of the team's response.
In short, the main objective of any tool or process within the agile framework is to help us streamline the organization of work and give us visibility of progress to know if we are on the right track. Results> Roles> Tickets.
Believing that agile rejects any type of documentation is another commonly recognized anti-pattern, since spending time creating technical documents is often perceived as a waste of time, more so when you want to be fast or "agile" in development due to the sense of urgency. Every creation process in its best practice carries a source of truth that serves to plan and synchronize team efforts, especially when changes are constant, and we must remember the greatest principle of agile: changes are a constant; In changing scenarios it’s critical to have a source of truth that helps the team react and adapt, otherwise the work integration effort will be affected, seeing cases such as outdated or invalid test cases, an interface that doesn’t match the proposed user flows, or even delivery conflicts with the client due to differences in expectations.
This is why in the discovery and requirements proposal processes, (usually done by the Product Owner) an initial and living source of truth is established that adapts to change during development, which in turn serves as a point of reference for the work of other roles. Some practices that we can follow to bring agile documentation are:
One of the greatest advantages of working with a client or stakeholders that understand agile collaboration is the ability to work with them as one more member of the team, seeking transparency of the processes and results together. However, there may be situations where this advantage can be turned into a disadvantage due to the client's lack of trust towards the team, which has an impact on taking away their autonomy to operate and even negotiate the development of the project. Symptoms of this deficiency are observed when the client or team leaders constantly skip processes (or lack of), urge to make changes without notice during the print, or measure the teamwork based on time spent on individual tickets; When these symptoms are constantly present, it turns into what we know as micromanagement, the opposite of what the agile methodology seeks, which are autonomous and independent teams with an adaptive capacity to challenges.
To counteract these deficiencies, it is necessary to always have and demonstrate absolute control of the product to make decisions and work on the vision in collaboration with the client and the team, justifying decisions with prepared arguments, presenting work plans made as a team and, if the clients are new to agile, gradually educate them about its methodologies and the benefits they provide.
It is also critical to document sprint goals, identified risks, incidents that occurred, and retrospectives that help us make evident our agile management with our clients. If the initial problem is trust, and we know trust is earned, the best way to do it is to operate in a 100% transparent manner.
Every project, no matter how small, always entails a plan, to what extent and how it is described is what differentiates each project and how it is managed. Although Agile promotes the idea that we must accept change and consider it in our internal processes as we learn more about our market and how it changes, it does not specify that due to this perspective we should not carry out and follow up on a plan based on achievable goals.
The project vision, usually established in a roadmap, is necessary for the previous principles mentioned and more; a roadmap outlines which interactions we want to create for individuals, which components we must define and work firstly on, and finally gives us the best tool for visibility of progress to collaborate with our client and with the team.
In the absence of a clear, updated and visible roadmap for everyone in the team, there is no sense of progress or justification for the changes we need to make; and this usually creates chaos in the project that later can make us fail our business objectives or alternatively, asking the team to perform constant overtime crunches due to lacking an agile plan with established milestones.
Some of the practices that we use to minimize this type of risk are to keep track of changes in the following artifacts or documents of the agile process:
What we are looking for is to create an internal mechanism or process so that whenever there’s an update to a source of truth (roadmap, PRD, or user flows), all related code, tests cases, or technical documentation is updated accordingly to match and keep our work consistent across efforts; that is in other words, to find a way to manage the constant changes of reality without getting lost in the agile development.
Keeping a constant organization of our sources of truth while adjusting the team goals is not an easy task for any agile project manager; however, if we create an internal process to deal with changes, we are more likely to promote an organized and stable work environment that outputs the best version of our product while maintaining good morale and pride in the craftsmanship.
Reality can challenge even the best organizational paradigms, and agile is no exception as we have seen it from the perspective of its manifesto principles and how these can become deficient on a day-to-day basis.
However, the main purpose of any paradigm is to help us coordinate a group of people with a common goal: to build something that impacts reality; And each time a more competitive one, where it is not necessarily whoever comes out first triumphs, but who manages to better adapt to constant change; and for that reason agile to this day, after 20 years remains one of the most popular philosophies in software development; It is also possible to combine it with other paradigms such as Scrum, Kanban, CI / CD or even partially with its predecessor Waterfall.
No paradigm is perfect by its very principle of being a theory, nor will it last forever; someday not too far away we may replace Agile with something better, perhaps not just one, but a combination of many as we have seen in recent years; Paradigms for creating software are popularized and sometimes forgotten, however, we as software creators, do know with certainty a lesson for those who have experienced Agile: change is a constant when we build something and the better you manage it as a team, the more competitive advantage you will have on the market.
