Defect Management: Defect Triage Meetings
Triage Meetings: Purpose and Process
If you’ve been following along with these blogs from the beginning, you have a good high level understanding of what Defect Management is (from Blog #1), how to set up your defect repository (from Blog #2), and know what is needed in order to train your eager team (from Blog #3). By now, everyone is testing and creating defects, but they are just being dumped into the defect repository tool and not being worked on in a timely manner.
What’s the solution here? Triage Meetings!
But just what is a triage meeting, how do you run it, what gets accomplished during it, and what should you expect from it?
The Triage: What is a Triage Meeting?
A triage meeting, like triage of patients in a hospital, looks at defects, analyzes them, and routes them to the appropriate department to be taken care of. Some questions you might have about it:
- What’s Addressed, What’s the Goal?
- Who Organizes, Who Attends?
- How Often, How Long?
- What are the Takeaways?
What’s Addressed, What’s the Goal?
There are no set rules for a triage other than what the Defect Manager, or the individual in that position to run the triage, sets – and this can change daily. Below are some of the more common discussion agenda:
- Review any NEW defects that have been logged since the last triage meeting, from highest to lowest severity, depending on duration of the meeting. If you find that defects are coming in too quickly to handle in a 60-minute triage, extend them to 90, or have 2 triage meetings per day (same observation goes for ANY of the agendas you choose to follow)
- Review all open Critical and High severity defects
- Review defects from oldest to newest, working to address those that have been hanging around for a while (they may or may not still be valid defects, that is what the discussion is about), to get them fixed or deferred or closed if possible
- Identify specific applications or application functionalities that are having more issues that others, and focus on getting through them
- Defects identified specifically from the testing of the last code build delivered to testers
- Production issues found after a given deployment
- Have an open forum and ask everyone (a day or two before a focused triage) if there are any specific defects they are having issues with, if they are having trouble moving forward, lost in a dark hole, needing help from others in the triage team, etc.
- Defects that have been identified as incorrectly logged or the fields incorrectly filled. These lists may be identified by regularly querying the defects by field, sorting, or filtering. Look for anomalies – testing environments that don’t agree with their cycles or releases or owners (department heads in charge of functional areas) that don’t agree with the application area the defect is assigned to.
You can see that the defect manager can have a pretty free range of areas to cover and even switch it up regularly if one agenda is no longer a reasonable focus. For instance, if defects are no longer pouring in, the first bullet point only takes 5 minutes, and the second bullet point is repetitive, the defect manager can throw something new in the agenda and follow the third bullet point, oldest to newest.
The goal of ALL of these agendas is to find errors on the defects and make corrections, to make sure they are assigned out to the appropriate resource (sometimes a developer may forget to assign to the tester when they are done, or vice versa), and to make sure that comments from the resource working on the defect at the time are up to date. The comment section guides the team on whether a defect is being worked or not and if it is moving forward to resolution or not. If comments are not being filled out (last one is several days old), this is a red flag that it may have dropped through the cracks and is not currently being addressed.
This is accomplished through discussions at all triage meetings with the team coming – usually quickly – to a consensus. Longer discussions may ensue, and this is a GOOD thing if a solution is in sight and the discussion does not begin to take over the triage. If the defect manager sees that happening, it is up to them to suggest that this particular discussion be taken offline and to get a resolution estimate entered into the comments. (“Team A and B will discuss this afternoon and update defect comments for tomorrow’s triage.”)
Who Organizes, Who Attends?
Usually the person in the role of the Defect Manager is the one who organizes the meetings, if not runs them. They will determine the scope and frequency of the meetings, lay down the meeting structure (the defects that will be discussed- most critical, newest, oldest, by department, by function, etc.), decide who should attend, decide if discussion needs to continue on resolving a defect, or if discussion needs to be taken offline. The defect manager will be charged with setting up, cancelling, or moving the triage meetings and sending out information after the triage (metrics on what was discussed, decisions made, next steps) to the attendees and/or Senior Management.
Okay, now just who SHOULD attend? Not a simple answer, as it will depend on availability, size of the teams, and especially the focus of the meetings. Groups you might consider are:
- Senior Management, if they are interested. They may have an interest in the beginning, then once they know Defect Triages are being run well, they may bow out.
- Department Heads (Directors or heads of the functional areas of the organization) – more likely to be involved longer term as the defects are addressing and affecting their specific departments
- Development Leads – probably the most important, as the individuals in this role SHOULD know everything that is going in about the development process- understanding where the defects are occurring in the application, knowing who is most familiar with the code, who it should be assigned to, and what progress the developers have made on each defect.
- Test Leads – also very important, as these individuals should know when, where, and how these defects occurred, as well as be able to answer general questions and assign back to testers to retest.
- Developers and Testers – except in specific cases (exploring a specific set of defects belonging to one or more developers or testers). The developers and testers should continue developing and testing rather than spending time in meetings. HOWEVER, when addressing a specific set of defects, it can be valuable to have the tester who discovered the defect and the developer working on it to be in the same meeting together. They will be able to talk through issues if needed and more quickly come to a resolution with the help of others in the meeting.
If the focus is to quickly review and assign new defects out to the appropriate team/developer, the attendees can be team leads that know the systems and their developers, have read and understood each of the defects in their area before the triage, and can assign them to the appropriate person. If the focus is to review old defects that are not being worked on as quickly as expected, the attendees should include the developers or testers that the defects belong to or the team leads that know exactly what is happening (or NOT happening) with the defects. They can either defend why progress is not being made on them or state definitively that they will be addressed right away.
How Often, How Long?
As mentioned in the last point, this also depends on the size of the project and the number and velocity of the defects. How long is probably easier to answer than how often. Duration of a triage meeting should probably be no longer than an hour or 90 minutes if there are lots of defects that need to be addressed. Even if the number of open defects is high, the Critical severity and usually the High severity defects can be addressed in a 60-minute triage. If you find that the agenda you have chosen is taking longer each day, than a 90-minute max, then you should probably think about adjusting the agenda to not be so all-encompassing. If it is indeed just a matter of a LOT of defects to review, then set up two separate meetings during the day until you can get them under control. For instance, you might have a morning triage to review everything from the prior evening and morning, then another one mid-afternoon to review everything that has been logged since the morning meeting. OR, if you are reviewing all defects oldest to newest, you might just take a reasonable chunk each meeting until you feel you have addressed the oldest and the team understands their duties to on their own take care of these.
Keep in mind that these triage meetings NEED to be seen by both the rank and file and the senior management as beneficial and working toward a goal rather than being a burden to everyone involved. They should be walking away from those triage meetings not feeling that it was a waste of time, but rather that the whole team is getting a better and better handle on the wayward defects.
What are the Takeaways?
Wonderful! You’ve just finished up with the last triage meeting. Now what? What should the resources in each of the main groups be doing for the rest of the day (to address the defects, not considering all the rest of their daily tasks)?
- Senior Management – If they are still involved, senior management should be interested in progress metrics, so they can track at the highest level the progress being made against the existing defects, the rate of defects coming in from development, and the rate of testing being performed. Some example charts are below:
How many defects are coming in daily? How many defects are being closed daily? How many defects are open currently daily?
How OLD, on average and by Severity, are the open defects?
Status of Defects Daily
Simple Trend of Open Critical and High Defects by Day
Defects by Status by Severity
SO many graphs are possible, but this might give you an idea of what can be presented to the Senior Management to start the ideas coming.
- Department Heads – The department heads should be at each of the triage meetings. Between the meetings, they need to make sure they are clearing the path for the rest of the team to be able to resolve issues, get defects fixed, and get them ready for testing. Are environments down? They need to drive to spin them back up. Is there dirty data or other data issues? They need to make sure data is fixed and flows correctly. Are there huge amounts of defects coming from the developers, or from a specific development team? They need to address WHY (poor development processes, no unit testing, etc.). This level of resource clears the path for others to get the job done.
- Development Leads – The development leads track the progress of each defect as it goes through the defect workflow from being assigned to a developer to be fixed, to the reassignment of the defect, after the fix, and to the tester to retest. They need to know the status of each of the defects being worked on by their developers to avoid the need for a bevy of developers to join the triage meetings.
- Test Leads – Test Leads take the same position as the development leads. They need to know the status of all the defects that have been fixed, as well as defects that still need to be or are in the middle of being tested. They should be able to answer what is happening to any defect in a “Ready to Test” or “Retest Failed” status (or whatever the equivalent is for your system) – WHY is it not tested and closed yet? Has the tester simply not gotten to it yet, and if so, when will they be able to address it? If the retest of the fixed defect failed, why and how did it fail? Has the tester reached out to the developer to explain the issuesmand to go over the retest and explain what happened?
- Developers and Testers – Developers and Testers should usually only join triage meetings if the team in a previous triage meeting had determined that one or more defects need direct attention of a developer and/or a tester in order to help resolve it quickly.
- Defect Manager – The Defect Manager is a busy one between triage meetings. They need to be, among other things:
- Reviewing newly detected and logged defects for inconsistencies, making corrections where obvious, taking notes to bring up during the next triage
- Scouring the data for anomalies to bring up during future triages
- Scouring the data for trends to report up to Senior Management
- Creating daily metrics on defects to report up to Senior Management and the rest of the teams
- Admin work on updating the defect management tool as needed/requested
- Admin work on adding/removing/unlocking users to the system
Tracking Timely Refactoring
SLAs, Proposed Resolution Date, Metrics
So now that the defects are flowing in, how do you ensure that they are being addressed in a timely fashion?
Three major ways:
1) Creating and socializing an expected due date for defect refactoring
2) Capturing on each defect the proposed due date for completion (may or may not be different that the expected due date from #1)
3) Creating metrics to identify the aging of defects throughout the refactoring process, and alerting Senior Management of issues
- Create SLAs (Service Level Agreements) for defect refactoring, depending on defect severity, so everyone knows what is expected. Reasonable refactoring time might be one day for Critical severity, two days for High severity, five days for Medium severity, and 10 days for Low severity. Note that these times may work well for some internal development teams or for smaller and less complex applications, but may have to be increased if the applications are more complex, if your development teams are offsite 3rd-party companies with their own sets of processes, or if the 3rd-party company is modifying code for which you are only one of many clients using that shared code (meaning that the company can’t just change code willy-nilly that would affect the base code for many other clients). Their process would include their own internal research, coding, unit testing, and rollout schedule that just may not fit within your SLAs requested.
- One of the fields on each defect might be something like “Proposed Resolution Date”, where during the first triage of the defect, the development team estimates when they can deliver the fix back to the defect creator for retest.
- Metrics can be pulled for Senior Management by comparing Open/Close time of defects against either/both of the SLA by severity or the Proposed Resolution Date and can reveal any major differences between the anticipated resolution time and the actual resolution time. Management can determine, given these results, if action needs to be taken to address the issue and work to improve turn-around time for the refactoring of the defects.
So, there it is – Triage Meetings and tracking timely refactoring in a nutshell. First: organize the triage, know what you want to discuss, make sure everyone knows their roles and their take-aways, ensure defects are continually being addressed and moved through the system, and that they don’t age longer than they should. Second, make sure to verify that they are being addressed in a timely fashion and identify trends and issues, so they can be addressed, if needed.
The next blog will also combine two steps and look at metrics creation to track and report progress and identify trends and issues.