Defect Management: Training and Verifying
What do you need for Training Materials?
If you’ve been following along with these blogs from the beginning, you have a good high–level understanding of what Defect Management is (from Blog #1), and how to set up your defect repository (from Blog #2). Now, we have everything set up and ready to go, except no one knows how to use this cool new tool you’ve set up to capture the defects. So: Training!
As usual there are many flavors of audiences, documentation, and training, that all need to be considered.
1. Documentation Types
The types of documentation you will need to create can be separated into two types: Informational and Educational. The first type, Informational, would usually be a Word-type document (sorry for the Microsoft-centric verbiage) where you can spend your time detailing everything out step–by–step, along with LOTS of pictures and diagrams. These documents will be archived, so anyone can retrieve them to learn how to work with the application or the processes being outlined. The idea is that they are COMPLETE. If a user needs to know something, all the information is detailed in one of these documents. Once they finish reading and digesting it, they should know everything to accomplish their tasks, which might include (but are certainly not limited to):
- Admin work – adding users, unlocking them, changing their passwords, etc.
- How to enter a defect into the system
- How to write up a defect well
- How to execute a test case
- What the overall Defect Management process is, why we need it
- What Triage Meetings are, and how they are run, who should be there, etc.
- How to import Requirements or Test Cases or Defects into the repository tool
- How to export data for metrics, etc.
This list goes on and on, depending on the features of your repository tool, the Defect Management process you have in place, what people are asking for, and, of course, the time available to write it all up.
If the first type – Informational – could be likened to a textbook for a college course, the second type – Educational – would be likened to the classroom lecture itself. It is a higher level, visual run-through of the topic, giving the most critical and needed information, but not necessarily diving deep into all the areas. It would be supported by the Informational documentation if people need that deep dive for specifics. At the end of the “class,” the user should be able to understand and apply what they have learned in order to get their job done, but not necessarily all the bits and pieces, details, and hidden features of the product.
These come in two flavors: videos and PowerPoint–type of presentations. The presentation decks are what you would put together if you planned, for instance, on standing in front of a room (virtual or in person) of testers eager to learn how to start logging defects. You would generally create a deck of slides at a fairly high level, and then speak to your audience about each one, going into details verbally. The decks themselves would not be a very useful tool on their own if a user looked at it alone, unless the notes for each slide were expansive. However, a recording of the classroom session would be worth archiving. Make a point of recording all your presentations. You can keep or discard them later, but as a first cut others who were not able to make the presentation will have something to work with.
Videos have a place all their own and are very important. The problem with a recording of a presentation is that they are frequently interrupted with questions (not bad, but an interruption none the less), have audio issues, or are live and not controlled. A video recording of your screen talking over everything covered in the class, including addressing any questions that were raised, is a much smoother presentation of the same information. It is more controlled and provides a step-by-step presentation. It can be laid out in advance, practiced, started and stopped, and even discarded and restarted if needed. In addition, you can show yourself working in the actual application itself rather than slides of pictures. Both have their place and their strengths, so consider them both as possible additions to your library of documentation.
By audiences here, I don’t mean the people sitting in the conference room watching you perform. Rather, I mean the TARGET audience(s) you are looking to educate on the ins and outs of Defect Management or the use of the repository tool (or other specifics):
- Testers and Developers
- Project Managers, Business Analysts, etc.
Each of these groups need to know about Defect Management, specifically about the defect repository tool, but at different levels– from everything at the top (Admins) to the bare bones at the bottom (Management).
Assuming you are the lead Admin (the person putting all of this together for your company) you will need to have a set of documentation for any other Admins working with you or who may come along in the future. This will be the most detailed documentation and doesn’t have to be a “pretty” presentation, just a document or set of documents detailing all the ins and outs of using the tool as an Admin.
Think of it as a technical manual rather than a classroom textbook: how do you add/remove a user, change their rights, unlock them, change their password? Once you have done that, what do you send them to let them know? How do you fix problems in the system if a user has done something wrong, like moving or deleting folders? How do you set up the different sections of the tool to have it do what you want, or look like what you want? How do you export data, import data? What about all the rest, what the testers do, or the PMs or BAs? Well, there is no need to create separate documentation for the Admins on all of that, since you are doing it for the other audiences anyway.
Testers and Developers
Testers will probably use the repository tool more than others, so the focus for them is specifically on how to use the tool: how to locate and execute test cases (if you are using your tool for that), how to log a defect from the execution of a test case, how to log a defect on the fly, how to attach screen shots and other documents, how to locate test cases that are assigned to them to retest, etc.
Developers, on the other hand, need to know their way around test cases and defects, less on entering them and more on how to open, read and comment on defects, look at the linked test cases, etc. They generally do not create test cases or create defects, but rather review and comment on them.
PMs, BAs, etc.
This group of people is more involved in entering Requirements and Test Cases and less on using them – or in executing test cases or even creating defects. Therefore, the focus for this group is at a higher level and should focus specifically on what they will be doing, most likely entering and modifying requirements and test cases. They will need to know how to mass-import them, how to enter them one at a time, how to manipulate them (move them from folder to folder, etc.), and how to extract them if they need to run metrics on them.
The final group has almost no need to understand the repository tool at all. They for the most part will not be entering requirements, test cases, or defects, or executing test cases out of the tool. However, the focus of what they will need to be educated on is the FULL Defect Management process, the Triage Meeting process, expectations coming out of it, what metrics they will be able to get out of the system.
NOTE: Individuals in ANY of these different groups of people may want to know about other areas, but there is no need to create different sets of documentation for each group. They can share! The documentation should be focused on who the key audience is and should cover everything THEY need to know.
- Admins – how to MAINTAIN users, and how to RUN the processes and tools
- Testers and Developers – how to USE the tools
- Project Managers, Business Analysts, etc. – how to SET UP the tools
- Management – how the processes WORK, and how to MANAGE them
Note that all the work that has now already been done is MOST of the work. Training on all these documents and areas is now up to either publication and promotion of the documents and the storage location of the documents, slide decks, and videos that have been created. AND – the big thing is scheduling in-person or virtual classes to go over the appropriate information with the appropriate groups prior to (preferably) the start of each project or phase of project.
As always, if you are doing the training in front of a live audience, in person or virtually, it is important to remember two things. First, KNOW YOUR STUFF! If you have #1 down, then #2 is easy: be relaxed and comfortable. You should be able to easily answer most (not all, but most) questions that come up, and if you can’t, let them know you will find out the answer and get back to them – and then actually follow up: get back to the entire class with the questions and answers.
All the documentation and training has been completed, and your testers are starting to test and enter defects. But what happens if they didn’t quite get all the classroom instructions correct, or if they are just aren’t quite filling out the defects the way they were instructed? These could be incorrect entries, such as stating UAT testing when the project is only in SIT, or stating the defect was found in Production when the application has not yet been released. It can ALSO be a matter of clarity of the summary/title field. This field is the one seen by everyone and how the defect is referred to, so it must be clear and concise. If the tester has not entered it in a clear and concise manner, it should be corrected, and the tester needs to be educated and reminded on how best to fill it out.
All of this is where the defect manager comes in. Before these new defects get to the Triage Meetings to be reviewed and assigned, the defect manager should be reviewing them to ensure the field values are as expected, so time is not wasted with these basics during the meeting. Depending on the volume of defects as a whole or the velocity they are being added to the system, the defect manager can address this in several different ways. First, they can scan each defect as it is entered into the system and correct obvious incorrect entries. Or, they can address any questionable field values with the tester who entered it with a quick email back to them.
A second key method is to regularly run queries on the defect repository, looking at values in fields and comparing across fields (defect found DURING SIT testing, but found IN the Production environment, probably an issue). Several quick queries run regularly can often catch mistakes that may have slipped through the cracks.
So, there it is: training and entry verification in a nutshell.
Understand what data you and all other entities in the organization need to gather, what that data needs to look like, who needs to see that data (or not see it), who needs editing capabilities, and what is or is not required. Then you must make sure that the entry and review of the data is clean and intuitive to use.
The next blog in the series will also combine two steps: how to prioritize and assign defects and verifying timely refactoring.