© 2023 Quality Digest. Copyright on content held by Quality Digest or by individual authors. Contact Quality Digest for reprint information.
“Quality Digest" is a trademark owned by Quality Circle Institute, Inc.
Brian Copeland Published: 10/23/2006
Every day, I hear from frustrated quality assurance (QA) managers who’ve been informed by project management that their six-week testing schedule has been reduced to two weeks or less. It usually involves some sob story about how the development team is a month late because of the customer’s last-minute changes (Isn’t it always someone else’s fault?). The testing team is expected to suck up the lost time and get their testing done in a fraction of the originally scheduled time.I typically ask them, “What have you done to prevent this from happening?” The manager usually explodes from sheer pressure and dumps a laundry list of reasons for the project’s poor condition—management committed to a date before the project even kicked off. There are problems with the requirements—either they’re nonexistent, vague, incomplete, or not well documented. There are problems with the involvement of the quality resources, such as being left out of key meetings. There are problems with the development team producing unit-tested code. There are problems with ever-changing requirements and a lack of concern for managing the rate or scope of the changes. There are problems getting enough test resources to adequately test the functionality. Finally, there are problems with management support for testing, test environments or managing defects. Now that the QA manager has had the opportunity to dump all of the pent-up frustration, I can once again ask, “What have you done to prevent this?” The now pressure-relieved manager typically responds that they provided a schedule up front and they made sure to remind the project manager (PM) at every opportunity that they weren’t going to make the date.
This is a typical project scenario for many organizations. Finding an organization where quality is an inherent part of the methodology and where everyone takes ownership for building quality into the process and products is rare. The rest of us have to live in that chaotic world where we’re doing our best to “test” quality into the end product. If this is the environment that most of us work in, we have to find better ways to manage the chaos of that environment to give ourselves the best possible chance for success.
The first way to get yourself out of the hole you’re in is to stop digging. Let’s explore some strategies for better managing the quality process within your organization and give you some tools that you can use to start to lift yourself out of that hole, so:
These four strategies can change your behaviors and the outcomes in your organization. Imagine a world where you have the time you need to fully test and verify that the product meets its intended need. This is an attainable goal.
| Four strategies QA managers should use to ensure successful projects | |
| Begin defensive project management | You’re the victim of others only if you allow yourself to become one. Take ownership of managing your tasks and any tasks that may affect your delivery. |
| Establish the fundamental value of quality | Make sure the organization fully understands the magnitude of testing efforts and that it values the benefit that testing brings to the process. Commit to using repeatable estimating models that are updated after each project. Produce metrics of where you are in the project and that measure the cost of poor quality. |
| Solicit assistance from leadership | Your manager should be your best ally. Routinely seek his or her help in the delivery of your work, and remember that you, too, have an obligation to provide your team members with what they need to be successful. Spend time with your manager soliciting help in removing barriers to the success of quality. |
|
| Understanding what can be manipulated within a project to deliver a successful project is key to success. Create a project degrees of freedom graph. Negotiating the level of freedom in each project dimension will remove pressure from the team and deliver higher customer and management satisfaction. Create a graph for each project you and your team are working on. Work with your project manager and customer to agree on the graph. Implement the graph as a normal part of your project chartering process. |
Begin defensive project management
If I asked each of you to tell me who was responsible for quality, I’m pretty confident that most of you would say, “Everyone is responsible for quality,” and you would be absolutely correct. On the other hand, if I asked each of you who was responsible for project management, most would likely say the project manager was responsible. Nothing could be further from the truth. Just as everyone is responsible for quality, everyone is responsible for managing the project. The person identified on the project contact list as the project manager can’t possibly do everything it takes to manage a project successfully. As the QA manager, you have the responsibility to manage the testing aspects of the project, not the PM.
The QA manager has a responsibility to manage the testing aspects of the project and the responsibility to defensively manage all of the aspects of the project that could potentially affect testing. “What? As a QA manager, I have to manage the entire project?” If it affects or could potentially affect testing, my answer to you is “absolutely.” If you’re tired of being the victim of the bad behaviors of other teams or functions, then you have to help manage their delivery. I have witnessed too many projects where the development team is allowed to miss dates, functionality, unit testing and code-freeze deadlines, only to have QA suck it up. If you want to be successful in getting the time you need to test adequately, then ensure that all teams are held to their committed dates and deliveries.
You have likely heard the term “defensive programming,” which describes a programming design technique where the developer designs the program to be very robust. It observes the adage: “Trust nothing you receive; validate everything you send.” This strategy has the programmers coding the application suspicious of any input and performing checks up front to validate it, making for better, more stable programs.
Defensive programming places the power in the hands of the developers, as they shield themselves from the bad habits of others. The same adage holds true for project management. As the QA manager, you can better shield yourself from the bad habits of others by taking a proactive role in managing those processes, teams and artifacts that drive the success of the quality team. While the PM has accountability for the day-to-day management of the project, you can have an enormous effect by watching the watcher, and by helping the PM manage the activities of those teams.
Project managers seldom manage interim project milestones with the same passion and urgency with which they manage the end date. As the QA manager, you have the responsibility to ensure that they recognize that the project isn’t red when the QA team is late; it’s red when any interim milestone that could affect the end date is late. You must ensure that the PM is managing the business analysts (BA), developers and the QA team with the same rigor. When was the last time you heard a PM asking the BA team if they were working nights and weekends to make their requirements baseline date? You’ve probably never heard of such a thing, because they always assume they can make up the time later, and you know who always pays when they don’t make up the time. When it comes down to the wire, the QA team will be the one giving up holidays and weekends and working extended hours, while the developers and BA teams get a much-needed break. If you’re not making sure your PM is managing those interim dates, don’t be upset when you’re forced into a corner.
Remember, project managers often report directly to the application manager who made the commitment to the customer in the first place. They often see themselves as having to keep their management happy and do what’s best for the customer. They may not understand that they place the entire project in jeopardy when they cut quality time. They mistakenly believe they’re serving the customer by making the date at all cost. If you aren’t going to every project status meeting, make sure you’re invited and under all circumstances attend the meeting. Do your homework first. Look at the tasks to be completed and determine the ability of each team to deliver on schedule. If it looks as though one of the teams isn’t going to make its delivery, ask how it’s going to get back on schedule. If the scheduled delivery date is going to be missed, ask why and identify what’s going to be affected. Ensure that all downstream effects are addressed immediately and that the project status correctly displays those consequences. If requirements are late, make sure the project is identified as red, unless an acceptable mitigation plan is in place. No one is going to look out for your team but you. Don’t rely on the PM to do it for you.
Establish the fundamental value of quality
When testing cycles are significantly reduced on virtually every project, there are one of two issues with the team or organization. The first issue may be that the organization doesn’t fundamentally believe that the testing schedule is accurate or true. For example, a team originally scheduled six weeks for testing, but was forced to reduce it to two weeks and still deliver the agreed-upon testing scope, or so it would appear. In reality, the team likely didn’t complete all of the testing and only did a cursory job of testing even the most critical features, resulting in latent defects in production, higher support costs and less confidence in the testing department. This becomes a vicious circle, as the testing team begins to pad schedules, knowing that they’re going to be asked to reduce it, further adding to leadership’s lack of confidence in the schedule.
The only way to combat this perception is with absolutely accurate numbers, any changes to which have to be adequately documented and justified. Use a standardized estimating model to establish your quality project plan. Whether the model is based on function points or some other method, make sure the model used is consistent. Have a boilerplate schedule that identifies the entire task list. Create a model of test case design effort, test execution effort, defect management effort, etc. This model will help you justify the effort you’re identifying. Always remember to refine the model based on actual effort expended after each project.
If you reduce the schedule by working overtime and weekends, make sure you document that the schedule wasn’t actually reduced—the calendar delivery was reduced by increasing the hours and days worked. Ensure that everyone understands that this isn’t an acceptable long-term trend for the project team. Otherwise you’ll have a higher-than-normal turnover. Everyone can get behind an occasional need for overtime, but you’ll begin to loose resources if every delivery is an emergency.
Lastly, manage the quality effort throughout the life cycle and make sure you’re constantly reporting on status to leadership. I highly recommend using a test management solution such as the Mercury Quality Center (TestDirector) solution to manage this activity. These toolsets provide you with very robust reporting capability. It’s hard to argue with numbers generated by these toolsets and they help the entire team to make informed decisions about testing efforts. When schedules are reduced and testing is sacrificed, make sure to produce post-production defect metrics that capture the cost of poor quality. If you want the organization to change, there has to be a problem to solve. Production issues always command attention.
The second possible cause of reduced testing cycles is that the organization doesn’t fundamentally believe in the value of the testing team. There can be many reasons for this, but commonly there’s a lack of confidence in the caliber of testing demonstrated through post-production defects. This lack of confidence can be due to the testing team taking an ad hoc approach to testing, or the testing being superficial and not discovering enough critical defects for the amount of effort expended. The relative value of the testing to the effort expended is in question. These teams usually rely heavily on the customer to identify defects during user acceptance testing (UAT). This issue involves the organization’s reputation.
To repair or improve the reputation of the testing organization is much harder to accomplish than to produce accurate schedules and execution metrics. First and foremost, testing requires a skilled professional. It’s not the department where all the developers who couldn’t cut it as developers should be placed. Nor is it a career step toward becoming a developer, project manager or business analyst. Testing is a discipline that requires a strong education in quality, as well as business and technical prowess. If you have a QA team made up of the dregs of your organization, it’s probably time to retool your QA function. If you want superior testing, you must seek out true testing professionals or invest in the best and brightest your organization has to offer and develop them from within.
Establish a risk-based approach to testing. This means that all functionality is prioritized and risk-rated, and that all testing is designed to accommodate the relative risk and priority of the functionality being tested. In this environment, the highest-priority and highest-risk features and functionality get tested first, and with the most rigor. If schedules must be compressed, the team can be confident that the most important features are thoroughly tested prior to launch of the application. Make sure that nominal testing is segregated from challenge testing. This means that nominal or positive tests that verify that the system does everything it’s supposed to do are separated from challenge or negative tests that verify that the system does nothing it’s not supposed to do. When these tests are separated, regression becomes much quicker and easier, as challenge can be eliminated in regression rounds of testing, where changes to that functionality haven’t occurred. All too often, massive test cases with nominal and challenge testing mixed together are run for each regression, causing inefficiency and wasted effort. Ensure that challenge testing is completed for all critical features of the application as identified by your prioritization. Always produce metrics that identify the number of features tested and the severity of defects discovered by your testing team. When the development team recognizes that you’re finding the defects rather then the customer, they’ll become your strong ally.
Solicit assistance from leadership
One of the biggest challenges that QA managers identify is pressure from their direct management to meet a date, usually veiled in terms of customer demands. The conversation usually begins with management telling the QA manager that the customer is demanding the solution right away, and that the IT organization must do everything possible to deliver it on time, including compressing testing schedules.
If you ask any customer what they mean when they want it “right now,” they mean “correct now” not “bad now.” That said, most of our customers would rather we did what it takes to develop a good product, and want to understand why it will take longer than they think it should. The problem is that IT has traditionally overcommitted and underdelivered, so the customer has a low level of confidence that IT will deliver a quality product on the first try. The customer holds IT to tight timelines, realizing it will take several iterations to get to a final product that satisfies their business need. IT organizations have to change that perception with their customers. They can only do that by delivering on commitments.
How can you get your manager to help?
In many IT organizations, managers are rewarded based on what they deliver, not on how they do it. I have attended project celebrations where senior management was walking around patting themselves on the back for delivering a product that was millions of dollars over budget with major defects, and only a portion of the functionality originally prescribed. As a matter of fact, the only reason the release date was met was because it was changed multiple times. Managers celebrate getting something— anything—delivered to production on the release date.
So how do we change this “deliver at all cost” mentality? You have to solicit your management’s support for delivering it “correct now.” Most managers want to help. You really stroke their ego when you honestly and sincerely ask for their help. Managers are wired to be problem solvers and will go out of their way to address a problem if you ask for help. The problem is that most of us don’t want to ask for help, because we see it as a sign of weakness, when in reality it’s a sign of strength. When was the last time you went to your manager and informed him or her that you really needed their help to be successful? In his book The Servant—A Simple Story About the True Essence of Leadership (Crown Business, 1998, ISBN 0761513698), James Hunter describes how the role of a manager is to serve their employees by ensuring that they have everything they need (not want) to be successful and accomplish their goals. It’s not your manager’s job to be the barrier to your success, but to remove barriers. Be honest with your managers and ask them to help you strategize about how to get the things your team needs to be successful.
Take special care to look at how you’re serving your team at the same time. If you aren’t effectively managing barriers to your team’s success, then you aren’t serving your team. Is asking them to work just one more weekend, just one more long night really enabling them to be a successful team?
Identify the project degrees of freedom
Conversations about compressing the testing schedule never happen at the beginning of the project, they always happen in the heat of the moment, usually in a challenging situation when it’s too late to have a calm and analytical discussion. Development is late and we only have a couple of weeks, sometimes days, to make the schedule. It’s too late to add resources, it’s too late to get consultants, it’s too late to reduce functionality and it’s too late to change the schedule, because the customer has already scheduled training and the marketing blitz. We put all of the pressure on the overworked, understaffed testing team and expect them to, once again, save the day.
It doesn’t have to be this way. Imagine a project where a discussion between IT and the customer was held at the beginning of the project, in which all of the aspects of a project were negotiated up front when everyone was calm and good business decisions could be made. What a difference it would make to know up front that you can adjust the amount of resources on the project to make it successful, or that the schedule could be extended to obtain the desired functionality. No more would you be faced with deciding between two bad choices.
In his book Creating a Software Engineering Culture (Dorset House Publishing Co., 1996, ISBN: 0932633331), Karl Wiegers identified the five dimensions of a software project as:
Wiegers identified that for each of these dimensions on the project, there’s a degree of freedom that each one of these dimensions can be adjusted on a given project. The key to the success is to obtain agreement between the customer and IT team at the initiation of a project to identify the degrees of freedom for each of these dimensions.
This contract between IT and the customer removes pressure from individual project team members and manages the project to a predefined standard, removing personalities and politics from the decision process. It protects the much maligned quality team and ensures that either quality is given sufficient recognition and support, or that quality is relieved from burden as the customer accepts defects in advance. This is truly a win–win outcome for the customer and the IT team, and ensures that more projects are delivered on time, within budget and at an acceptable quality level.
Each organization will need to determine what components make up each of the degrees for each dimension. Imagine the power QA managers can have when the PM approaches them to reduce the testing effort. They can then explore all of the alternatives based on a predisposed and agreed-upon decision criterion. What a work that would be.
QA managers aren’t powerless victims of the schedule and bad project discipline. QA managers have the power to be champions for good project management, taking ownership for the success of their teams and ultimately the success of the products that IT delivers to customers.
While there is no magic formula for a project’s success, using the concepts and tools in this article will help you to become a more effective manager and will help your organization to deliver effectively on projects. You will have an organization where testers are valued and turnover is reduced. Ultimately, your customer will gain confidence in the ability of IT to deliver on commitments and quality. Imagine that world and begin to make it happen.
© 2023 Quality Digest. Copyright on content held by Quality Digest or by individual authors. Contact Quality Digest for reprint information.
“Quality Digest" is a trademark owned by Quality Circle Institute, Inc.