Managing Test Organization Transformation

Recently, I attended the 9th Workshop on Software Testing in Financial Services (STiFS9) in New York, hosted by Liquidnet. The Theme of the workshop was “Organizational Structure Models for Test Groups at Financial Firms.” One of the discussions that struck a rich vein of ideas was what is needed for successful transitions between types of organizational models in quality organizations.

While the experiences that the participants drew on tended to be with larger organizations in financial services industries, these ideas could be useful for any testing organization’s transformation – or maybe even outside of testing.  Specifically, we were talking about organizations that were moving from a centralized to decentralized testing organizational model – or from decentralized to centralized in organizations with dozens or even hundreds of testers.

STiFS is a LAWST-inspired peer workshop, meaning that the value of the workshop is the product of all of the participants in the workshop, which included Bernie Berger, Kaveri Biswas, Margaret Boisvert, Ross Collard, Joe Lopez, Mike Pearl, Don Pierce, Anna Royzman, Ben Weber, and myself. We captured pages and pages of ideas from these very smart and experienced testers and test managers. Bernie Berger, the CO of STiFS9 (and the driving force behind STiFS) has provided a report on the workshop here.

This post is not an official finding of STiFS9. It doesn’t include all of the discussion or notes and only represents my interpretation and reflection on what I heard. Other participants may have a different take on what was discussed and what it meant. This post is what it meant to me.

A decentralized testing organization, as I mean it here, has testers reporting into specific projects as part of the staff. This approach might be marketed as “(More) Agile“, though it is certainly not necessary for agile, Agile, Scrum, or Certified Pokemon Master processes.

Arguing Over User Stories
What I Think of When I Hear Scrum

A centralized testing organization usually has a reporting structure that includes most or all testers in an organization in one department, which allocates testers for providing testing services to specific projects. Testers report upwards in the testing department and may work on several different projects, depending on the needs and desires of the whole organization. One marketing term associated with this approach is “Testing Center of Excellence.”

2,857 Career Story Points
A True Centre of Excellence

For the testing organization’s transformation project to start successfully, there needs to be top-down support, from the executive levels of the company. This support can be demonstrated and described by clear communication of the organization’s commitment to the transformation and the goals of the transformation. These goals may be to respond to changing market conditions, updated regulations, concerns about the quality and speed of delivery in the organization, or some combination of these and other factors. They should be articulated as the mission that the transformation is designed to accomplish with clear objectives and sufficient justification.

This communication of goals is usually understood as an essential step for aligning the organization, but less obvious is the need for selling the transition to the organization without jargon or “business speak.” This transparency builds trust with employees, who need to feel safe during transitional periods, need to feel included in the process and planning, and to be optimistic about their jobs and the organization’s future after the transformation. Many people are uncomfortable with and even resistant to change; laying the groundwork will help them process the changes to come. Buy-in enlists people in the organization to support the transformation and its goals, and helps retain key employees through periods of uncertainty. This was called “bottom-up trust.”

This leads to another core requirement: an organization’s leadership may draw up a new org chart (and the new org chart should be clear to everyone), but that doesn’t explain *how* the organization changes successfully, can’t describe all of the details and requirements, or fully define the new roles. In fact, almost all of these details will not be understood and defined at the start of an organizational transformation, making broad buy-in even more essential.

The real hard work of solving problems and helping define new structures is likely to be undertaken by “Champions” – key contributors throughout the company who are influential and respected by their co-workers. Many of these champions should *not* be managers, so that their motivation and contributions are clear. Some influence could come from outside consultants, but that will vary by situation and consultant.

The champions should be known to and perhaps recruited by the person that owns the transformation – the transformation owner. This person must be able to solve technical and political problems, make decisions that stick, and manage an ever-mushrooming list of details. The organization should expect that they will get clear, credible, and authoritative communication on progress from this person – and receive it regularly. A trusted source of information helps keep the organization aligned, and may help counter-program the rumor mill that uncertain people will be paying extra attention to.

The transformation owner should drive collaboration and keep people aligned. One way to support this collaboration is impact analysis on stakeholders, to identify problems, and to help create patience during periods of lower efficiency and missed deadlines that will surely occur as the organization is changing direction. This allows the stakeholders to plan and adjust accordingly, by making it explicit that expectations should be renegotiated.

Communication is only one requirement of the role, though. The owner must be able to advocate upwards and across the organization for accountability and buy-in from everyone to the transformation mission – and get it. Short-term revenue and budget pressures will contend with the work needed to create new processes.

The transformation owner can sell leadership on the need for patience and tolerance for initial failures, and run pilot projects to help make the transformation smoother for the rest of the organization. Teams should be able to experiment – to test the transformation – and provide feedback. The willingness to adjust the plan based on this feedback increases the chance of success.

One thing that was noted as frequently missing in transformation plans was engagement with and consideration of risks. There are lots of different kinds of risks; by engaging and enumerating risks, mitigations can be found. Not only should success criteria be understood – failure criteria may be helpful, too! Frequently, rollback is not an option, but contingency planning may be very helpful in getting through rough patches, particularly if the “worst-case” scenario has been understood.

As the details of the plan are being built, sufficient effort should be allocated to workflow analysis. How does work get done today? How will it be done in the future? Some things will fall between the cracks, and the people who would have noticed previously may be doing different things at that point.

An important part of the transformation plan is knowledge transfer. A concept of an “Organizational Level of Knowledge” was described; the cumulative knowledge of the organization is an important asset, and its value should be recognized, and hopefully preserved. Process documentation is easy to identify, but the unwritten rules are harder. Who can really get something done in a given area? How do you put an announcement out on the grapevine? These are not issues that can get resolved with an increased training budget; they took years to develop and be discovered.

Enter beer.

Wine is fine, liquor might indicate a problem...
The Secret Sauce of Transformation

Whether it is used for team building, to provide opportunities for horizontal communication, or as a coping mechanism, there were broad and strong recommendations for using beer to create social situations for team building, casual/informal communication, and reducing stress levels. This touched off a series of comments about the critical effects on employees going through and adjusting to transformation.

Perhaps the most critical issue is having well-defined new responsibilities. People need to know what is expected of them, and they need to trust that they have clear guidelines. This helps them feel secure, helps makes reporting relationships clear, helps set expectations, and helps them adjust to their new position.

It was suggested that there should be additional focus given by HR (and overtime, if necessary) to understand and hellp mitigate impacts on specific people who are dealing with change. People working with new managers are more likely to take complaints or issues to HR instead of the manager; they need to feel that they are heard and their concerns are addressed. This also gives HR a chance to focus attention and effort on keeping key people such as star performers happy and productive.

One of the strong signifiers (or moment where it becomes real) is when people have to move their desks. Movers, phones, networking, etc. all have their own issues and schedules and will need to be coordinated. Offices and workspaces may need to be redesigned.

Sitting with the new team is important, but there are real impacts beyond a change of scenery. If the geographic location changes and people have to move, there should be some care in how to help them understand it. Even changes like an impact to the commute are going to hit harder when surrounded by uncertainty.

People will need training and coaching for new roles and processes. Agile was cited as a commonly experienced part of a transformation, but even if methodology isn’t changing, there are certainly other changes in workflows, and/or required domain/technical knowledge. Testers should feel supported and that there is patience and tolerance for them to skill up to their new roles.

Some of technical issues around testing were collected by the group. Maintenance of automation and preservation of code and test assets was reported as often overlooked. There is a need to resolve the ownership of shared test code and data – or a decision to abandon it. Test labs and equipment will have to find new owners, and it should be clear how to schedule shared resources. Some cleanup should take place after the project completes.

More subtle impacts include ownership and responsibility for compliance concerns and go-forward updates/forwards/changes/retirements of email aliases, message groups, and distribution lists. Managing configuration changes may need significant rethinking, and permissioning was mentioned as a specific problem that had been encountered.

Even when testing is broadly distributed, some testing functions such as Automation or Performance Testing may still be best supported horizontally. This should be considered as part of the plan, including how to request support, and who will lead these functions.

In addition to all of this input about planning, the group had suggestions for conducting the transformation project. Once the transformation project is underway, continuing updates against frequently occurring milestones will help maintain cohesion and focus. Phases of the project are helpful for keeping things on track.

The plan and progress should be highly visible, and some advocated for a metrics regime as part of understanding where the organization was before the transformation, how the organization is managing during the transformation, and how successful the transformation project was in improving the organization’s testing efficiency. There was a lot of energy around engaging with the measurements that would be used by the organization to evaluate the project. Some of the proposed uses of metrics included the awareness of strengths and weaknesses, and current efficiencies and costs. Skepticism about metrics led to a discussion about the need to avoid inherited bias and future gaming of the system by using clearly documented and tangible metrics.

For project management, phases were suggested to keep the steps small enough to manage. There should be frequent check-ins to discuss how things are going, with different groups of people. Milestones should occur regularly, so that efforts remain bite-sized. This also helps build in flexibility by providing opportunities for assessment and adjustment.

It is also important to celebrate reaching these milestones. Celebration of successes should not wait until the entire project is complete, but occur throughout to help build positive momentum and maintain visibility.

Even with frequent phase events, patience will be necessary, both with schedule and results. Significant changes are being implemented, and everyone involved has to give them a chance to succeed. A couple of knowing comments came up here: “Projects unravel in unexpected ways.” and “Don’t panic because things are going perfectly.”

This patience has to include some tolerance for failure, and patience with understanding and correcting failure. There will be some risk for people to speak up and identify failures and their causes; transformation is frequently a top-down exercise. There is likely to be some disagreement about whether something has failed, or about the reasons for failure when the failure is acknowledged. When the organization is stuck, those responsible for planning are likely to blame the execution.

Failures must be engaged with thoughtfully, so that the organization can learn from it and not repeat it. Managing failure is important for the safety of the organization, and everyone involved. There are first order costs on others such as stakeholders, but the testers are going be frustrated and potentially frightened by failures. The organization has to know when to re-evaluate the situation, when to put parts of the plan on hold, when problems are fixable or not, and – perhaps most importantly – to understand and expect that these will be tactical decisions made during the project, when more information is known.

Discussion about completing the transformation (or declaring it done) came up again and again. As one participant put it, “Sooner or later, you have to land the plane”. People need to understand that the new reality is here, and that they can exhale and focus on doing their new jobs well.


Comments are closed.