A Quick Question

I am in the beginning stages of investigating what it would take for me to start a tester meetup of some sort in my area.  I had been thinking for awhile that I might write about the process, but so far I have met many people who already run or otherwise participate in a meetup, or some other form of tester community.

The question I have for you is, if I wrote about the process of creating a from scratch tester meetup, would you read it?

Please let me know via the comments below, or via Twitter.

EDIT: OK, so I had enough interest within the hour that I first put this up that I decided I will in fact start my piece (or pieces) on creating a tester meetup.  I don’t know when it’ll be up, but I will for sure do it.  If you still feel like you want to tell me you’d read it, that’s cool (and would make me feel good) but it will affect the existence of the piece any.

Balance to Avoid Inattentional Blindness

NPR had an interesting piece on this morning about a study of inattentional blindness on radiologists. link

I was a little sad that they didn’t include ‘tester’ of any kind in their short list of professional searchers. It hit me while listening to the story how easy it is to train someone to NOT see potentially important information. Give them too detailed of instructions and most people will do exactly what yo stated, nothing more. Tell them too specifically what they are looking for, and many will miss everything else.

How does this relate to testing? I see it as a divide between two (for purposes of this example, extreme) camps: exploratory testing vs. testing via test plans.

Personally, I like exploratory testing, I find it more freeing, and it allows me to just play with stuff. The downside is, without any structure around it (charters, time boxing, outline of goals, etc) I tend to dive into the minutia of some esoteric part, and I miss important, big things. On the opposite end of the spectrum you have detailed do-this-expect-that type test plans (or scripts if you want to go even further in that direction). These can be beneficial in verifying that a previous error no longer occurs in the exact way it did in the past (if you need to do that), or for giving brand new testers an idea of things to look for in testing. If all you ever do is follow plans though, you will not train your brain to search for anything ‘weird’. You will be repeatedly teaching yourself to only do what the paper says. Machines do that. Anyone else remember these existed?

Balance is the key. Use some sort of frame for your exploratory testing. Don’t just dive in totally free form; create enough structure around what you are doing to allow you to stop and take a breath once  . Coming back to reality will help keep you from getting stuck too far down in the details. If you are tasked with following a plan, stop every once in a while and look at the product. Look at it outside of the context of the expected results and try things. Give yourself a set amount of time and then go back to the plan.

Stop defeating yourself

<02/03/13 Edited for clarity>

What do you do when you are told an idea you had, something you’ve done, or a decision you made was wrong?  Maybe someone told you your idea was not heading in a direction they would have taken? What if your boss tells you you made a choice that they would not have made? What do you do when someone tells you an idea you had, or something you’ve done, or a decision you’ve made is not the direction they would have taken, or not the choice they would have made? Based on my experience, most people do one of two things.

1. They get defeated. They take the criticism to heart and shut down. They drop their idea, or work to undo the work they’ve done. Each time they do this, it destroys a little bit of their will to make decisions for themselves.

This can be somewhat the fault of the person giving the feedback, but it is mostly the fault of the person who let themselves be defeated so easily.


2. They realize that at the end of the conversation, when the smoke clears, that nothing changed. If your supervisor tells you they wouldn’t have done things the way you did, but then DOESN’T tell you to stop, or to undo what you did, who won?  You did!  If you feel strongly that you are doing things the right way, you get to keep doing the right thing.

An even better option (I know, I know, I said there were only two, but it’s my blog), is to win like option two, and then think about the feedback you got. Why did someone tell you they didn’t like your idea?  Could you have communicated it in a better way?  Did you leave out key pieces of information that would have made your idea or decision more palatable to your audience?  Do you need to even share your ideas with that person in the future?

I have watched too many people take negative (non-constructive) feedback that included no consequences as a sign that they should stop thinking or doing like they did. Pay close attention. Even if you feel scolded, if you weren’t told to stop, you just won.

If you really want to do the best you can, you should be looking for whatever feedback you can get. For more on this, please read Keith Klain’s excellent post on scrutiny.

Trying on a few hats

As I have intimated here before, I have held several different roles in my career, all in the testing arena (or should I call it a stable, what with the farming metaphor-for-a-title thing and all…).  I have been a tester at three different companies of varying size/environment/culture.  At my current company, I started as an individual contributor, testing various parts of our product suite.  I was promoted to be a team lead, then a manager, and then a senior manager.  After four+ years in that role, that layer of management was removed and I moved back to a manager position.

After spending some time reflecting, especially on my more recent position change, I began to realize how much I had learned by wearing all of those different hats over time.

The time I spent at my first company (testing ATM software) taught me that I don’t like working in heavily regimented environments.  This company was (at the time) an ISO shop.  That meant I had stickers on all of my books and binders telling people, who may accidentally pick them up, that they contained out of date information.  It meant I had to go talk to the tester-turned-guard who manned the locked file cabinet if I wanted to see an actual “current” version of a document (this was the late 90’s).  I was eventually let go (along with all of the other contractors and some full time people) to help raise their stock price.  With that, I learned one of the consequences of working for a public company.

In my next position, at my second company (testing the print accuracy of engineering documents on wide format printers), I learned that I liked working in an open environment where I was allowed, and even encouraged, to work with my developers.  I was eventually let go (along with the newest 20% of the staff) to help prevent the office from closing (it was gone within a year).

At my current company, as a tester, I learned to really enjoy testing.   I was able to finally remove a (in my eyes) wrong approach to testing that my first employer had ingrained in me.  Later, as a team leader, I learned that I liked coaching people.  I felt satisfaction helping others to grow and mature as testers.  When I was promoted to manager, I had to completely change what I did every day.  I was no longer involved in testing the software.  I spent most of my time managing teams of varying sizes, trying to create training out of thin air, figuring out how to help develop and grow a team of people (up to 28 strong at one point), oh yeah, and making sure the software got tested.  After some time, practice, training, advice from others, and just possibly a little luck, I got my hands around most of the role.  I would never say I was awesome at it (I am a fairly strong introverted thinker if you follow MBTI), but I was able to manage (pun intended) and grow my people (yeah, and get some software tested as well).  Then, in the spring of 2008, I was asked if I would take on the new, not yet defined, role of Senior Manager.  A coworker and I were each given half of the department as reports.  I spent the next four years trying to manage my managers and stay connected with their teams.  I was too far removed from testing at this point to have any role in the development or advancement of testing.  Or at least I felt I did, despite some people’s best efforts.  I tried very hard to metricize everything.  I built Excel spreadsheet after Excel spreadsheet.  I reached the data filter limit of our internal support ticket and bug tracking system trying to report on the age of issues per manager.  I spent a lot of time and energy trying to figure out how to sum everything everyone was doing into a simple, easy to digest number that I could pass up the chain.

NOTE: If you are thinking of trying this yourself, please take a look at Paul Holland’s wonderful post on Bad Metrics.

Eventually, I realized this was all in vain.  Sure, I could measure some things, like how long issues have been in test, or how long it’s been since a support ticket was opened, but trying to boil it all down so that I could compare testers of differing backgrounds, on differing teams, testing issues of differing size on different portions of the product in differing environments with differing complexities is ludicrous.  It took me too many years to figure that last line out.  So, my one and only peer and I started trying to affect change as best we could.  We took control of our hiring process and worked to find new places and ways to find tester candidates , we worked to improve processes, and tried to find ways to get the testers we hired to stay testers for the long term.

Last March, when this layer was removed, I was initially dumbstruck.  I was angry.  I felt let down, and didn’t know what I was going to do.  A couple of days later I came up with two demands:  to be sent to our user conference to regain a connection with our customer base (I ended up participating in three half day training sessions, and co-presenting in a total of five informational sessions) and I wanted to go  to CAST for the first time to reconnect with testing (I was at CAST2012).  Over the past year I realized that in the end, I didn’t really like the senior manager hat that much.  Sure, I learned a lot about myself.  I was eventually able to function at that level fairly well.  I spent four years focused at a level above where I currently work.  That makes it even easier to do what I do now (manage a team of nine instead of 65).  I’ve experienced managing translation teams, international test teams, remote employees (in other states, and other continents).  I grew more confident as a leader in that role, making many of the decisions I make in my current role seem like child’s play.  As of today, despite all I learned from the role, I don’t know that I would take that position again if it were offered to me. I wouldn’t go back and undo my time as a senior manager, but that is not where my passions currently lie.

The point of all of this rambling is that I learned something from each of the hats I have worn.  Each position has helped to mold me into the person I am today.  I may be lucky, but I work in an environment where changing positions is generally looked at as a positive thing, especially if your passion can be better utilized in a role still within the company.  With this environment, I felt comfortable stepping back into a role I was in previously, without feeling I would be looked at as some sort of pariah.

My advice is to not be afraid to try on different hats.  Sometimes you might even find that an old hat fits much better the second time around.  Without retreading an existing post by my host, I would expand a bit on Paul’s Reinventing Yourself, to say don’t just look at reinventing your tests, or how you approach things.  Don’t be afraid to change your whole role.  Feeling stuck in the day to day grind of testing bug fixes?  Have a penchant for usability?  Begin showing the value of having a dedicated usability tester.  If wouldn’t expect it to work overnight, but if you show the right people where your passion lies, and what they can gain from letting you loose in that direction, it will happen.

If your environment frowns upon people changing roles (and possibly asking to switch back if things don’t work out) maybe it’s time to look for a new environment. If you happen to like the greater Cleveland area, I know a company that’s looking for testers.

So long 2012, and thanks for all the tests

As some people know, 2012 has really been a rebuilding year for me with regards to my career and my love of software testing.  I have been involved in testing for over 13 years, but I spent a good chunk (4-5 years) in a senior management role, which effectively removed me from anything remotely involving testing.  I managed managers and spent all of my energy trying to be good at that.  After that role was eliminated and I moved back to a line manager role, I discovered (again) everything I had been missing all those years.  I really like testing.  I like using the part of my brain that I need to use to think like a tester (which I found had been turned off for years thanks to my focus on HR issues and processes and procedures).  I like sitting with my team (now 10 instead of 65) so that we can have thought provoking discussions.  I like touching software.

Sure, my position change had a part in this, but I really see two other pieces to this puzzle as key to where I am today.  The first thing I did when I got over being mad for being pushed out of my old role (which in the end was a blessing, I just couldn’t see it at the time) was demand that I be sent to CAST to reconnect with testing.  The second key step was making a Twitter account and connecting with anyone I could find that I thought was interesting, starting with the context-driven testing community.

So, with a bit of my back story (and returning to the original intent of this post), I wanted to thank a few of the people that helped me along my journey this year.

Thanks to James Bach for coming out to speak at two NOSQAA events, and to Shaun Hershey for getting him to come talk at Hyland.  Thanks to Michael Bolton for the many Twitter discussions, the offers to help me start a tester community and for a very awakening full day session at CAST 2012.  Thanks also for sitting in on a game of Zendo at CAST and for sharing your beer (I still don’t assume beer on the table is for sharing by default).  Thanks to Eric Proegler for continuing to give me good advice on testing and testers.  Sorry it took me years to catch up with what you were saying.  Thanks to Claire Moss for being friendly to a new-comer at CAST and for incessantly favoriting many of my tweets.  Thanks to Dean Biron for introducing me to Zendo (much to my bank accounts chagrin).  Thanks for Paul Holland for showing me the dice game, blowing my mind, for sharing your The MacCallan and for being patient.  Thanks also to Paul for the many Twitter discussions, and for (reason still to be determined) hosting my blog.  You were key to me actually getting this thing going.  Thanks to Wendy Robinson for being my testing buddy and co-advocate for change during much of the past year.  I am really glad we got you out to CAST.  You started or helped start our internal testing game nights, the re-awakening of our testing forum and our lunch time tester discussions.  Thanks to Mike Talks for the earl (and deserved) teasing about being a recovering senior manager (I got better).  It helped me further confirm that I did in fact want to get away from managing managers and get back in touch with testing and testers.  Thanks to Jari Laakso for the interesting discussions, for letting me ‘peer review’ one of your posts, for pushing me to blog and for giving me the inspiration for the name.  Thanks to Matt Heusser for the numerous discussions, tips, introductions and for the (soon to be) training.  A bit of a warning, I’m looking to pick your brain off hours in January if possible.  Thanks to Anna Royzman for the session at CAST, and for the invite to join the AST Leadership SIG seemingly out of the blue.  I am really looking forward to this group taking off and making a difference in the community.  Thanks to Pete Walen.  I’ve been following you for awhile now on Twitter and find many of your insights interesting.  I enjoyed (finally) getting to speak on the Leadership SIG call last month and I look forward to seeing you in January with Matt.  Thanks to Mike Larsen (and Matt again) for the TWiST podcasts; I’m not caught up on all of them, but I have really liked the topics so far.

Thanks to Eric Brickarp, Martin Hynie, Paul Clewell, Mike Lyles, Keith Klain, qualitycaptain, Guy Mason, Simon Morley, Matt Hutchinson, and numerous others for the conversations, tips, help, info, re-tweets and everything else along the way.

I appreciate everything that you’ve all done to help me this year.  Whether you realized it or not, you’ve helped me tremendously.  I am a better person, and a better, more connected, more involved and more passionate tester and test advocate thanks to all of you (any the many I have forgotten in my rush to put this together).

So I Started a Blog

First, I need to thank Paul (thanks Paul) for offering to host my blog on his site.  I probably would have eventually gotten around to putting one together myself without his nudge in the right direction, but it definitely would not have happened this quickly.  I’ve been thinking about writing a testing blog for a few months now, but never did anything about it.  At the end of November of this year, I had a Twitter conversation (can you call them conversations if they only take place via 140 character long strings?) with Jari Laakso about possible blog topics.  Somewhere in that discussion, I halfheartedly mentioned needing to set up a blog. The next day, I had a DM from Paul stating “If you are seriously interested in blogging, I could probably set you up on my site. Let me know and I can I investigate the logistics.”

Let me digress a tiny bit here. I may have been in test for over 13 years, but I still feel very new to the context driven community (even if I was unknowingly agreeing with them for some time).  Going to CAST2012 and then researching testers on Twitter and blogs, I came across a tier of people in the community that I saw as the ‘big names’: Michael Bolton, the Bach brothers, Dr. Cem Kaner, Lynn McKee, Scott Barber, Matt Heusser and others.  From my time working with Eric Proegler, I had also heard of, and come to respect Paul Holland. Sure, I played ‘the dice game’ with him at CAST and shared some scotch, but I was still in little bit of awe.

So here I was, with an offer to be hosted by none other than Paul Holland.  What could I do?  I’d look stupid if I said no, and then eventually gotten around to putting a blog up somewhere.

So here I sit, with a blog all configured, several topics in my head (OK, they’re in a mindmap really, my memory is pretty much rubbish), and I have no idea what to call the thing.  So I do what many people do nowadays.  I asked for ideas on Twitter.

I got several things back:

  • use puns
  • make a play on words with your initials
  • write a blog post about trying to name your blog (thank you Martin)
  • an entire chain of silliness revolving around the maximum number of times “enterprise” could be used in the title

I then moved on to the internet for ideas.  How about words containing ELD?

  • unwieldy – That means I’m either fat, or awkwardly pointy
  • sheldrake – Cool sounding word, but I am not duck that masquerades as a goose
  • chield – young man or fellow, it’s cool because it’s Scottish in origin, but I may be too old to be a “young” man
  • Testing Minefield – dramatic, catchy, serious, I’ve just made my blog into the NBC Nightly News, or this
  • wergeld – wow, the monetary value assigned to every person in every class of society to be paid upon their death by the perpetrator
  • Dieldrin – oh awesome, a deadly insecticide that biomagnifies as it moves up the food chain?  No thanks
  • Aceldama – OK, I gotta stop this.  The field of blood so named because it was purchased with the money Judas got for betraying Jesus

I look at songs.  Chemical Brothers “The Test” is a great song, but it’s a but too drug referency for a blog about testing.  Maybe movies would be better?  I love Monty Python’s Quest for the Holy Grail.  There’s even the scene with the bridge keep at the bridge of doom, “answer me these questions three” and all.  How do I turn that into a blog title?

For now, I think I’ll have to fall back on a derivation of an idea that Jari threw out in our chat back in November.  Sowing Seeds: A Testing Blog

On Guessing

While at #CAST2012, I visited each of the (maybe four?) vendor booths. I had good discussions at two of them (with test suite vendors, mind you), and even got two free shirts. (I’m a sucker for free shirts.) At the Telerik booth, I also snagged a sticker that reads “I never guess. I test.”

At the time, I thought it was cool and proceeded to stick it on my portable HDD. A few days later, it hit me differently. Is it true that I don’t guess when testing? Is guessing a bad thing when it comes to testing?

After thinking about it a bit, I realized we guess all the time in testing. For example, when you first start to test a new feature or module, you have to make guesses as to how it might work. Sure, you may have some info from dev on what the feature is supposed to do; maybe you even sat in on design discussions, if you were lucky. But when it comes down to test time, the first thing you are doing is looking for patterns to help you uncover bugs. You start out with an opinion about how the module/feature should work, without having sufficient evidence to support your opinion (otherwise, why would you be testing it?). The results of your testing ARE the evidence that either confirms or contradicts your guess.

That IS the definition of guessing.

No, really, it is. Look: dictionary.reference.com/browse/guess

  • “to arrive at or commit oneself to an opinion about (something) without having sufficient evidence to support the opinion”

When most people first begin a career in testing, they follow the recommendations of other testers, do what their developer told them to or follow a test plan/test case (without much deviation). This is fine in the beginning, as it gives you a chance to see how others do things and to understand the expectations for testers here.

In the end, we NEED to guess/speculate/conjecture/hypothesize to advance as testers, even if that goes against some of the more rigid testing methodologies out there. To make an educated guess is an intellectual activity. Testing should be a challenging, intellectual process.