Improving the Testing Skills through Social Transparency

During their first real-world software projects, inexperienced developers often have trouble applying their knowledge about systematic software testing. They are overwhelmed with different engineering and collaboration tasks and testing tasks take a backseat. This, however, can lead to a misperception of the importance and value of systematic software testing - and may ultimately hinder the engineer in becoming a high quality engineer.

What keeps novice developers from systematic testing?

There are several reasons for this behavior. Sometimes, novice developers have a conceputal misunderstanding of basic testing concepts. For example, GUI automation is mostly associcated with the checking of the correct displaying of GUI elements ("does my button show up right here?"). While this is useful in certain cases, GUI automation can and should also be used for automation of interaction. Another example is JUnit. Inexperienced developers often associate it with numerical comparison in a mathematical context - the concept of a more complex target value and its proper definition seem problematic. Further, technical barriers (e.g. complex tool chains) or environmental constraints keep the novice developer from testing. For example, testing tutorials online are not easily transferrable to one's current test case and novice developers struggle in finding supporting educational material.

Model of the influences of an inexperienced developer's testing behavior.

Figure: A model of the influences of the testing behavior of inexperienced software developers.

 

How do software companies deal with this?

So, how do software companies deal with novice developers who lack practical testing skills? After all, when inexperienced new hires join the workforce of a software development company, the company will want to get them up to speed as quickly as possible.

We are interested in

Social Transparency can help!

We believe, that social transparency can help overcoming the skill gap between industry demands and inexperienced developers' testing skills more effective and more efficiently. Social transparency describes the "ability to observe and monitor the interactions of others within and across applications on the internet" [3]. The social coding site github.com is a thriving example of a collaboration platform with a high degree of social transparency: every user has a profile and can upload their project to an online repository. Projects can be browsed by attributes and followed. Contributing to other user’s project can be archived with several browser clicks. Development artifacts - such as commits - can be commented and discussed.

 

Model of the influences of an inexperienced developer's testing behavior.

Figure: Contribution process with a high degree of social transparency on github.com; the diamond represents the project owner's (PO) online repository, the package are the newly added improvements by an unknown contributor (C).

On github.com, software teams quickly diffuse their testing culture to mostly unknown contributors or new onboarders to the team. Clear testing signals - a badge signaling the use of a continuos integrations service or the existence of an extensive test suite - makes it clear for contributors that only tested contributions would be accepted. Also, when writing tests, contributors and onboarders heavily rely on existing test code: Similar tests are copied, pasted and adapted to one's own needs. The open environment and social transparency traits make it easier for contributors to understand the testing culture and mimic it.

These effects of social transparency can be used to specifically improve the testing skills of novices during onboarding. For new hires, it is helpful to collaborate closely with experienced engineers and to understand their rationale of solutions. However, close collaboration - or mentorship - implies a productivity cost for the mentor. Here, a high degree of social transparency makes this information exchange easier: the onboarder can to look up the mentor's test code and try to understand the rationale behind it - or use it to overcome technical issues. Clear testing signals during the onboarding phase helps the onboarder to adapt to the prevailing testing culture. Traits of social transparency can be integrated into the onboarder's IDE and help her find related testing solutions or resourcful colleagues more quickly.

Contact

The systematic use of social transparency for improving the testing skills of onboarders is currenty researched by Raphael Pham (profile link). Feel free to contact him for questions or research collaborations at Raphael.Pham@inf.uni-hannover.de.

Further Reading

In [1], we investigated how inexperienced developers test their software and what enables or inhibits systematic testing.

If you are interested in the testing behavior of software teams on github.com, you will find more information in [2] (see below). Leif Singer has written a blog post about our research on the testing behavior on github.com. This study was presented at the technical track of ICSE 2013 in San Francisco. A video of the presentation can be accessed here.