A Multiple Comparative Study of Test−With Development Product Changes and their Effects on Team Speed and Product Quality

Abstract

Researchers have typically studied the effects of Test-First Development (TFD), compared to Test-Last Development (TLD), across groups or projects, and for relatively short durations. We define Test-With Development (TWD) as more general than the fine-grained step of TFD, but also in contrast to the large-grained phase of TLD. With this definition, we performed a multiple comparative study to explore and describe TWD product changes, and the effects of those changes on two attributes re- lated to team speed and two attributes related to product quality, within six long-term open-source projects. Our results indicate that when developers exercised some of their changes with automated tests, on average they made significantly larger changes over time while significantly reducing their product’s complexity. And, when they exercised all of their changes with tests, on average they made significantly smaller changes over time. We interpret these results to indicate that practicing TWD supports faster sim- plification of a product. Therefore, we conclude that teams that need to simplify their product can benefit from practicing TWD.