摘要:众测是一种新兴的软件测试方法，它依靠网络上的工作者帮助完成测试任务.对于某个测试任务来说，谁来执行对于发现缺陷以及覆盖测试需求关键点是至关重要的.然而众测平台上一般有大量的候选工作者，他们拥有不同的测试经验，也常常提交重复的测试报告.由于众测工作者随机地参与测试任务，同时满足较高缺陷检测率和较高测试需求关键点覆盖度是很困难的.因此，该文关注如何为新的测试任务选择一组合适的众测工作者，从而提高缺陷检测率和需求关键点覆盖度.首先设计了3个实验，试图发现选择什么样的众测工作者能够提升缺陷检测率和需求关键点覆盖度.通过实验验证，发现众测工作者的主动性、相关性和多样性从不同的角度影响测试质量，并且给出了它们的度量方法.然后，提出一种同时考虑这3个方面工作的选择方法.基于众测平台之一——百度众测上46个真实的测试任务对该方法进行了验证，结果显示，该方法能够显著提高缺陷检测率和测试需求关键点覆盖度.



Abstract:Crowdsourced testing is an emerging trend in software testing, which relies on crowd workers to accomplish test tasks. Thus, who performs a test task is extremely important for detecting bugs and covering key points of test requirements in crowdsourced testing. There are a lot of candidate crowd workers who may have different testing experience but can also produce duplicate test reports for the same task due to the lack of cooperation. As crowd workers can freely participate in a test task, high quality of testing in terms of bug detection and coverage of key points of test requirements is not guaranteed. Thus, to improve bug detection and coverage of key points of test requirements, selecting an appropriate subset of workers to perform a test task is becoming an important problem. In this paper, three motivating studies are first conducted to investigate important aspects of workers in detecting bugs and covering key points of test requirements. Accordingly, the studies identify three aspects:initiative, relevance and diversity are identified, and produce a novel approach for selecting workers considering all these three aspects. This new approach is evaluated based on 46 real test tasks from Baidu CrowdTest, and the experimental results show the effectiveness of the approach.



PDF全文下载地址:

http://jos.org.cn/jos/article/pdf/5329