Sign Up

Provide Developer-Approved Descriptive Names for Unit Tests

 

Unit tests are an important artifact that supports the software development process in several ways. In addition to helping developers ensure the quality of their software by checking for failures, they can also serve as an important source of documentation not only for human developers but also for automated software engineering tools. For example, when a test fails, its name can provide the first step towards understanding the purpose of the test and ultimately fixing the cause of the observed failure. Similarly, a test’s name can help developers decide whether a test should be left alone, modified, or removed in response to changes in the application under test and whether the test should be included in a regression test suite. In this proposal, we believe that test names are “good” if they are descriptive (e.g, they accurately summarize both the scenario and the expected outcome of the test) and “bad” if they are not descriptive. This is because descriptive names: (1) make it easier to tell if some functionality is not being tested—if a behavior is not mentioned in the name of a test, then the behavior is not being tested, (2) help prevent tests that are too large or contain unrelated assertions—if a test cannot be summarized, it likely should be split into multiple tests, and (3) serve as documentation for the class under test-a class’s supported functionality can be identified by reading the names of its tests.

Unfortunately, unit tests often lack descriptive names. For example, an exploratory study by Zhang et al. found that only 9 % of the 213,423 test names they considered were complete (i.e., fully described the body of test) while 62 % were missing some information and 29 % contained no useful information (e.g., tests named “test”). Regardless of the cause, non-descriptive test names complicate comprehension tasks and increase the costs and difficulty of software development. Because non-descriptive test names negatively impact software development, there have been several attempts to address this issue. For example, Zhang et al. and Daka et al. used static and dynamic analysis, respectively, to extract important expressions from a test’s body and natural language processing techniques to transform such expressions into test names. Researchers have also proposed test generation techniques that attempt to give the tests they create meaningful names. Unfortunately, while such techniques can be effective in generating descriptive names, they only work in the context of test generation. They can not be applied to generate names for existing tests. More recent work attempts to generate names that more closely match existing names by using machine learning to create approaches that can map method bodies to method names. However, the success of these approaches heavily depend on their training sets and they are not always successful at creating models that match well with human cognition [60]. In addition, these approaches are primarily geared towards general methods and have problems when applied to generating descriptive names for tests. For example, we have found that, due to the similarity of test bodies, these  approaches often generate duplicate names for tests in the same class. While such names may make sense in isolation, they are useless in practice as duplicate names are not possible and, even if they were, they would not serve the goal of helping developers comprehend the purposes of their tests.

The goal of my dissertation work is to develop a set of approaches that can address the problem of poor, non-descriptive test names. At a high-level my work is divided into two pieces. The first piece is to develop a pattern-based approach to detect non-descriptive test names and, if possible, suggest improvements. This approach is based on a mined set of test patterns and can: (1) detect non-descriptive test names by finding information mismatches between the test name and body of a given JUnit test, (2) provide descriptive information that is a summarization of the test’s action, predicate, and scenario, and (3) use the descriptive information to facilitate the improvement of non-descriptive test names by suggesting improvements to developers. From the collected results of the evaluation, we believe our approach is feasible, accurate at identifying the action, predicate, and scenario from tests, and effective at classifying descriptive and non-descriptive test names. The second piece is to develop an approach to generate descriptive test names. This piece of work contains three components. The first component was to conduct an empirical study of naming rationales that investigated whether tests are named after what makes them unique. A total of 440 tests were selected from 11 open-source projects on Github and used as the experimental subjects in this empirical study. The results of the study (1) demonstrate that a majority of tests are named, either wholly or in part, after what makes them unique, and (2) identify additional aspects that influence how tests are named.The second component was to build an automated approach for extracting the unique attributes of tests. The approach encodes the knowledge gained from the study into a tool that uses a combination of static analysis and formal concept analysis (FCA) to identify unique attributes. Our evaluation of the approach on a set of 920 randomly chosen tests shows that it is effective. The unique attributes it identifies match human judgment about ≈94 % of the time. The final component will build a new test name generation approach that builds on both results of the empirical study and the approach for extracting the unique attributes of tests. While the results of the empirical study demonstrated that tests are often named after what makes them unique, it also revealed that a test’s name is rarely a straightforward reproduction of its unique attributes. Instead, names appear to be the result of a complex transformation process that can add to, remove from, or modify the unique attributes. The goal of this component is to understand and then automate this process in order to create a test name generation approach that can create names that are not only descriptive but also match existing naming rationales.

Event Details

See Who Is Interested

0 people are interested in this event


User Activity

No recent activity