Using TDD (Test Driven Development)

If you read my previous post (WordPress Recovery), you know I’ve been writing some code to recover my old posts. It occurred to me I could take a small segment of what I’ve been doing with that code to demonstrate my approach to TDD.

Since I’m a hacker from way back, and also because I was in semi-panic mode about losing the content, I didn’t approach this task with testing in mind. Now that doesn’t always result in bad code: I’ve been doing this long enough that I can usually think through a fairly good model and code something that isn’t one long method full of inline code.

In this case however, once I had started coding, I realized again that this was an opportunity to practice coding the “right” way. I had already begun with a Maven project, and generated unit tests as I went through the process of starting to build the code, so I had at least some good functioning unit tests.

Most of the modern IDE’s let you run the unit tests directly and present you with a nice UI that can let you see where the tests are working. And with Maven, your unit tests have to succeed in order for you to actually run the code. I personally like the way the NetBeans UI shows you the test results and lets you filter out the ones that succeed or have warnings.

Anyway, on my WP recovery code, I had originally taken some shortcuts and hard coded a few values that I realized I could find in the posts. To illustrate TDD, I decided I needed to add a method to get the actual categories from the input files, and put them into the proper tags in the output XML.

In the current tests, the category is hardwired to be “Recovered Post”, since that was what I had put into the original code. So thinking as if testing were the most natural way of coding, I needed to add:

1. A new test for the getCategories method using an Article that was instantiated with the appropriate HTML
2. Updates to the tests that have comparisons with the hardwired values for categories.

So diving right in, I add the following to my ArticleTest.java:

    /**
     * Test of getCategories method ..
     */
    @Test
    public void testGetCategories(){
        System.out.println("getCategories");
        List expectedCategories = new <a class="zem_slink" title="Dynamic array" href="http://en.wikipedia.org/wiki/Dynamic_array" target="_blank" rel="wikipedia" data-mce-href="http://en.wikipedia.org/wiki/Dynamic_array">ArrayList</a><>();
        Article article = new Article();
        List resultCategories = article.getCategories();
        assertEquals(expectedCategories, resultCategories);
    }

Now in the IDE, there’s the red squiggle under the getCategories method that indicates something is wrong, and at the left side of the edit pane there’s the little light bulb that indicates there’s a suggestion for you. Clicking on the light bulb, you get the option of adding a getCategories method to the Article class:

Clicking the little popup actually stubs out the method in the Article class, which looks like:

    List<String> getCategories() {
        throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }

At this point you can run your unit tests, and the code will fail with the UnsupportedOperationException  shown above. Since I’m trying to add a field to my bean, I update the stubbed out method like:

    /**
     * Get the categories
     * 
     * @return Array of Strings representing the categories ...
     * 
     */
    List<String> getCategories() {
        return this.categories;
    }

Once again I see squiggly red underline and a light-bulb:

This time there are multiple hints. Just in case we were meaning to call some external library, Netbeans asks us if we want to search Maven for “categories” that would return the List<String> that we specified. But since we want to add a new field, we click the “Create field” choice, which generates:

private List<String> categories;

At this point I typically add comments and have NetBeans generate the setter by right clicking on the field name and choosing “Refactor” and “Encapsulate Fields …” from the menu:

That brings up the “Encapsulate Fields” dialog, and you simply click the “Refactor” button to generate the setter:

So now you have both setter and getter:

    /**
     * Get the categories
     * 
     * @return Array of Strings representing the categories ...
     * 
     */
    List<String> getCategories() {
        return this.categories;
    }

    /**
     * @param categories the categories to set
     */
    public void setCategories(List<String> categories) {
        this.categories = categories;
    }

So a quick run of the tests and we see a failure (which is what I expect):

The reason for this is that we haven’t done anything to set the field when the Article is constructed, and our test is expecting to get an empty List of String objects. To correct this, we either need to change what we’re expecting, or add some code to initialize the categories field to the empty List.

The way I’ve coded the Article object, I will normally be instantiating it with the contents of the input files as a List of Strings which get passed to the addItem method where we actually pull the data out of the file contents.

From my analysis of the input files, I know the categories are contained in the <article> tags’ attributes, and I’ve previously added code to parse out the Post ID and the tags:

            /**
             * Begin article - this includes some meta data
             *
             * <article id="post-17" class="post-17 post type-post
             * status-publish format-standard hentry
             * category-restful-web-services category-web tag-data
             * tag-data-formats tag-html tag-markup-language tag-tools
             * tag-uniform-resource-locator tag-xhtml tag-xml content-single ">
             *
             * A "page" will look more like:
             * <article id="post-1958" class="post-1958 page type-page
             * status-publish hentry content-page">
             */
            if (s.contains("<article")) {
                // If we don't see "type-post", then it's not a post
                if (!s.contains("type-post")) {
                    break;
                }
                // Parse the tags from this line
                parseTags(s);
                // Parse out the post ID
                parsePostId(s);

            }

So, I’m going to add a new parseCategories in that same general area, and then write the code to get the tags. Again I let the IDE help me out by stubbing the method for me, and then update it to be:

    /**
     * Parse out the categories from the line ..
     * 
     * @param s String to parse the categories from ...
     */
    private void parseCategories(String s) {
         if (s.contains("category-")) {
            String[] firstCategory = s.split(" category-");
            // Remove the content after the categories ...
            firstCategory[firstCategory.length-1] = firstCategory[firstCategory.length-1].split(" tag-")[0];
            // Copy everything but the first one ...
            firstCategory = Arrays.copyOfRange(firstCategory, 1, firstCategory.length);
            // Initialize the array ...
            categories = new ArrayList<>(firstCategory.length);
            categories.addAll(Arrays.asList(firstCategory));
        }
    }

To make this more testable, I could expose it as public and return the List of category Strings instead of pushing the results into the categories field, but for now this is really what I want. But this really hasn’t addressed my testing. What I need to do is change my failing test to expect null when the object is first instantiated, and add tests for a couple of other scenarios:

1. A test to make sure the values are right after calling the setter.
2. A test that will instantiate with a string that has no categories on the article line and expect an empty array
3. A test that will instantiate with a string that has categories and expect matching values in the categories array.

So I update my testGetCategories method as:

    /**
     * Test of getCategories method ..
     */
    @Test
    public void testGetCategories() throws IOException{
        System.out.println("getCategories");
        List<String> expectedCategories = new ArrayList<>();
        Article article = new Article();
        List<String> resultCategories = article.getCategories();
        assertEquals(null, resultCategories);

        // Now set the categories ...
        article.setCategories(new ArrayList());
        resultCategories = article.getCategories();
        assertEquals(expectedCategories, resultCategories);

        // And repeat with some actual values ...
        expectedCategories.add("test");
        expectedCategories.add("test");
        article.setCategories(expectedCategories);
        resultCategories = article.getCategories();
        assertEquals(expectedCategories, resultCategories);

        // Now test the constructor with a PAGE article 
        // Expect null categories because we throw these ones away
        List<String> inputList = new ArrayList<>();
        inputList.add(PAGE_ARTICLE);
        article = new Article(inputList);
        resultCategories = article.getCategories();
        assertEquals(null, resultCategories);

        // Now test the contstructor with a POST article
        // Expect the categories we have on the article tag
        inputList = new ArrayList<>();
        inputList.add(POST_ARTICLE);
        article = new Article(inputList);
        expectedCategories = new ArrayList<>();
        expectedCategories.add("restful-web-services");
        expectedCategories.add("web");
        resultCategories = article.getCategories();
        assertEquals(expectedCategories, resultCategories);

    }

I think strictly speaking, each of these tests should be in a separate method in the ArticleTest class. The disadvantage to the above code is that JUnit stops the test once it hits a failure, so if for instance the test for the constructor on the PAGE_ARTICLE fails, it won’t even run the following tests.

At any rate, running the tests again gives me a success. But I’m still not doing anything with these categories: my output file is the same, and those tests are still looking for the hardwired values.

So now I have to go through the unit test code and look for the hardcoded tag values, and update the methods to expect the values parsed from the input. I know that a couple of these tests are using input from a specific test file, so my first step is to look at that file and see what categories I should be expecting, which turns out to be “marketing” and “networking”.

I update all the methods that compare against the testInput to expect those values (leaving in the hard-coded “recovered” value):

                + "<category domain=\"category\" nicename=\"marketing\"><![CDATA[marketing]]></category>"
                + "<category domain=\"category\" nicename=\"networking\"><![CDATA[networking]]></category>"
                + "<category domain=\"category\" nicename=\"recovered\"><![CDATA[Recovered Post]]></category>"
                + "</item>";

And as expected the test fails:

So now it’s time to make that test succeed, so find the place in the code where that “recovered” tag gets spit out, and add some code to spit out the categories. This particular test is on the addItem method, which is the one that parses our input list of Strings, so there’s a little sleuthing to get to where the data actually gets built. Since I was building this as static strings, I named the method getFixedStatusStrings, which simply generates a String that gets appended after the article contents.

So even before I fix this, I refactor to name this something more meaningful: getPostMetaData, and then add a method to dump the categories in just before the hardwired one. After adding the new method and the call:

    /**
     * Returns a string with all of the "fixed" fields
     *
     * TODO: go through these and see if there are fields that can be gleaned
     * from the index.html
     *
     * @return
     */
    public String getPostMetaData() {
        StringBuilder sb = new StringBuilder();
        sb.append("<wp:status>publish</wp:status>");
        sb.append("<wp:post_parent>0</wp:post_parent>");
        sb.append("<wp:menu_order>0</wp:menu_order>");
        sb.append("<wp:post_type>post</wp:post_type>");
        sb.append("<wp:post_password></wp:post_password>");
        sb.append("<wp:is_sticky>0</wp:is_sticky>");
        sb.append(getPostMetaDataCategories());
        sb.append("<category domain=\"category\" nicename=\"recovered\"><![CDATA[Recovered Post]]></category>");
        sb.append("<category domain=\"post_tag\" nicename=\"recovered\"><![CDATA[recovered]]></category>");
        return sb.toString();
    }

    /**
     * Format the categories into the appropriate meta data strings
     * 
     * @return 
     */
    private String getPostMetaDataCategories() {
        StringBuilder sb = new StringBuilder();
        for(String s : categories){
            sb.append("<category domain=\"category\" nicename=\"");
            sb.append(s);
            sb.append("\"><![CDATA[");
            sb.append(s);
            sb.append("]]></category>");
        }
        return sb.toString();
    }

Run the tests again, and no surprise a couple more tests fail, which is the beauty of TDD: we didn’t have to figure out that our changes impacted other methods, only that they need to be updated.

The testGetContent and testGetFixedStatusString methods are throwing NullPointerExceptions in the new method, which means we didn’t code defensively enough on that method. Since we built the code to allow the categories field to be null, we have to add a check for that and now all our tests succeed.

Ultimately, TDD saves you time by improving the quality of your code. It helps you by making you think a bit longer about what it is you’re trying to accomplish, and helps you to find problems caused by making changes in your code.

Just like anything else, it takes time to feel comfortable with the idea, and just like any other coding practice, use your best judgement on how granular you want to go with it.