Recipe 15.6. Testing Boundary and Error Conditions

Problem

You are writing utility templates to be used by others, and you want them to be robust.

Solution

Boundary-condition testing

In all programming languages, bugs most often appear at boundary conditions. Thus, you should choose test data in which values lie along data extremes. Boundary values include maximum, minimum, and just inside/outside boundaries. If your templates work correctly for these special values, then they will probably work correctly for all other values. It is impossible to provide an exhaustive list of boundary conditions because they vary from problem to problem. Next is a list of typical cases you should consider.

If a template acts on node sets (sequences in 2.0), then be sure to test the following cases:

• An empty node set

• A node set with one element

• A node set with two elements

• A node set with an odd number of elements other than 1

• A node set with an even number of elements other than 2

If a template acts on a string, be sure to test the following cases:

• The empty string

• A string of length 1

• Other strings of varying sizes

If your template uses substring-before or substring-after for searches, be sure to test the following cases:

• Strings that do not contain the test string

• Strings that start with the search string

• Strings that end with the search string

• Strings that contain only the search string

• Strings that contain multiple occurrences of the search string (back to back and separated by other text)

If a template acts on numbers, be sure to test:

• The number 0

• The number 1

• Negative numbers

• Fractional numbers (0 < x < 1)

• Numbers with whole and fractional parts

• Prime numbers

• Other special boundary numbers that are unique to your problem

If a template compares two numbers X and Y, be sure to test cases in which:

• X < Y, especially X = Y - 1 and X = Y - d, where d is a small fraction

• X = Y

• X > Y, especially X = Y + 1 and X = Y + d, where d is a small fraction

When you know or have access to the schema of a document that a stylesheet will process, be sure to test inputs where:

• Optional elements are absent

• Optional elements are present

• Unbounded elements have only one instance

• Unbounded elements have several instances

Error-condition testing

A robust reusable template or stylesheet should fail gracefully in the face of erroneous input. Here you often use xsl:message with terminate=yes to report illegal parameter values.

If a template acts on numbers, be sure to test error handling for:

• NaN (0 div 0)

• Infinity ( 1 div 0)

• -Infinity (-1 div 0)

• Zero when undefined (e.g., logarithms)

• Negative numbers when undefined (e.g., for factorial)

• Non-numeric input (e.g., "foo")

When templates or stylesheets use parameters, be sure to test what happens when:

• Parameters without default values are not set

• Parameters receive out-of-bound values

• Parameters receive values of the wrong type

You can check for parameters that aren't set by using the following trick:

<xsl:param name="param1">
   <xsl:message terminate="yes">
     $param1 has not been set.
   </xsl:message>
</xsl:param>

However, this trick is not guaranteed to work because nothing in the standard setup states the value of a parameter not evaluated if a value is passed for that parameter. However, most XSLT processors are friendly to this technique. If you wanted to be absolutely safe, you could set the value to an illegal value (such as 1 div 0) and test for it in the body of the template:

<xsl:param name="param1" select="1 div 0" />
   
  <xsl:if test="$param1 = 1 div 0">
    <xsl:message terminate="yes">
      $param1 has not been set, or has been set to Infinity, which is
      invalid.
    </xsl:message>
  </xsl:if>

When you know or have access to the schema of a document a stylesheet is expected to process, see how the stylesheet responds to documents that:^[1]

^[1] This test assumes that the XSLT processor uses a nonvalidating parser or that you remove the schema reference from the input document.

• Completely violate the schema (e.g., an unrelated XML document as input)

• Contain some elements that violate the schema

• Violate minOccurs and maxOccurs constraints

• Violate data type constraints

Discussion

When developing XSLT for your own consumption, you are free to choose just how robust you want the code to be. However, when others use your code, it is a good practice to include reasonable error handling. Your clients will also thank you for delivering code that works for legal but unusual input.

When you create templates that use recursion, dividing the implementation into two templates is a good idea. The main template does the error checking and, if no errors are detected, calls an implementation template that computes the result recursively. Recipe Recipe 3.5 used this tactic for logarithms.