Using assertions

ScalaTest makes three assertions available by default in any style trait. You can use:

• assert for general assertions;
• assertResult to differentiate expected from actual values;
• assertThrows to ensure a bit of code throws an expected exception.

To get moving quickly in ScalaTest, learn and use these three assertions. Later if you prefer you can switch to the more expressive matchers DSL.

ScalaTest's assertions are defined in trait Assertions, which is extended by Suite, the supertrait to all style traits. Trait Assertions also provides:

• assume to conditionally cancel a test;
• fail to fail a test unconditionally;
• cancel to cancel a test unconditionally;
• succeed to make a test succeed unconditionally;
• intercept to ensure a bit of code throws an expected exception and then make assertions about the exception;
• assertDoesNotCompile to ensure a bit of code does not compile;
• assertCompiles to ensure a bit of code does compile;
• assertTypeError to ensure a bit of code does not compile because of a type (not parse) error;
• withClue to add more information about a failure.

All of these constructs are described below.

The assert macro

In any Scala program, you can write assertions by invoking assert and passing in a Boolean expression, such as:

val left = 2
val right = 1
assert(left == right)

If the passed expression is true, assert will return normally. If false, Scala's assert will complete abruptly with an AssertionError. This behavior is provided by the assert method defined in object Predef, whose members are implicitly imported into every Scala source file. This Assertions trait defines another assert method that hides the one in Predef. It behaves the same, except that if false is passed it throws TestFailedException instead of AssertionError. Why? Because unlike AssertionError, TestFailedException carries information about exactly which item in the stack trace represents the line of test code that failed, which can help users more quickly find an offending line of code in a failing test. In addition, ScalaTest's assert provides better error messages than Scala's assert.

If you pass the previous Boolean expression, left == right to assert in a ScalaTest test, a failure will be reported that, because assert is implemented as a macro, includes reporting the left and right values. For example, given the same code as above but using ScalaTest assertions:

import org.scalatest.Assertions._
val left = 2
val right = 1
assert(left == right)

The detail message in the thrown TestFailedException from this assert will be: "2 did not equal 1".

ScalaTest's assert macro works by recognizing patterns in the AST of the expression passed to assert and, for a finite set of common expressions, giving an error message that an equivalent ScalaTest matcher expression would give. Here are some examples, where a is 1, b is 2, c is 3, d is 4, xs is List(a, b, c), and num is 1.0:

assert(a == b || c >= d)
// Error message: 1 did not equal 2, and 3 was not greater than or equal to 4

assert(xs.exists(_ == 4))
// Error message: List(1, 2, 3) did not contain 4

assert("hello".startsWith("h") && "goodbye".endsWith("y"))
// Error message: "hello" started with "h", but "goodbye" did not end with "y"

assert(num.isInstanceOf[Int])
// Error message: 1.0 was not instance of scala.Int

assert(Some(2).isEmpty)
// Error message: Some(2) was not empty

For expressions that are not recognized, the macro currently prints out a string representation of the (desugared) AST and adds "was false". Here are some examples of error messages for unrecognized expressions:

assert(None.isDefined)
// Error message: scala.None.isDefined was false

assert(xs.exists(i => i > 10))
// Error message: xs.exists(((i: Int) => i.>(10))) was false

You can augment the standard error message by providing a String as a second argument to assert, like this:

val attempted = 2
assert(attempted == 1, "Execution was attempted " + left + " times instead of 1 time")

Using this form of assert, the failure report will be more specific to your problem domain, thereby helping you debug the problem. This Assertions trait also mixes in the TripleEquals, which gives you a === operator that allows you to customize Equality, perform equality checks with numeric Tolerance, and enforce type constraints at compile time with sibling trait TypeCheckedTripleEquals.

Expected results

Although the assert macro provides a natural, readable extension to Scala's assert mechanism that provides good error messages, as the operands become lengthy, the code becomes less readable. In addition, the error messages generated for == and === comparisons don't distinguish between actual and expected values. The operands are just called left and right, because if one were named expected and the other actual, it would be difficult for people to remember which was which. To help with these limitations of assertions, Suite includes a method called assertResult that can be used as an alternative to assert. To use assertResult, you place the expected value in parentheses after assertResult, followed by curly braces containing code that should result in the expected value. For example:

val a = 5
val b = 2
assertResult(2) {
  a - b
}

In this case, the expected value is 2, and the code being tested is a - b. This assertion will fail, and the detail message in the TestFailedException will read, "Expected 2, but got 3."

Forcing failures

If you just need the test to fail, you can write:

fail()

Or, if you want the test to fail with a message, write:

fail("I've got a bad feeling about this.")

Achieving success

In async style tests, you must end your test body with either Future[Assertion] or Assertion. ScalaTest's assertions (including matcher expressions) have result type Assertion, so ending with an assertion will satisfy the compiler. If a test body or function body passed to Future.map does not end with type Assertion, however, you can fix the type error by placing succeed at the end of the test or function body:

succeed // Has type Assertion

Expected exceptions

Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit 3 style, like this:

val s = "hi"
try {
  s.charAt(-1)
  fail()
}
catch {
  case _: IndexOutOfBoundsException => // Expected, so continue
}

If charAt throws IndexOutOfBoundsException as expected, control will transfer to the catch case, which does nothing. If, however, charAt fails to throw an exception, the next statement, fail(), will be run. The fail method always completes abruptly with a TestFailedException, thereby signaling a failed test.

To make this common use case easier to express and read, ScalaTest provides two methods: assertThrows and intercept. Here's how you use assertThrows:

val s = "hi"
assertThrows[IndexOutOfBoundsException] { // Result type: Assertion
  s.charAt(-1)
}

This code behaves much like the previous example. If charAt throws an instance of IndexOutOfBoundsException, assertThrows will return Succeeded. But if charAt completes normally, or throws a different exception, assertThrows will complete abruptly with a TestFailedException.

The intercept method behaves the same as assertThrows, except that instead of returning Succeeded, intercept returns the caught exception so that you can inspect it further if you wish. For example, you may need to ensure that data contained inside the exception have expected values. Here's an example:

val s = "hi"
val caught =
  intercept[IndexOutOfBoundsException] { // Result type: IndexOutOfBoundsException
    s.charAt(-1)
  }
assert(caught.getMessage.indexOf("-1") != -1)

Checking that a snippet of code does or does not compile

Often when creating libraries you may wish to ensure that certain arrangements of code that represent potential “user errors” do not compile, so that your library is more error resistant. ScalaTest's Assertions trait includes the following syntax for that purpose:

assertDoesNotCompile("val a: String = 1")

If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:

assertTypeError("val a: String = 1")

Note that the assertTypeError call will only succeed if the given snippet of code does not compile because of a type error. A syntax error will still result on a thrown TestFailedException.

If you want to state that a snippet of code does compile, you can make that more obvious with:

assertCompiles("val a: Int = 1")

Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.

Assumptions

Trait Assertions also provides methods that allow you to cancel a test. You would cancel a test if a resource required by the test was unavailable. For example, if a test requires an external database to be online, and it isn't, the test could be canceled to indicate it was unable to run because of the missing database. Such a test assumes a database is available, and you can use the assume method to indicate this at the beginning of the test, like this:

assume(database.isAvailable)

For each overloaded assert method, trait Assertions provides an overloaded assume method with an identical signature and behavior, except the assume methods throw TestCanceledException whereas the assert methods throw TestFailedException. As with assert, assume hides a Scala method in Predef that performs a similar function, but throws AssertionError. And just as you can with assert, you will get an error message extracted by a macro from the AST passed to assume, and can optionally provide a clue string to augment this error message. Here are some examples:

assume(database.isAvailable, "The database was down again")
assume(database.getAllUsers.count === 9)

Forcing cancelations

For each overloaded fail method, there's a corresponding cancel method with an identical signature and behavior, except the cancel methods throw TestCanceledException whereas the fail methods throw TestFailedException. Thus if you just need to cancel a test, you can write:

cancel()

If you want to cancel the test with a message, just place the message in the parentheses:

cancel("Can't run the test because no internet connection was found")

Getting a clue

If you want more information than is provided by default by the methods of this trait, you can supply a "clue" string in one of several ways. The extra information (or "clues") you provide will be included in the detail message of the thrown exception. Both assert and assertResult provide a way for a clue to be included directly, intercept does not. Here's an example of clues provided directly in assert:

assert(1 + 1 === 3, "this is a clue")

and in assertResult:

assertResult(3, "this is a clue") { 1 + 1 }

The exceptions thrown by the previous two statements will include the clue string, "this is a clue", in the exception's detail message. To get the same clue in the detail message of an exception thrown by a failed assertThrows call requires using withClue:

withClue("this is a clue") {
  assertThrows[IndexOutOfBoundsException] {
    "hi".charAt(-1)
  }
}

The withClue method will only prepend the clue string to the detail message of exception types that mix in the ModifiableMessage trait. See the documentation for ModifiableMessage for more information. If you wish to place a clue string after a block of code, see the documentation for AppendedClues.

Next, learn about tagging your tests.