Unit Tests and Transactions

A question came up on StackOverflow recently that got me to thinking about how to unit test transaction-dependent behavior.

Generally we start with data access tests that are transactional so that each test can roll back when it's complete. This way each test can modify an in-memory database without affecting subsequent tests. 

Let's say we are tasked with writing code for a common workflow in our application that loads, updates, and saves an entity. We might write a test like this that hits a live in-memory database:

// everything is in one transaction
@Transactional
public void test() {
  String newValue = "somethingnew";
  Entity t1 = dao.getById(id);
  t1.setProperty(newValue);
  dao.update(t1);
  Entity t2 = dao.getById(id);
  assertEquals(newValue, t2.getProperty());
}

We then write code to make the test pass, and move on believing that All Is Good. But all is Not Quite Good.

Imagine that something is wrong with our dao.update() method and it actually does not update the database.

1. In that case, the object t1 is stored in the current database session
2. Retrieving t2 by id actually returns t1 by virtue of being called in the same transaction
3. The test compares t1 with its own property, instead of comparing the new value with the persisted value
4. The test passes but should have failed.

One way around this is to remove the transactional boundaries from the test method and allow the methods we're testing to begin/commit their own transactions (which they will normally be doing anyway). This way if the update method does not update the entity, the bug will quickly be exposed and All Is Good again.

// test method is not @Transactional

// DAO methods are themselves @Transactional

public void test() {

  String newValue = "somethingnew";

  Entity t1 = dao.getById(id);    // transaction 1

  t1.setProperty(newValue);

  dao.update(t1);                 // transaction 2

  Entity t2 = dao.getById(id);    // transaction 3

  assertEquals(newValue, t2.getProperty());

}

One thing to be aware of with the second test is that if we are committing against an in-memory database like HSQLDB, our changes made in one test will not be automatically rolled back, and will be visible to subsequent tests. We will have to manually undo all of the changes at the end of the test, or ensure that the test data does not interfere with other tests which is not so easy.

In summary: when would we use one kind of test vs the other? In general I think the benefits outweigh the risks for rolling back after each test, so that is usually my starting point. Testing a workflow that crosses multiple transactions is really a job for an integration test and can easily be done with Selenium or SoapUI. So I would favor a full integration test to test that kind of behavior, and use a multi-transactional unit test to experiment with a new sequence of calls or debug a certain scenario quickly and without the overhead of a full integration test. 

What do you think? Do you favor multiple or single transaction tests? Have you seen the transaction test bug described here?