Test Attribute #9 – Deterministic

I keep hammering on trust and how it’s crucial that we trust our tests. If a test is deterministic, it raises the level of our trust.  If it isn’t, we may question its result, which will be followed by questioning other tests as well.

Let’s start with a simple example. What’s wrong with this picture?
 
 
 
 
 
 

public class CreditCard
{
    DateTime expirationDate = new DateTime(2017,12,15);
    public bool IsExpired()
    {
        return (DateTime.Now > expirationDate);
    }
}

This is the tested code. So we can write this test:

[TestMethod]public void IsExpired_Today_False()
{
    var card = new CreditCard();
    Assert.IsFalse(card.IsExpired());
}

Which passes. Until a specific day arrives, and then it fails. Or this code:

public class Settings
{
    public string GetFirstValue()
    {
        var lines = File.ReadLines("config.sys");
        return lines.First();
    }
}

And its test:

[TestMethod]public void FirstLine_OfConfigSys_ContainsDevice()
{
    var settings = new Settings();
    var result = settings.GetFirstValue().Contains("Device");
    Assert.IsTrue(result);
    
}

That passes, as long if finds the file where it expects it, or somebody edits it “the wrong way”. These may look like dependencies we can’t trust all the time. It’s more deep than that.

Never Assume

keep-calm-and-assume-nothing-64When we write code, we have many assumptions. The “happy path” we usually code first, when we assume nothing will go wrong. The other paths are the the paths where something might go wrong. Unfortunately, we can test and code only the things we think of. Actually, writing helps us think of other cases. The downside, is that if we use a library we didn’t write, we think less of the side effects.

To make sure we remove uncertainty from our tests, we need to remove being dependent on:

  • Geographical location
  • Time
  • Hardware speed, CPU, memory
  • Files and data, both existing and missing

The obvious solution is to mock the dependencies, or setup the system to be independent. By mocking the date we can check the “before” and “after” cases, or “plant” the file there and simulate reading it.

However, mocking comes with its own drawbacks, so you should consider the tradeoff.

And sometimes, mocking isn’t an option. We had a performance test, that was supposed to run under a certain limit. It started failing and passing on and off. Instead of checking the problem we said: “Oh, leave it, this test is just like that”. And we missed opportunities for fixing real problems.

To handle the root cause – assumptions, we’ll go back to our friends. Review the tests and code, and try to either validate the assumptions, or invalidate them (in that case, remove the code and test). Unverified assumptions may cause not only bugs. The code you’ve added will make it harder to add new code in the future. Use code that not only works, but that is also valid.

Next time: Isolation.

Reference: Test Attribute #9 – Deterministic from our JCG partner Gil Zilberfeld at the Geek Out of Water blog.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

JPA Mini Book

Learn how to leverage the power of JPA in order to create robust and flexible Java applications. With this Mini Book, you will get introduced to JPA and smoothly transition to more advanced concepts.

JVM Troubleshooting Guide

The Java virtual machine is really the foundation of any Java EE platform. Learn how to master it with this advanced guide!

Given email address is already subscribed, thank you!
Oops. Something went wrong. Please try again later.
Please provide a valid email address.
Thank you, your sign-up request was successful! Please check your e-mail inbox.
Please complete the CAPTCHA.
Please fill in the required fields.

Leave a Reply


6 − three =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close