Testing with tempfiles and environment variables

There are many applications that need access to files on the filesystem or to a database. In order to have clear separation between the test runs I create a temporary folder (or a temporary database) and then I need to pass this to the test somehow. While programming in Perl or Python I used to do this by setting an environment variable in the test and then reading it in the application itself.

Sometimes this environment variable would contain a path to a configuration file that is then used by the application.

This patters is somewhat broken when trying use it in Rust. The reason is that Rust by default runs the tests in parallel in threads and environment variables are per process. Thus if I set two different temporary paths in two test-files one will override the other. Then depending on the timing one test might look at the temporary file of the other test.

Here is an example. I tried to simplify it, but it still feels a bit convoluted.

The dependencies

examples/test-tempfile/Cargo.toml

[package]
name = "test-tempfile"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
rand = "0.8.5"
tempfile = "3.9.0"

The code with the tests

examples/test-tempfile/src/main.rs

use std::fs::File;
use std::io::Write;

fn main() {
    let args = std::env::args().collect::<Vec<String>>();
    let name = &args[1];
    println!("Hello {name}");
}

fn add(x: i32, y: i32) -> i32 {
    let time: u64 = rand::random();
    let time = time % 3;

    std::thread::sleep(std::time::Duration::from_secs(time));

    match std::env::var("RESULT_PATH") {
        Ok(file_path) => {
            let mut file = File::create(&file_path).unwrap();
            println!("add({x}, {y}) file {file_path}");
            writeln!(&mut file, "{}", x + y).unwrap();
        }
        Err(_) => {}
    };
    x + y
}

#[test]
fn test_add_2_3_5() {
    let tmp_dir = tempfile::tempdir().unwrap();
    println!("tmp_dir: {:?}", tmp_dir);
    let file_path = tmp_dir.path().join("result.txt");
    println!("2+3 - file_path {:?}", file_path);
    std::env::set_var("RESULT_PATH", &file_path);

    let result = add(2, 3);
    assert_eq!(result, 5);

    let result = std::fs::read_to_string(file_path).unwrap();
    assert_eq!(result, "5\n");
}

#[test]
fn test_add_4_4_8() {
    let tmp_dir = tempfile::tempdir().unwrap();
    println!("tmp_dir: {:?}", tmp_dir);
    let file_path = tmp_dir.path().join("result.txt");
    println!("4+4 - file_path {:?}", file_path);
    std::env::set_var("RESULT_PATH", &file_path);

    let result = add(4, 4);
    assert_eq!(result, 8);

    let result = std::fs::read_to_string(file_path).unwrap();
    assert_eq!(result, "8\n");
}

Running the tests

Running the tests in the usual way will randomly break as the environment variable get mixed between the two test-cases.

cargo test

The solution, for now, is to run the tests in a single thread:

cargo test -- --test-threads=1

Unfortunately people new to the project will likely try to run the default command and the test will fail for them.

We could mention this in the README file, but can we really rely on people reading the documentation?

My question on the users forum got a suggestion to include the setting in the cargo/config.toml file:

examples/test-tempfile/.cargo/config.toml

[alias]
t = "test -- --test-threads=1"

[env]
RUST_TEST_THREADS = "1"

Actually I include two suggestions.

Alias

The first one was to create an alias for the command. This way I can run

cargo t

and it will run the tests with the --test-threads=1. This is convenient, but people will still run cargo test and complain about the failures.

Set the RUST_TEST_THREADS env variable

The second advice was to set the RUST_TEST_THREADS environment variable in the .cargo/config.toml file.

That worked. Now if I run

cargo test

It will run in a single thread.

Future

I am not entirely happy with this whole solution as it means I needed some special code in my application and I can't run the tests in a thread. I asked about this and I'll update this post when I have a better solution.