Testing in Rust

Towards fearless development

Alex Povel

June 5, 2024

You will get

  • an introduction to more advanced approaches & tools, to
    • …uncover bugs the type system can’t
    • …keep documentation in sync
    • …building more robust software with more confidence, and stronger resistance to regression

$ whoami

  • started Rust late 2022
  • implemented srgn & b4s for practice
  • today, also use Rust at my work at Cloudflare

Exploring testing

  • Why?
  • When?
  • How?
  • Demos 💥

Topics

  • Unit tests ↗
  • Integration tests ↗
  • Doc tests ↗
  • Doc tests for README.md
  • Fuzzing
  • Property testing
  • Snapshot testing

Extra?

  • CI setup ↗, with…
    • multi-OS tests
    • test coverage reports, by category ↗
    • feature powerset tests ↗
    • MSRV checks ↗
    • binary artifact publishing e2e test ↗

README doc tests

  • your users’ first, often only point of contact
  • determines your interface
  • Rust doc tests are specific to Rust code
    • not applicable to binary artifacts
    • do not extend to README.md

When?

  • you have a binary crate, aka a user-facing program
  • DTDD?

How?

  • write custom code ↗

    • minimal bash interpreter written with nom
    • exercises code snippets in README.md
    • uses actual program binary

  • hack for libraries: include in source code ↗

    #![doc = include_str!("../README.md")]

Fuzzing

  • world out there is
    • 👽 scary
    • 🤡 arbitrary
  • cannot think of every scenario

When?

  • untrusted, arbitrary user input
  • avoid crashing

    • handle any input gracefully

      let s = String::from_utf8(user_input).unwrap();

    • cannot compare results

      assert_eq!(result, expected);
//                 ^^^^^^^^
//                    🤷

How?

  • cargo-fuzz ↗
    • based on libFuzzer ↗
  • source of random-ish bytes
  • mold into valid structure
  • observe

Property

  • fuzzing’s cousin
  • like contracts: ensure property holds
    • testing all sort(input) permutations is impossible
    • asserting that for any input, results are sorted is trivial
  • more abstract than regular tests, more guided than fuzzing

When?

  • your system has observable properties which should hold
  • cannot unit test the world

How?

  • proptest ↗
    • based on Python’s Hypothesis, Haskell’s QuickCheck

Demo

🧑‍💻

let mut ranges = vec![0..2, 5..10, 40..50];
ranges.push(1..7); // Overlaps
ranges.merge(); // Idempotent!
assert_eq!(ranges, vec![0..10, 40..50]);

Snapshot

  • not a technique in itself
  • enhance unit and integration tests on complex data
  • fast reference data creation and review

When?

  • you have complex, nested or large data structure to craft
  • which are perhaps additionally subject to change

How?

Demo

🧑‍💻

Thanks!

Thank you very much for your attention!

github.com/alexpovel/rust-testing-explored

Further Reading