Settings

TestingUtilities makes use of the Preferences package to store package-side settings for various test failure scenarios.

Comparison Tests

Strings

When performing an equality comparison test between two String values, say x and y, if the test fails and the provided io object supports printing colours, the matching shared prefix of x and y will be rendered in green while the differing components of x and y will be rendered in red.

a = "abcd"
b = "abef"
c = "abeghik"
@testset NoThrowTestSet "" begin
    @Test a == b
    @Test isequal(c, "abeg")
end

Test `a == b` failed:
Values:
a = "abcd"
b = "abef"
Test `isequal(c, "abeg")` failed:
Values:
expected = "abeg"
c        = "abeghik"

If you're unable to distinguish between the default colours (or colours more generally), you can set the styles used to render the matching components and differing components of the strings by invoking TestingUtilities.set_show_diff_styles. You can use any key => value pair corresponding to the keyword arguments of Base.printstyled.

TestingUtilities.set_show_diff_styles(; matching=:bold => true, differing=:underline => true)
@testset NoThrowTestSet "" begin
    @Test a == b
    @Test isequal(c, "abeg")
end

Test `a == b` failed:
Values:
a = "abcd"
b = "abef"
Test `isequal(c, "abeg")` failed:
Values:
expected = "abeg"
c        = "abeghik"

DataFrames

Similarly, when performing an equality test between two DataFrames, TestUtilities will display a nicely-formatted message detailing the reason for the difference between the two values, e.g.,

using DataFrames

TestingUtilities.set_show_diff_styles(; matching=:color => :green, differing=:color => :red) # hide

x = DataFrame(:a => [1, 2, 4], :b => [false, true, true])
x_diffcols = DataFrame(:a => [1, 2, 4], :z => [false, false, true])
x_diffnrows = DataFrame(:a => [1, 2], :b => [false, true])
x_diffvalues = DataFrame(:a => [1, 2, 3], :b => [false, false, true])
@testset NoThrowTestSet "" begin
    @Test x == x_diffcols
    @Test x == x_diffnrows
    @Test x == x_diffvalues
end

Test `x == x_diffcols` failed:
Reason: `propertynames(x) != propertynames(x_diffcols)`
`propertynames(x)`          = {:a, :b}
`propertynames(x_diffcols)` = {:a, :z}
Test `x == x_diffnrows` failed:
Reason: `nrow(x) != nrow(x_diffnrows)`
`nrow(x)`           = 3
`nrow(x_diffnrows)` = 2
Test `x == x_diffvalues` failed:
Reason: Mismatched values
┌───────────────────┬──────────────┬───────┬───────┐
│           row_num │           df │     a │     b │
│ U{Nothing, Int64} │       Symbol │ Int64 │  Bool │
├───────────────────┼──────────────┼───────┼───────┤
│                 2 │            x │     2 │  true │
│                   │ x_diffvalues │     2 │ false │
│                 3 │            x │     4 │  true │
│                   │ x_diffvalues │     3 │  true │
└───────────────────┴──────────────┴───────┴───────┘