Skip to content

Commit

Permalink
chore: make failure report's scope & format configurable (#192)
Browse files Browse the repository at this point in the history 
* pr-fix: correct merge w/ 'main'

* chore: place diff horizontally

* chore: make report format and scope configurable

* pr-fix: remove unnecessary using

* pr-fix: correct merge w/ main

* pr-fix: more clear description of original row retrieval
  • Loading branch information
@stijnmoreels
stijnmoreels authored Oct 28, 2024
1 parent 8d7d14b commit d412f85
Show file tree
Hide file tree
Showing 9 changed files with 661 additions and 64 deletions.
62 changes: 44 additions & 18 deletions docs/preview/02-Features/02-assertion.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,10 @@ AssertXml.Equal(expected, actual);
// AssertXml.Equal failure: expected and actual XML documents do not match
// Expected element tag name 'root' but was 'diff-root' at /root
//
// Expected:
// <root>
// ...
// </root>
//
// Actual:
// <diff-root>
// ...
// </diff-root>
// Expected: Actual:
// <root> <diff-root>
// ... ...
// </root> </diff-root>
```

💡 Currently, the input contents are trimmed in case the input is too big to be shown in a humanly readable manner to the test output. In case of large files, it might be best to log those files (or parts that interest you) separately before using this test assertion.
Expand All @@ -59,6 +54,18 @@ AssertXml.Equal(..., options =>
// Sets the maximum characters of the expected and actual inputs that should be written to the test output.
// Default: 500 characters.
options.MaxInputCharacters = 1000;

// Sets the format in which the different input documents will be shown in the failure report.
// Useful for documents that either expand in length or width to see the difference more clearly.
// Default: Horizontal
options.ReportFormat = ReportFormat.Vertical;

// Sets position in the input document that should be included in the failure report.
// Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
// Useful for bigger documents where it would be hard to see the difference in the full document.
// Or Complete to include the entire input document in the failure report.
// Default: Limited
options.ReportScope = ReportScope.Complete;
});
```

Expand Down Expand Up @@ -102,15 +109,10 @@ AssertJson.Equal(expected, actual);
// AssertJson.Equal failure: expected and actual JSON contents do not match
// Actual JSON misses property at $.root
//
// Expected:
// {
// "root": ...
// }
//
// Actual:
// {
// "diff-root": ...
// }
// Expected: Actual:
// { {
// "root": ... "diff-root": ...
// } {
```

💡 Currently, the input contents are trimmed in case the input is too big to be shown in a humanly readable manner to the test output. In case of large files, it might be best to log those files (or parts that interest you) separately before using this test assertion.
Expand All @@ -133,6 +135,18 @@ AssertJson.Equal(..., options =>
// Sets the maximum characters of the expected and actual inputs that should be written to the test output.
// Default: 500 characters.
options.MaxInputCharacters = 1000;

// Sets the format in which the different input documents will be shown in the failure report.
// Useful for documents that either expand in length or width to see the difference more clearly.
// Default: Horizontal
options.ReportFormat = ReportFormat.Vertical;

// Sets position in the input document that should be included in the failure report.
// Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
// Useful for bigger documents where it would be hard to see the difference in the full document.
// Or Complete to include the entire input document in the failure report.
// Default: Limited
options.ReportScope = ReportScope.Complete;
});
```

Expand Down Expand Up @@ -244,6 +258,18 @@ AssertCsv.Equal(..., options =>
// Sets the maximum characters of the expected and actual inputs should be written to the test output.
// Default: 500 characters.
options.MaxInputCharacters = 1000;

// Sets the format in which the different input documents will be shown in the failure report.
// Useful for documents that either expand in length or width to see the difference more clearly.
// Default: Vertical
options.ReportFormat = ReportFormat.Vertical;

// Sets position in the input document that should be included in the failure report.
// Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
// Useful for bigger documents where it would be hard to see the difference in the full document.
// Or Complete to include the entire input document in the failure report.
// Default: Limited
options.ReportScope = ReportScope.Complete;
});
```

Expand Down
2 changes: 1 addition & 1 deletion docs/versioned_docs/version-v1.1.0/02-Features/02-assertion.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -323,4 +323,4 @@ JsonNode json = AssertXslt.TransformToJson(transformer, ...);

> ❓ Why use the `AssertXslt.Load` to load something that is already available with `XslCompiledTransform.Load`?
The `AssertXslt.Load` is a special variant on the existing load functionality in such a way that it provides more descriptive information on the input file that ws trying to be parsed, plus by throwing an assertion message, it mes the test output more clear on what the problem was and where it happened.
The `AssertXslt.Load` is a special variant on the existing load functionality in such a way that it provides more descriptive information on the input file that ws trying to be parsed, plus by throwing an assertion message, it mes the test output more clear on what the problem was and where it happened.
53 changes: 51 additions & 2 deletions src/Arcus.Testing.Assert/AssertCsv.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,16 @@ public int MaxInputCharacters
_maxInputCharacters = value;
}
}

/// <summary>
/// Gets or sets the position in the input document that should be included in the failure report (default: <see cref="ReportScope.Limited"/>).
/// </summary>
public ReportScope ReportScope { get; set; } = ReportScope.Limited;

/// <summary>
/// Gets or sets the format in which the different input documents will be shown in the failure report (default: <see cref="ReportFormat.Vertical"/>).
/// </summary>
public ReportFormat ReportFormat { get; set; } = ReportFormat.Vertical;
}

/// <summary>
Expand Down Expand Up @@ -314,6 +324,9 @@ public static void Equal(CsvTable expected, CsvTable actual, Action<AssertCsvOpt

if (diff != null)
{
string expectedCsv = diff.ExpectedDiff != null && options.ReportScope is ReportScope.Limited ? diff.ExpectedDiff : expected.ToString();
string actualCsv = diff.ActualDiff != null && options.ReportScope is ReportScope.Limited ? diff.ActualDiff : actual.ToString();

string optionsDescription =
$"Options: {Environment.NewLine}" +
$"\t- ignored columns: [{string.Join($"{options.Separator} ", options.IgnoredColumns)}]{Environment.NewLine}" +
Expand All @@ -326,7 +339,12 @@ public static void Equal(CsvTable expected, CsvTable actual, Action<AssertCsvOpt
.AppendLine(diff.ToString())
.AppendLine()
.AppendLine(optionsDescription)
.AppendDiff(expected.ToString(), actual.ToString(), options.MaxInputCharacters)
.AppendDiff(expectedCsv, actualCsv, opt =>
{
opt.MaxInputCharacters = options.MaxInputCharacters;
opt.Format = options.ReportFormat;
opt.Scope = options.ReportScope;
})
.ToString());
}
}
Expand Down Expand Up @@ -501,7 +519,11 @@ private static CsvDifference CompareRows(CsvTable expectedCsv, CsvTable actualCs
{
return shouldIgnoreOrder
? new(ActualMissingRow, expectedRow, actualRow)
: new(ActualOtherValue, expectedCell, actualCell);
: new(ActualOtherValue, expectedCell, actualCell)
{
ExpectedDiff = expectedCsv.GetOriginalRowAtOrAll(expectedRow.RowNumber),
ActualDiff = actualCsv.GetOriginalRowAtOrAll(actualRow.RowNumber)
};
}
}
}
Expand All @@ -519,6 +541,9 @@ internal class CsvDifference
private readonly string _expected, _actual, _column;
private readonly int _rowNumber;

internal string ExpectedDiff { get; init; }
internal string ActualDiff { get; init; }

internal CsvDifference(CsvDifferenceKind kind, CsvRow expected, CsvRow actual)
: this(kind, expected.ToString(), actual.ToString(), expected.RowNumber)
{
Expand Down Expand Up @@ -635,6 +660,8 @@ protected CsvTable(string[] headerNames, CsvRow[] rows, string originalCsv, Asse

internal AssertCsvHeader Header => _options.Header;

internal string HeaderNamesTxt => string.Join(_options.Separator, HeaderNames);

/// <summary>
/// Gets the names of the headers of the first row in the table.
/// </summary>
Expand Down Expand Up @@ -793,6 +820,28 @@ private static void EnsureAllRowsSameLength(string csv, string[][] rawRows, Asse
}
}

/// <summary>
/// Gets the original CSV row at a given <paramref name="index"/>,
/// or the entire table when no such row could be found.
/// </summary>
/// <remarks>
/// Safely implemented to fallback on the original CSV when the row cannot be found.
/// </remarks>
internal string GetOriginalRowAtOrAll(int index)
{
string[] rows = _originalCsv.Split(_options.NewLine, StringSplitOptions.RemoveEmptyEntries);
int i = _options.Header is AssertCsvHeader.Present ? index + 1 : index;

string row = rows.ElementAtOrDefault(i);
if (row is null)
{
return _originalCsv;
}

string headerLine = string.Join(_options.Separator, HeaderNames);
return headerLine + _options.NewLine + row;
}

/// <summary>
/// Returns a string that represents the current object.
/// </summary>
Expand Down
56 changes: 48 additions & 8 deletions src/Arcus.Testing.Assert/AssertJson.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
using System.Linq;
using System.Text.Json;
using System.Text.Json.Nodes;
using Arcus.Testing.Failure;
using static Arcus.Testing.JsonDifferenceKind;

namespace Arcus.Testing
Expand Down Expand Up @@ -90,6 +89,16 @@ public int MaxInputCharacters
_maxInputCharacters = value;
}
}

/// <summary>
/// Gets or sets the position in the input document that should be included in the failure report (default: <see cref="ReportScope.Limited"/>).
/// </summary>
public ReportScope ReportScope { get; set; } = ReportScope.Limited;

/// <summary>
/// Gets or sets the format in which the different input documents will be shown in the failure report (default: <see cref="ReportFormat.Horizontal"/>).
/// </summary>
public ReportFormat ReportFormat { get; set; } = ReportFormat.Horizontal;
}

/// <summary>
Expand Down Expand Up @@ -157,8 +166,8 @@ public static void Equal(JsonNode expected, JsonNode actual, Action<AssertJsonOp
JsonDifference diff = CompareJsonRoot(expected, actual, options);
if (diff != null)
{
string expectedJson = expected?.ToString() ?? "null";
string actualJson = actual?.ToString() ?? "null";
string expectedJson = diff.ExpectedNodeDiff != null && options.ReportScope is ReportScope.Limited ? diff.ExpectedNodeDiff : expected?.ToString() ?? "null";
string actualJson = diff.ActualNodeDiff != null && options.ReportScope is ReportScope.Limited ? diff.ActualNodeDiff : actual?.ToString() ?? "null";

string optionsDescription =
$"Options: {Environment.NewLine}" +
Expand All @@ -170,7 +179,12 @@ public static void Equal(JsonNode expected, JsonNode actual, Action<AssertJsonOp
.AppendLine(diff.ToString())
.AppendLine()
.AppendLine(optionsDescription)
.AppendDiff(expectedJson, actualJson, options.MaxInputCharacters)
.AppendDiff(expectedJson, actualJson, opt =>
{
opt.MaxInputCharacters = options.MaxInputCharacters;
opt.Format = options.ReportFormat;
opt.Scope = options.ReportScope;
})
.ToString());
}
}
Expand Down Expand Up @@ -219,7 +233,11 @@ private static JsonDifference CompareJsonArray(JsonNode expected, JsonArray actu

if (actualChildren.Length != expectedChildren.Length)
{
return new(DifferentLength, expectedArray.GetPath(), actualChildren.Length, expectedChildren.Length);
return new(DifferentLength, expectedArray.GetPath(), actualChildren.Length, expectedChildren.Length)
{
ExpectedNodeDiff = expectedArray.ToString(),
ActualNodeDiff = actualArray.ToString(),
};
}

if (options.Order is AssertJsonOrder.Ignore)
Expand Down Expand Up @@ -263,14 +281,27 @@ private static JsonDifference CompareJsonObject(JsonNode expected, JsonObject ac

private static JsonDifference CompareJsonObject(Dictionary<string, JsonNode> expectedDir, Dictionary<string, JsonNode> actualDir, AssertJsonOptions options)
{
var serializerOptions = new JsonSerializerOptions()
{
WriteIndented = true
};

if (TryGetValue(expectedDir, key => !actualDir.ContainsKey(key), out JsonNode missingActualPair))
{
return new(ActualMissesProperty, missingActualPair.GetPath());
return new(ActualMissesProperty, missingActualPair.GetPath())
{
ExpectedNodeDiff = JsonSerializer.Serialize(expectedDir, serializerOptions),
ActualNodeDiff = JsonSerializer.Serialize(actualDir, serializerOptions)
};
}

if (TryGetValue(actualDir, key => !expectedDir.ContainsKey(key), out JsonNode missingExpectedPair))
{
return new(ExpectedMissesProperty, missingExpectedPair.GetPath());
return new(ExpectedMissesProperty, missingExpectedPair.GetPath())
{
ExpectedNodeDiff = JsonSerializer.Serialize(expectedDir, serializerOptions),
ActualNodeDiff = JsonSerializer.Serialize(actualDir, serializerOptions)
};
}

foreach (KeyValuePair<string, JsonNode> expectedPair in expectedDir)
Expand Down Expand Up @@ -371,7 +402,11 @@ or JsonValueKind.False
#endif
if (!identical)
{
return new(ActualOtherValue, expectedValue, actualValue);
return new(ActualOtherValue, expectedValue, actualValue)
{
ExpectedNodeDiff = expectedValue.Parent?.ToString(),
ActualNodeDiff = actualValue.Parent?.ToString()
};
}

return null;
Expand Down Expand Up @@ -429,9 +464,14 @@ internal class JsonDifference
private readonly string _path;
private readonly object _actual, _expected;

internal string ExpectedNodeDiff { get; init; }
internal string ActualNodeDiff { get; init; }

internal JsonDifference(JsonDifferenceKind kind, JsonNode expected, JsonNode actual)
: this(kind, expected?.GetPath() ?? actual?.GetPath() ?? "<not-available>", expected: Describe(expected), actual: Describe(actual))
{
ExpectedNodeDiff = expected?.ToString();
ActualNodeDiff = actual?.ToString();
}

internal JsonDifference(JsonDifferenceKind kind, string path, object actual, object expected)
Expand Down
Loading

0 comments on commit d412f85

Please sign in to comment.