diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..7fc6a79 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,120 @@ +# TestBox Documentation Conventions + +This is the GitBook source for the official TestBox documentation, published at +. Each major version lives on its own branch (`v7.x`, +and so on); the version switcher in GitBook maps to those branches. + +## Engine Preference + +**BoxLang is the preferred engine.** When a feature exists on both BoxLang and CFML, +BoxLang comes first in every example, tab and list. Write BoxLang as the default voice +of the documentation and treat CFML as the companion, not the other way round. + +## Code Examples + +### Dual-engine features + +When a feature works on **both** BoxLang and CFML, wrap the example in a GitBook tab +group with **BoxLang first, CFML second**: + +````markdown +{% tabs %} +{% tab title="BoxLang" %} +{% code title="MyFirstSpec.bx" %} +```java +class extends="testbox.system.BaseSpec"{ + + function run(){ + describe( "My First Test", () => { + it( "can add", () => { + expect( sum( 1, 2 ) ).toBe( 3 ) + } ) + } ) + } + +} +``` +{% endcode %} +{% endtab %} + +{% tab title="CFML" %} +{% code title="MyTest.cfc" %} +```cfscript +component extends="testbox.system.BaseSpec"{ + + function run(){ + describe( "My First Test", function(){ + it( "can add", function(){ + expect( sum( 1, 2 ) ).toBe( 3 ); + } ); + } ); + } + +} +``` +{% endcode %} +{% endtab %} +{% endtabs %} +```` + +Where a page separates BDD from xUnit style, the four-tab form is +`BDD - BoxLang`, `xUnit - BoxLang`, `BDD - CFML`, `xUnit - CFML` — again BoxLang first. + +**Group by section, not by line.** One tabbed block covering a section's examples reads +far better than a tab group wrapped around every one-line snippet. A tab group whose two +sides differ only by a trailing semicolon is noise; fold those snippets into the nearest +substantive example instead. + +### Engine differences to honor + +| | BoxLang | CFML | +| --- | --- | --- | +| Fence language | ```` ```java ```` (or ```` ```groovy ```` for xUnit) | ```` ```cfscript ```` | +| File extension | `.bx` | `.cfc` | +| Class keyword | `class` | `component` | +| Statement terminator | no semicolons | semicolons | +| Closures | `() => {}` preferred | `function(){}` | + +### BoxLang-only features + +When a feature requires BoxLang and has no CFML equivalent, **do not use tabs** — there +is no second side. Mark it with a hint callout immediately after the heading, and write +the examples in BoxLang: + +```markdown +{% hint style="info" %} +These matchers require BoxLang. On CFML engines they are guarded and report unsupported +behavior cleanly. +{% endhint %} +``` + +Say what actually happens on CFML rather than only that the feature is unavailable — +guarded, ignored, or throwing a named exception such as +`TestBox.BoxLangFeatureNotAvailable`. + +Set expectations, Range expectations and the Data Navigator matchers are BoxLang-only. + +## Callouts + +Use GitBook hints, not plain `>` blockquotes: + +- `{% hint style="info" %}` — engine requirements, cross-references, context +- `{% hint style="warning" %}` — behavioral changes and upgrade notes +- `{% hint style="success" %}` — tips worth acting on + +Behavioral changes carry the version that introduced them, for example +**Changed in TestBox 7.1:**. + +## Structure + +- Register every new page in `SUMMARY.md`; an unregistered page will not appear in the book. +- Release notes live in `readme/release-history/whats-new-with-.md`, newest first + in `SUMMARY.md`, with a `description:` frontmatter key holding the release date. +- Add a short paragraph to `readme/release-history/README.md` for each minor release. +- Keep the existing frontmatter on a page you edit, including `metaLinks` and `icon`. + +## Before Committing + +- Every `SUMMARY.md` target resolves to a file that exists. +- Every relative `.md` link resolves from its own directory. +- New APIs appear in the reference pages people actually browse, not only in the release notes. diff --git a/SUMMARY.md b/SUMMARY.md index a1ce869..e9beed4 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -2,6 +2,7 @@ * [Introduction](README.md) * [Release History](readme/release-history/README.md) + * [What's New With 7.1.0](readme/release-history/whats-new-with-7.1.0.md) * [What's New With 7.0.0](readme/release-history/whats-new-with-7.0.0.md) * [About This Book](readme/about-this-book/README.md) * [Author](readme/about-this-book/author.md) @@ -64,6 +65,9 @@ * [Matchers](digging-deeper/expectations/matchers.md) * [Not Operator](digging-deeper/expectations/not-operator.md) * [Expecting Exceptions](digging-deeper/expectations/expecting-exceptions.md) + * [Set Expectations](digging-deeper/expectations/set-expectations.md) + * [Range Expectations](digging-deeper/expectations/range-expectations.md) + * [Data Navigator Expectations](digging-deeper/expectations/data-navigator.md) * [Custom Matchers](digging-deeper/expectations/custom-matchers.md) * [Output Utilities](digging-deeper/output-utilities.md) * [Runner Listeners](digging-deeper/run-listeners.md) diff --git a/digging-deeper/assertions/README.md b/digging-deeper/assertions/README.md index 67468d7..32989f3 100644 --- a/digging-deeper/assertions/README.md +++ b/digging-deeper/assertions/README.md @@ -32,18 +32,23 @@ assertCloseTo() Here are some common assertion methods: ```javascript +all( closures, [heading] ) assert( expression, [message] ) between( actual, min, max, [message] ) closeTo(expected, actual, delta, [datePart], [message]) deepKey( target, key, [message] ) fail( [message] ) includes( target, needle, [message] ) +includesAll( target, needles, [message] ) +includesAny( target, needles, [message] ) +includesNone( target, needles, [message] ) includesWithCase( target, needle, [message] ) instanceOf( actual, typeName, [message] ) isEmpty( target, [message] ) isEqual(expected, actual, [message]) isEqualWithCase(expected, actual, [message]) isFalse( actual, [message] ) +isFalsy( actual, [message] ) isGT( actual, target, [message]) isGTE( actual, target, [message]) isLT( actual, target, [message]) @@ -51,6 +56,7 @@ isLTE( actual, target, [message]) isNotEmpty( target, [message] ) isNotEqual(expected, actual, [message]) isTrue( actual, [message] ) +isTruthy( actual, [message] ) key( target, key, [message] ) lengthOf( target, length, [message] ) match( actual, regex, [message] ) @@ -70,3 +76,115 @@ skip( message, detail ) throws(target, [type], [regex], [message]) typeOf( type, actual, [message] ) ``` + +### Truthiness + +`isTrue()` and `isFalse()` require an actual boolean. `isTruthy()` and `isFalsy()` are looser, and are useful when the value under test is "something or nothing" rather than a strict boolean. + +{% tabs %} +{% tab title="BoxLang" %} +```java +$assert.isTruthy( "hello" ) +$assert.isTruthy( [ 1, 2 ] ) + +$assert.isFalsy( "" ) +$assert.isFalsy( 0 ) +$assert.isFalsy( [] ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +$assert.isTruthy( "hello" ); +$assert.isTruthy( [ 1, 2 ] ); + +$assert.isFalsy( "" ); +$assert.isFalsy( 0 ); +$assert.isFalsy( [] ); +``` +{% endtab %} +{% endtabs %} + +### Multiple Inclusions + +`includes()` checks for one needle. These three check for several at once: + +{% tabs %} +{% tab title="BoxLang" %} +```java +$assert.includesAll( roles, [ "admin", "editor" ] ) +$assert.includesAny( roles, [ "admin", "superuser" ] ) +$assert.includesNone( serializedUser, [ "password", "salt", "apiToken" ] ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +$assert.includesAll( roles, [ "admin", "editor" ] ); +$assert.includesAny( roles, [ "admin", "superuser" ] ); +$assert.includesNone( serializedUser, [ "password", "salt", "apiToken" ] ); +``` +{% endtab %} +{% endtabs %} + +### Grouped Assertions + +By default a failing assertion aborts the test, so you only ever see the first failure and fix them one run at a time. `$assert.all()` runs a set of assertion closures and reports **every** failure at once. + +{% tabs %} +{% tab title="BoxLang" %} +```java +$assert.all( [ + () => $assert.isEqual( "Luis", user.getName() ), + () => $assert.isEqual( "luis@ortussolutions.com", user.getEmail() ), + () => $assert.isTrue( user.isActive() ) +], "user profile" ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +$assert.all( [ + function(){ return $assert.isEqual( "Luis", user.getName() ); }, + function(){ return $assert.isEqual( "luis@ortussolutions.com", user.getEmail() ); }, + function(){ return $assert.isTrue( user.isActive() ); } +], "user profile" ); +``` +{% endtab %} +{% endtabs %} + +If the name and the active flag are both wrong, both are reported: + +``` +user profile: 1 of 3 assertions passed + [1] expected [Luis] but received [Alice] + [3] expected [true] but received [false] +``` + +The optional second argument is a heading prepended to the failure summary. + +`assertAll()` is available as a spec-level shortcut for the same thing: + +{% tabs %} +{% tab title="BoxLang" %} +```java +assertAll( [ + () => $assert.isEqual( 200, response.status ), + () => $assert.key( response, "data" ) +], "response envelope" ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +assertAll( [ + function(){ return $assert.isEqual( 200, response.status ); }, + function(){ return $assert.key( response, "data" ); } +], "response envelope" ); +``` +{% endtab %} +{% endtabs %} + +{% hint style="info" %} +Grouped assertions are the assertion-style counterpart to [collection expectations](../expectations/#collection-expectations). Reach for them when several independent facts about one object should all be reported together. +{% endhint %} diff --git a/digging-deeper/code-coverage/configuring-code-coverage.md b/digging-deeper/code-coverage/configuring-code-coverage.md index dc6be9d..9d22916 100644 --- a/digging-deeper/code-coverage/configuring-code-coverage.md +++ b/digging-deeper/code-coverage/configuring-code-coverage.md @@ -18,11 +18,11 @@ Most of the coverage settings are devoted to helping TestBox know what files to ## Default Settings -Code coverage is enabled by default and set with a default configuration. You can control how it behaves with a series of `` tags in your `/tests/runner.cfm` file. If you created a fresh new ColdBox app from our app templates using `coldbox create app`, you'll see there are already configuration options ready for you to change. If you are working with an existing test suite runner, place the following lines PRIOR to the `` in your runner.cfm. +Code coverage is **disabled by default** and set with a default configuration. You can control how it behaves with a series of `` tags in your `/tests/runner.cfm` file. If you created a fresh new ColdBox app from our app templates using `coldbox create app`, you'll see there are already configuration options ready for you to change. If you are working with an existing test suite runner, place the following lines PRIOR to the `` in your runner.cfm. ```markup - + @@ -36,14 +36,18 @@ Let's go over the options above and what they do. Feel free to comment/uncomment ## coverageEnabled -Set this to `true` or `false` to enable the code coverage feature of TestBox. This setting will default to `true` if TestBox detects that you have FusionReactor installed, `false` otherwise. Setting this to `true` without FusionReactor installed will be ignored. +Set this to `true` or `false` to enable the code coverage feature of TestBox. Setting this to `true` without FusionReactor installed will be ignored. -The following setting would turn off code coverage: +The following setting would turn on code coverage: ```markup - + ``` +{% hint style="warning" %} +**Changed in TestBox 7.1:** this setting now defaults to `false`. Previously it defaulted to `true`, so every runner hit attempted to start coverage even on installs with no FusionReactor. Coverage is now opt-in: set the param to `true`, or pass `?coverageEnabled=true` on the URL. +{% endhint %} + ## coveragePathToCapture Use this to point to the root folder that contains code you wish to gather coverage data from. This must be an absolute path and feel free to use any CF Mappings defined in your `/tests/Application.cfc` to make the path dynamic. This is especially useful if the app being tested is in a subfolder of the actual web root. There is nominal overhead in gathering the coverage data from files, so set this to the correct folder and instead of using the whitelist to filter down from your web root if possible. diff --git a/digging-deeper/code-coverage/running-code-coverage.md b/digging-deeper/code-coverage/running-code-coverage.md index dafe36d..8fbe7fe 100644 --- a/digging-deeper/code-coverage/running-code-coverage.md +++ b/digging-deeper/code-coverage/running-code-coverage.md @@ -47,7 +47,7 @@ coldbox create app server start ``` -Inside your directory will be a folder called `/tests` which has our test runner `/tests/runner.cfm`. You will need to open your runner.cfm and default code coverage enabled to true. +Inside your directory will be a folder called `/tests` which has our test runner `/tests/runner.cfm`. Code coverage is opt-in, so you will need to open your `runner.cfm` and default code coverage enabled to true. ``` diff --git a/digging-deeper/expectations/README.md b/digging-deeper/expectations/README.md index cda4759..ecec00e 100644 --- a/digging-deeper/expectations/README.md +++ b/digging-deeper/expectations/README.md @@ -13,3 +13,86 @@ TestBox allows you to create BDD expectations with our expectations and matcher expect( 43 ).toBe( 42 ); expect( () => calculator.add(2,2) ).toThrow(); ``` + +## Adding Context To Failures + +When several expectations in a spec assert against similar values, a raw matcher failure such as `expected [100] to be [108]` does not tell you which one broke. `withContext()` attaches a semantic label that is prepended to the failure message. + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( order.getSubtotal() ).withContext( "subtotal" ).toBe( 100 ) +expect( order.getTotal() ).withContext( "total after tax" ).toBe( 108 ) + +// Failure: total after tax: expected [100] to be [108] +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( order.getSubtotal() ).withContext( "subtotal" ).toBe( 100 ); +expect( order.getTotal() ).withContext( "total after tax" ).toBe( 108 ); + +// Failure: total after tax: expected [100] to be [108] +``` +{% endtab %} +{% endtabs %} + +The context flows through standard matchers, negated matchers and [custom matchers](custom-matchers.md) alike. + +{% hint style="success" %} +`withContext()` shines in loops and data-driven specs, where the same expectation runs many times and the failure message alone cannot identify the iteration. +{% endhint %} + +## Collection Expectations + +Instead of looping and expecting per element, you can assert against an entire array or struct at once. The matcher you chain is applied to every element, and the mode decides what has to pass. + +| Mode | Passes when | +| --- | --- | +| `expectAll( collection )` | Every element satisfies the matcher | +| `expectAny( collection )` | At least one element satisfies the matcher | +| `expectSome( collection, min, max )` | Between `min` and `max` elements satisfy the matcher | +| `expectNone( collection )` | No element satisfies the matcher | + +{% tabs %} +{% tab title="BoxLang" %} +```java +// every user must have an id +expectAll( users ).toHaveKey( "id" ) + +// at least one order is over the free-shipping threshold +expectAny( orders ).toBeGT( 50 ) + +// between 2 and 5 of them are flagged +expectSome( flags, 2, 5 ).toBeTrue() + +// no serialized user may carry a password +expectNone( serializedUsers ).toHaveKey( "password" ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +// every user must have an id +expectAll( users ).toHaveKey( "id" ); + +// at least one order is over the free-shipping threshold +expectAny( orders ).toBeGT( 50 ); + +// between 2 and 5 of them are flagged +expectSome( flags, 2, 5 ).toBeTrue(); + +// no serialized user may carry a password +expectNone( serializedUsers ).toHaveKey( "password" ); +``` +{% endtab %} +{% endtabs %} + +Failure messages report the pass and fail counts plus per-element detail, including the array index or struct key of each failing element, so you learn which elements failed rather than only that something did. + +``` +expectAll: 3 of 5 elements passed + [2] expected [null] to have key [id] + [5] expected [null] to have key [id] +``` diff --git a/digging-deeper/expectations/custom-matchers.md b/digging-deeper/expectations/custom-matchers.md index 09460be..2fe41c1 100644 --- a/digging-deeper/expectations/custom-matchers.md +++ b/digging-deeper/expectations/custom-matchers.md @@ -74,3 +74,27 @@ You can also register an instance: ```javascript addMatchers( new models.util.MyMatchers() ); ``` + +### Failure Messages And Context + +Custom matcher failures route through the same internal fail path as built-in matchers, so any [context](./#adding-context-to-failures) set on the expectation is applied to your matcher's message too: + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( 10 ).withContext( "retry budget" ).toBeGreaterThan( 50 ) +// Failure: retry budget: expected [10] to be greater than [50] +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( 10 ).withContext( "retry budget" ).toBeGreaterThan( 50 ); +// Failure: retry budget: expected [10] to be greater than [50] +``` +{% endtab %} +{% endtabs %} + +{% hint style="info" %} +**Fixed in TestBox 7.1:** custom matchers previously raised their failure message directly, bypassing the internal fail method. That meant `withContext()` applied to built-in matchers but was silently dropped for custom ones. Both behave the same way now. +{% endhint %} diff --git a/digging-deeper/expectations/data-navigator.md b/digging-deeper/expectations/data-navigator.md new file mode 100644 index 0000000..4c655a6 --- /dev/null +++ b/digging-deeper/expectations/data-navigator.md @@ -0,0 +1,120 @@ +--- +description: Assert against deeply nested data structures by path +--- + +# Data Navigator Expectations + +Asserting against a deeply nested structure usually means a chain of intermediate variables and `structKeyExists()` guards before you reach the value you actually care about. The data navigator matchers collapse that into a single path expression, built on BoxLang's `dataNavigate()` built-in. + +Paths support dot-notation, array indexes, wildcards, filters, recursive descent and the rest of the JSONPath-style expression set. + +{% hint style="info" %} +These matchers require BoxLang. On CFML engines they throw `TestBox.BoxLangFeatureNotAvailable`. +{% endhint %} + +The examples below all assume this structure: + +```java +var data = { + "app" : { "name" : "TestApp", "settings" : { "debug" : true, "port" : 8080 } }, + "users" : [ { "name" : "Alice", "age" : 30 } ] +} +``` + +## Existence + +### `toHavePath()` + +Asserts a path resolves. + +```java +expect( data ).toHavePath( "app.name" ) +expect( data ).toHavePath( "app.settings.debug" ) +expect( data ).toHavePath( "users[1].name" ) + +expect( data ).notToHavePath( "nonexistent" ) +expect( data ).notToHavePath( "app.nonexistent" ) +``` + +## Values + +### `toHavePathValue()` + +Asserts the value at a path equals an expected value. + +```java +expect( data ).toHavePathValue( "app.name", "TestApp" ) +expect( data ).toHavePathValue( "app.settings.port", 8080 ) +expect( data ).toHavePathValue( "users[1].name", "Alice" ) + +expect( data ).notToHavePathValue( "app.name", "WrongApp" ) +``` + +### `toHavePathType()` + +Asserts the type at a path. Accepts standard types (`string`, `numeric`, `boolean`, `struct`, `array`) and the aliases `str`, `num`, `bool`, `arr`, `obj` and `map`. + +```java +expect( data ).toHavePathType( "app.name", "string" ) +expect( data ).toHavePathType( "app.settings.port", "numeric" ) +expect( data ).toHavePathType( "app.settings", "struct" ) +expect( data ).toHavePathType( "users", "array" ) +expect( data ).toHavePathType( "app.settings.port", "num" ); // alias + +expect( data ).notToHavePathType( "app.name", "numeric" ) +``` + +### `toHavePathSatisfying()` + +Asserts the value at a path satisfies a predicate closure, for anything equality cannot express. + +```java +expect( data ).toHavePathSatisfying( "app.settings.port", port -> port > 1000 ) +expect( data ).toHavePathSatisfying( "app.settings", s -> s.keyExists( "debug" ) ) + +expect( data ).notToHavePathSatisfying( "app.name", name -> name == "WrongName" ) +``` + +## Chaining + +### `path()` + +Navigates to a path and hands back a normal `Expectation` on the value there, so you can chain any matcher you like rather than being limited to the path matchers above. + +```java +expect( data ).path( "app.name" ).toBe( "TestApp" ) +expect( data ).path( "app.settings.port" ).toBeGT( 8000 ) +expect( data ).path( "users" ).toHaveLength( 1 ) +expect( data ).path( "nonexistent" ).toBeNull() +``` + +### `queryPath()` + +Navigates to a path and hands back an `Expectation` on an **array of every match**. This is the one to use with wildcards, filters and recursive descent, where a path resolves to many values rather than one. + +```java +expect( data ).queryPath( "users[*].name" ).toHaveLength( 1 ) +expect( data ).queryPath( "users[*].name" ).toInclude( "Alice" ) +expect( data ).queryPath( "nonexistent" ).toBeEmpty() +``` + +## A Real Example + +Validating an API response is where this earns its keep: + +```java +it( "returns a well-formed user payload", function(){ + var response = api.get( "/users" ) + + expect( response ).toHavePathValue( "status", 200 ) + expect( response ).toHavePathType( "data.users", "array" ) + + // every user has an email, nobody leaks a password + expect( response ).queryPath( "data.users[*].email" ).notToBeEmpty() + expect( response ).notToHavePath( "data.users[*].password" ) + + // pagination is sane + expect( response ).toHavePathSatisfying( "meta.total", t -> t >= 0 ) + expect( response ).path( "meta.page" ).toBeGTE( 1 ) +} ) +``` diff --git a/digging-deeper/expectations/matchers.md b/digging-deeper/expectations/matchers.md index a00b585..b4d3f55 100644 --- a/digging-deeper/expectations/matchers.md +++ b/digging-deeper/expectations/matchers.md @@ -33,4 +33,165 @@ toBeGT( target, [message] ) : Assert that the actual value is greater than the t toBeGTE( target, [message] ) : Assert that the actual value is greater than or equal the target value toBeLT( target, [message] ) : Assert that the actual value is less than the target value toBeLTE( target, [message] ) : Assert that the actual value is less than or equal the target value +toBeTruthy( [message] ) : Assert the value is truthy: not false, not zero, not an empty string, not null +toBeFalsy( [message] ) : Assert the value is falsy: false, zero, an empty string or null +toBeSameInstanceAs( expected, [message] ) : Assert both references point at the very same object instance, not merely equal values +toHaveSize( expected, [message] ) : Assert the size of an array, struct, string or query. Alias of toHaveLength() reading more naturally for collections +toThrowMatching( predicate, [message] ) : Assert an exception is thrown AND that it satisfies the passed closure/lambda predicate +toIncludeAll( needles, [message] ) : Assert the target contains every one of the passed values +toIncludeAny( needles, [message] ) : Assert the target contains at least one of the passed values +toIncludeNone( needles, [message] ) : Assert the target contains none of the passed values ``` + +{% hint style="info" %} +Every matcher above has a negated counterpart via the [not operator](not-operator.md), for example `expect( x ).notToHaveSize( 3 )`. +{% endhint %} + +## Truthiness: `toBeTruthy()` and `toBeFalsy()` + +`toBeTrue()` and `toBeFalse()` require an actual boolean. `toBeTruthy()` and `toBeFalsy()` are looser, and are useful when a function returns "something or nothing" rather than a strict boolean. + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( "hello" ).toBeTruthy() +expect( [ 1, 2 ] ).toBeTruthy() +expect( 1 ).toBeTruthy() + +expect( "" ).toBeFalsy() +expect( 0 ).toBeFalsy() +expect( [] ).toBeFalsy() +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( "hello" ).toBeTruthy(); +expect( [ 1, 2 ] ).toBeTruthy(); +expect( 1 ).toBeTruthy(); + +expect( "" ).toBeFalsy(); +expect( 0 ).toBeFalsy(); +expect( [] ).toBeFalsy(); +``` +{% endtab %} +{% endtabs %} + +## Identity: `toBeSameInstanceAs()` + +`toBe()` compares values. `toBeSameInstanceAs()` compares identity, which is what you want when asserting that a singleton really is a singleton, or that a factory handed back the cached object rather than a fresh one. + +{% tabs %} +{% tab title="BoxLang" %} +```java +var a = getInstance( "UserService" ) +var b = getInstance( "UserService" ) + +expect( a ).toBeSameInstanceAs( b ) // same object in memory +expect( a ).notToBeSameInstanceAs( {} ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +var a = getInstance( "UserService" ); +var b = getInstance( "UserService" ); + +expect( a ).toBeSameInstanceAs( b ); // same object in memory +expect( a ).notToBeSameInstanceAs( {} ); +``` +{% endtab %} +{% endtabs %} + +## Size: `toHaveSize()` + +Works on arrays, structs, strings and queries. + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( [ 1, 2, 3 ] ).toHaveSize( 3 ) +expect( { a : 1, b : 2 } ).toHaveSize( 2 ) +expect( "TestBox" ).toHaveSize( 7 ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( [ 1, 2, 3 ] ).toHaveSize( 3 ); +expect( { a : 1, b : 2 } ).toHaveSize( 2 ); +expect( "TestBox" ).toHaveSize( 7 ); +``` +{% endtab %} +{% endtabs %} + +## Exceptions: `toThrowMatching()` + +`toThrow()` matches on exception type and a message regex. `toThrowMatching()` hands you the exception so you can assert anything about it. + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( () => paymentService.charge( amount = -5 ) ) + .toThrowMatching( e => e.type == "InvalidAmount" && e.detail contains "negative" ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( function(){ + paymentService.charge( amount = -5 ); +} ).toThrowMatching( function( e ){ + return e.type == "InvalidAmount" && e.detail contains "negative"; +} ); +``` +{% endtab %} +{% endtabs %} + +This is the escape hatch for exceptions whose interesting detail is not in the type or the message: a custom `extendedInfo` payload, an error code, a nested cause. + +## Collections: `toIncludeAll()`, `toIncludeAny()`, `toIncludeNone()` + +`toInclude()` checks for a single needle. These three check for several at once against arrays, lists and strings. + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( [ "admin", "editor", "viewer" ] ).toIncludeAll( [ "admin", "editor" ] ) +expect( [ "admin", "viewer" ] ).toIncludeAny( [ "admin", "superuser" ] ) +expect( [ "viewer" ] ).toIncludeNone( [ "admin", "superuser" ] ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( [ "admin", "editor", "viewer" ] ).toIncludeAll( [ "admin", "editor" ] ); +expect( [ "admin", "viewer" ] ).toIncludeAny( [ "admin", "superuser" ] ); +expect( [ "viewer" ] ).toIncludeNone( [ "admin", "superuser" ] ); +``` +{% endtab %} +{% endtabs %} + +Use `toIncludeNone()` to assert the absence of things that must never leak, which reads better than chaining several negated `toInclude()` calls: + +{% tabs %} +{% tab title="BoxLang" %} +```java +expect( serializedUser ).toIncludeNone( [ "password", "salt", "apiToken" ] ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +expect( serializedUser ).toIncludeNone( [ "password", "salt", "apiToken" ] ); +``` +{% endtab %} +{% endtabs %} + +## Specialized Matcher Families + +TestBox 7.1 adds three dedicated matcher families with their own pages: + +- [Set Expectations](set-expectations.md) for BoxLang `Set` objects +- [Range Expectations](range-expectations.md) for BoxLang `Range` objects +- [Data Navigator Expectations](data-navigator.md) for asserting against deeply nested structures by path diff --git a/digging-deeper/expectations/range-expectations.md b/digging-deeper/expectations/range-expectations.md new file mode 100644 index 0000000..9975c39 --- /dev/null +++ b/digging-deeper/expectations/range-expectations.md @@ -0,0 +1,140 @@ +--- +description: Matchers for BoxLang Range objects +--- + +# Range Expectations + +BoxLang has a native `Range` type covering containment, ordering, bounds, stepping and clamping. TestBox 7.1 adds matchers that assert against ranges directly instead of picking them apart into endpoints first. + +{% hint style="info" %} +These matchers require BoxLang. On CFML engines they are guarded and report unsupported behavior cleanly rather than erroring. +{% endhint %} + +## Creating Ranges + +Ranges are built with the `..` operator and stepped with `.step( n )`. There is no `rangeNew()` built-in. + +```java +var base = 1..10 +var stepped = ( 0..100 ).step( 5 ) +var chars = "a".."z" +var dates = createDate( 2024, 1, 1 )..createDate( 2024, 1, 31 ) + +var openEnd = 1.. // half bounded +var openStart = ..10 // half bounded +var open = .. // unbounded +``` + +## Containment + +### `toBeRange()` + +```java +expect( 1..10 ).toBeRange() +``` + +### `toContainValue()` + +Asserts a value falls inside the range. + +```java +expect( 1..10 ).toContainValue( 5 ) +expect( "a".."z" ).toContainValue( "m" ) +expect( dates ).toContainValue( "2024-01-15" ) +``` + +### `toContainRange()` + +Asserts an entire range fits inside another. + +```java +expect( 1..10 ).toContainRange( 3..7 ) +``` + +### `toBeInRange()` + +The inverse reading of `toContainValue()`, with the value as the subject. Use whichever makes the sentence read better. + +```java +expect( 8 ).toBeInRange( 1..10 ) +``` + +### `toBeBeforeRange()` / `toBeAfterRange()` + +```java +expect( 0 ).toBeBeforeRange( 1..10 ) +expect( 11 ).toBeAfterRange( 1..10 ) +``` + +## Shape And Direction + +### `toBeBounded()`, `toBeUnbounded()`, `toBeHalfBounded()` + +```java +expect( 1..10 ).toBeBounded() +expect( .. ).toBeUnbounded() +expect( 1.. ).toBeHalfBounded() +``` + +### `toBeIterable()` + +A range is iterable when it can actually be walked, which an unbounded range cannot. + +```java +expect( 1..10 ).toBeIterable() +``` + +### `toBeAscending()` / `toBeDescending()` + +```java +expect( 1..10 ).toBeAscending() +expect( 10..1 ).toBeDescending() +expect( "a".."z" ).toBeAscending() +``` + +## Step And Clamp + +### `toHaveStep()` + +```java +expect( ( 0..100 ).step( 5 ) ).toHaveStep( 5 ) +``` + +### `toClampTo()` + +Asserts what the range clamps a given value to. Takes the input value and the expected clamped result. + +```java +expect( 1..10 ).toClampTo( 15, 10 ); // 15 clamps down to 10 +expect( 1..10 ).toClampTo( -3, 1 ); // -3 clamps up to 1 +``` + +## Native Equivalents + +These matchers sit on top of the native Range API, so the following are equivalent to the assertions above: + +```java +base.contains( 5 ) +base.contains( 3..7 ) +stepped.getStep() // 5 +base.clamp( 15 ) // 10 +``` + +## A Real Example + +```java +describe( "Pagination window", function(){ + + it( "clamps a requested page into the available range", function(){ + var pages = 1..totalPages + + expect( pages ).toBeRange() + expect( pages ).toBeBounded() + expect( pages ).toBeAscending() + + expect( pages ).toContainValue( currentPage ) + expect( pages ).toClampTo( 9999, totalPages ) + } ) + +} ) +``` diff --git a/digging-deeper/expectations/set-expectations.md b/digging-deeper/expectations/set-expectations.md new file mode 100644 index 0000000..cfc6fda --- /dev/null +++ b/digging-deeper/expectations/set-expectations.md @@ -0,0 +1,110 @@ +--- +description: Matchers for BoxLang Set objects +--- + +# Set Expectations + +BoxLang ships a native `Set` type. Asserting against one with array matchers means sorting first and hoping the ordering is stable, which obscures what the test actually means. TestBox 7.1 adds a family of matchers that speak set semantics directly. + +{% hint style="info" %} +These matchers require BoxLang. They are not available on Lucee or Adobe ColdFusion. +{% endhint %} + +## Creating Sets + +Use the `setOf()` built-in to build a set inline: + +```java +var roles = setOf( "admin", "editor", "viewer" ) +``` + +## Membership And Equality + +### `toBeASet()` + +Asserts the actual value is a `Set`. + +```java +expect( setOf( 1, 2 ) ).toBeASet() +expect( [ 1, 2 ] ).notToBeASet() +``` + +### `toEqualSet()` + +Asserts two sets contain the same members. Order is irrelevant, which is the whole point of a set. + +```java +expect( setOf( 1, 2 ) ).toEqualSet( setOf( 2, 1 ) ) +``` + +## Subsets And Supersets + +### `toBeSubsetOf()` / `toBeSupersetOf()` + +```java +expect( setOf( "admin" ) ).toBeSubsetOf( setOf( "admin", "editor" ) ) +expect( setOf( "admin", "editor" ) ).toBeSupersetOf( setOf( "admin" ) ) +``` + +Use these for permission checks, where the assertion is "the granted roles include at least these" rather than an exact match. + +### `toBeDisjointFrom()` + +Asserts the two sets share no members at all. + +```java +expect( setOf( "read" ) ).toBeDisjointFrom( setOf( "write", "delete" ) ) +``` + +## Set Algebra + +Each of these takes the other operand and the expected result. + +### `toHaveUnion()` + +```java +expect( setOf( 1 ) ).toHaveUnion( setOf( 2 ), setOf( 1, 2 ) ) +``` + +### `toHaveIntersection()` + +```java +expect( setOf( 1, 2 ) ).toHaveIntersection( setOf( 2, 3 ), setOf( 2 ) ) +``` + +### `toHaveDifference()` + +Members in the actual set that are not in the other set. + +```java +expect( setOf( 1, 2 ) ).toHaveDifference( setOf( 2 ), setOf( 1 ) ) +``` + +### `toHaveSymmetricDifference()` + +Members in either set but not both. + +```java +expect( setOf( 1, 2 ) ).toHaveSymmetricDifference( setOf( 2, 3 ), setOf( 1, 3 ) ) +``` + +## Negated Forms + +Every matcher here has a negated counterpart: `notToBeASet()`, `notToEqualSet()`, `notToBeSubsetOf()`, `notToBeSupersetOf()`, `notToHaveUnion()`, `notToHaveIntersection()`, `notToHaveDifference()` and `notToHaveSymmetricDifference()`. + +## A Real Example + +```java +describe( "Menu permissions", function(){ + + it( "shows only the menu items the user may reach", function(){ + var visible = menuService.visibleFor( user ) + var granted = setOf( "dashboard", "reports" ) + + expect( visible ).toBeASet() + expect( visible ).toEqualSet( granted ) + expect( visible ).toBeDisjointFrom( setOf( "admin", "billing" ) ) + } ) + +} ) +``` diff --git a/getting-started/test-bundles.md b/getting-started/test-bundles.md index b66bd20..84175bb 100644 --- a/getting-started/test-bundles.md +++ b/getting-started/test-bundles.md @@ -159,7 +159,7 @@ These methods assist you with identifying environment conditions. ```java // Which language/engine are you running on isAdobe() -isLucee +isLucee() isBoxLang() // What OS are we on @@ -171,6 +171,10 @@ isWindows() getEnv() ``` +{% hint style="info" %} +**Fixed in TestBox 7.1:** `isLucee()` used to return `true` under BoxLang, because BoxLang registers a `lucee` server scope key for compatibility. Specs that branched or skipped on engine took the Lucee path when running on BoxLang. `isLucee()` now returns `true` only on actual Lucee. +{% endhint %} + ### Java Environment You can use the `getEnv()` to get access to our Environment utility object. From there you can use the following methods: diff --git a/getting-started/testbox-bdd-primer/skipping-specs-and-suites.md b/getting-started/testbox-bdd-primer/skipping-specs-and-suites.md index 2a7d802..bf6d5a8 100644 --- a/getting-started/testbox-bdd-primer/skipping-specs-and-suites.md +++ b/getting-started/testbox-bdd-primer/skipping-specs-and-suites.md @@ -53,6 +53,64 @@ describe("A spec", function() { }); ``` +## Skipping An Entire Class + +As of TestBox 7.1, a BDD test class can carry a class-level `skip` annotation, so the whole class is skipped without prefixing every `describe()` or editing your runner filters. + +{% tabs %} +{% tab title="BoxLang" %} +{% code title="PaymentGatewaySpec.bx" %} +```java +/** + * @skip + */ +class extends="testbox.system.BaseSpec"{ + + function run(){ + describe( "Payment gateway", () => { + // none of this runs while @skip is present + } ) + } + +} +``` +{% endcode %} +{% endtab %} + +{% tab title="CFML" %} +{% code title="PaymentGatewayTest.cfc" %} +```cfscript +/** + * @skip + */ +component extends="testbox.system.BaseSpec"{ + + function run(){ + describe( "Payment gateway", function(){ + // none of this runs while @skip is present + } ); + } + +} +``` +{% endcode %} +{% endtab %} +{% endtabs %} + +You can supply a reason, which shows up in the reporters: + +```java +/** + * @skip Waiting on the sandbox credentials + */ +``` + +Skipped classes are reported as skipped rather than silently dropped, so your totals stay honest and the class does not quietly rot. + +{% hint style="info" %} +This is the class-level equivalent of `xdescribe()`. Use it when an entire bundle is blocked on something external, and prefer the [skip argument](#skip-argument) below when the decision depends on the engine or on runtime state. +{% endhint %} + ## Skip Argument The `skip` argument can be a boolean value or a closure. If the value is **true** then the suite or spec is skipped. If the return value of the closure is **true** then the suite or spec is skipped. Using the closure approach allows you to dynamically at runtime figure out if the desired spec or suite is skipped. This is such a great way to prepare tests for different CFML engines. diff --git a/getting-started/testbox-xunit-primer/assertions.md b/getting-started/testbox-xunit-primer/assertions.md index f33dd44..32cd956 100644 --- a/getting-started/testbox-xunit-primer/assertions.md +++ b/getting-started/testbox-xunit-primer/assertions.md @@ -261,6 +261,42 @@ component displayName="TestBox xUnit suite for CF9" labels="railo,cf"{ } ``` +## Grouped Assertions + +A failing assertion aborts the test method, so you only see the first failure per run. `assertAll()` runs several assertion closures and reports every failure at once: + +{% tabs %} +{% tab title="BoxLang" %} +```groovy +function testUserProfile(){ + var user = userService.get( 1 ) + + assertAll( [ + () => $assert.isEqual( "Luis", user.getName() ), + () => $assert.isEqual( "luis@ortussolutions.com", user.getEmail() ), + () => $assert.isTrue( user.isActive() ) + ], "user profile" ) +} +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +function testUserProfile(){ + var user = userService.get( 1 ); + + assertAll( [ + function(){ return $assert.isEqual( "Luis", user.getName() ); }, + function(){ return $assert.isEqual( "luis@ortussolutions.com", user.getEmail() ); }, + function(){ return $assert.isTrue( user.isActive() ); } + ], "user profile" ); +} +``` +{% endtab %} +{% endtabs %} + +See [Assertions](../../digging-deeper/assertions/#grouped-assertions) for the full details, plus the newer `isTruthy()`, `isFalsy()`, `includesAll()`, `includesAny()` and `includesNone()` assertions. + ## Custom Assertions You can also register custom assertions within the $assert object. You will do this by reading our Custom Assertions section of our TestBox docs. diff --git a/mocking/mockbox/mocking-methods/usdargs-method.md b/mocking/mockbox/mocking-methods/usdargs-method.md index 41c1f97..cf7526f 100644 --- a/mocking/mockbox/mocking-methods/usdargs-method.md +++ b/mocking/mockbox/mocking-methods/usdargs-method.md @@ -36,3 +36,51 @@ mockConfig = getMockBox().createEmptyMock("coldbox.system.beans.ConfigBean"); //mock the method for named arguments mockConfig.$("getKey").$args(name="debugmode").$results(true); ``` + +## Matching Complex Arguments + +`$args()` matches structurally, so a struct argument matches whatever order its keys were built in: + +{% tabs %} +{% tab title="BoxLang" %} +```java +mockService.$( "charge" ) + .$args( { amount : 100, currency : "USD" } ) + .$results( true ) + +// matches, despite the different key order at the call site +mockService.charge( { currency : "USD", amount : 100 } ) +``` +{% endtab %} + +{% tab title="CFML" %} +```cfscript +mockService.$( "charge" ) + .$args( { amount : 100, currency : "USD" } ) + .$results( true ); + +// matches, despite the different key order at the call site +mockService.charge( { currency : "USD", amount : 100 } ); +``` +{% endtab %} +{% endtabs %} + +{% hint style="warning" %} +Before TestBox 7.1, nested structures were hashed in a way that depended on struct iteration order, so two structurally-equal structs built in a different order could fail to match and the mock would return `null` instead. This was always latent but became reproducible on Lucee 7.1, which changed its underlying map implementation. Upgrade to 7.1 or later if you mock methods that take struct arguments. +{% endhint %} + +As of TestBox 7.1, `$args()` also understands BoxLang `Set` and `Range` objects when matching: + +{% hint style="info" %} +`Set` and `Range` argument matching requires BoxLang. On CFML engines these types do not exist, so the rest of `$args()` behaves as documented above. +{% endhint %} + +```java +mockService.$( "grant" ) + .$args( setOf( "admin", "editor" ) ) + .$results( true ) + +mockService.$( "paginate" ) + .$args( 1..10 ) + .$results( results ) +``` diff --git a/readme/release-history/README.md b/readme/release-history/README.md index cdcfb9c..c6aa5ad 100644 --- a/readme/release-history/README.md +++ b/readme/release-history/README.md @@ -14,6 +14,8 @@ In this section, you will find the release notes for each version we release und In this release, we focused on BoxLang CLI runner enhancements, real-time streaming test execution via SSE, a powerful dry run capability for spec discovery, Lucee 7 certification, and dropping Adobe 2021 support. Developers gain a wealth of new output-control and performance-analysis options directly from the CLI runner. +The **7.1** minor release builds on that foundation with a major expansion of the assertions and expectations library: grouped assertions, collection expectation modes, expectation context for richer failure messages, and dedicated matcher families for BoxLang `Set` and `Range` objects plus a data navigator for asserting against deeply nested structures. + ## Version 6.x - September 2024 In this release, we have dropped more legacy engines and added full support for not only running TestBox in [BoxLang](https://www.boxlang.io/), but writing every spec in [BoxLang](https://www.boxlang.io/). We have added tons of bug fixes and major improvements but also a great CLI runner for BoxLang. diff --git a/readme/release-history/whats-new-with-7.1.0.md b/readme/release-history/whats-new-with-7.1.0.md new file mode 100644 index 0000000..7c75d57 --- /dev/null +++ b/readme/release-history/whats-new-with-7.1.0.md @@ -0,0 +1,600 @@ +--- +description: September 10, 2026 +--- + +# What's New With 7.1.0 + +TestBox 7.1.0 introduces a significant upgrade to the assertions and expectations library, adding grouped assertions, collection expectation modes, rich failure diagnostics, and a suite of new matchers inspired by JUnit 5 and Jasmine. + +* * * + +## Expectation Context: `withContext()` + +Add semantic context to any expectation so failure messages include identifying information. Works with all matchers, negated matchers, and custom matchers. + +```javascript +it( "validates a user record", () => { + expect( user.age ) + .withContext( "user.age" ) + .toBeGT( 0 ) + + expect( user.email ) + .withContext( "user.email" ) + .toMatch( "@" ) +} ) +``` + +* * * + +## Collection Expectation Modes + +Three new collection modes complement the existing `expectAll()`. + +### `expectAny()` + +Passes when at least one element in the collection passes the chained matcher. + +```javascript +expectAny( products ).toSatisfy( p => p.onSale ) +``` + +### `expectSome()` + +Passes when a bounded number of elements pass. `max = 0` means no upper bound. + +```javascript +expectSome( users, min = 2, max = 5 ).toSatisfy( u => u.role == "admin" ) +expectSome( items, min = 3 ).toSatisfy( i => i.stock > 0 ) +``` + +### `expectNone()` + +Passes when zero elements pass the chained matcher. + +```javascript +expectNone( users ).toSatisfy( u => u.banned ) +``` + +### Improved `expectAll()` Failure Messages + +All collection modes now produce detailed failure summaries with pass/fail counts and per-element failure context including the element index or struct key. + +```javascript +try { + expectAll( [ 2, 4, 10, 8 ] ).toBeLT( 10 ) +} catch ( any e ) { + // e.message: "expectAll() failed: 1 of 4 element(s) did not pass the [toBeLT] expectation" + // e.detail: "Passed: 3 / 4\n\n[3]: The actual [10] is not less than [10]" +} +``` + +* * * + +## Grouped Assertions: `assertAll()` + +Run multiple assertion closures and report every failure at once — instead of stopping at the first. Non-assertion exceptions are rethrown immediately. + +```javascript +it( "validates a user record completely", () => { + assertAll( [ + () => expect( user.name ).notToBeEmpty(), + () => expect( user.email ).toMatch( "@" ), + () => expect( user.age ).toBeGTE( 18 ), + () => expect( user.age ).toBeLT( 120 ) + ] ) +} ) +``` + +Using the `$assert` style: + +```javascript +it( "xUnit style grouped assertions", () => { + $assert.all( + executables = [ + () => $assert.isTrue( true ), + () => $assert.isEqual( 1, 2 ), + () => $assert.notNull( javacast( "null", "" ) ) + ], + heading = "Basic checks" + ) +} ) +``` + +* * * + +## New Matchers + +### `toIncludeAll` / `toIncludeAny` / `toIncludeNone` + +Assert that a string or array contains all, any, or none of the given needles with case-insensitive matching. + +```javascript +expect( "hello world" ).toIncludeAll( [ "hello", "world" ] ) +expect( "hello world" ).toIncludeAny( [ "hello", "foo" ] ) +expect( "hello world" ).toIncludeNone( [ "foo", "bar" ] ) +``` + +### `toBeTruthy` / `toBeFalsy` + +Assert that a value is truthy (not false, `0`, empty string, or null) or falsy. + +```javascript +expect( 42 ).toBeTruthy() +expect( "" ).toBeFalsy() +``` + +### `toHaveSize` + +Alias for `toHaveLength()` — works on strings, arrays, structs, and queries. + +```javascript +expect( "abc" ).toHaveSize( 3 ) +expect( [ 1, 2 ] ).toHaveSize( 2 ) +expect( { a : 1, b : 2 } ).toHaveSize( 2 ) +``` + +### `toBeSameInstanceAs` + +Assert two references point to the exact same object instance. + +```javascript +var obj = { name : "test" } +expect( obj ).toBeSameInstanceAs( obj ) +``` + +### `toThrowMatching` + +Assert a function throws an exception that matches a predicate closure. + +```javascript +expect( () => { + throw( type = "FooException" ) +} ).toThrowMatching( e => e.type == "FooException" ) +``` + +* * * + +## New Assertion BIFs + +For xUnit-style testing, the following new `$assert` methods are available: + +| Method | Description | +| --- | --- | +| `$assert.isTruthy( actual, message )` | Value is truthy | +| `$assert.isFalsy( actual, message )` | Value is falsy | +| `$assert.includesAll( target, needles, message )` | Target contains every needle | +| `$assert.includesAny( target, needles, message )` | Target contains at least one needle | +| `$assert.includesNone( target, needles, message )` | Target contains no needle | +| `$assert.all( executables, heading )` | Run all assertions, report every failure | + +```javascript +$assert.isTruthy( "hello" ) +$assert.isFalsy( 0 ) +$assert.includesAll( "hello world", [ "hello", "world" ] ) +$assert.includesAny( [ "a", "b" ], [ "b", "z" ] ) +$assert.includesNone( "hello", [ "x", "y" ] ) +``` + +* * * + +## Set Expectations + +TestBox now provides a comprehensive suite of set-related matchers for working with BoxLang `Set` objects. These matchers leverage the global `setOf()` function to create sets and provide powerful assertions for set operations. + +### Creating Sets with `setOf()` + +```javascript +var set1 = setOf( 1, 2, 3 ) +var set2 = setOf( 3, 4, 5 ) +var set3 = setOf( 1, 2, 3, 4, 5 ) +var emptySet = setOf() +``` + +### `toBeASet()` / `notToBeASet()` + +Assert that a value is (or is not) a Set object. + +```javascript +expect( set1 ).toBeASet() +expect( [ 1, 2, 3 ] ).notToBeASet() +``` + +### `toEqualSet()` / `notToEqualSet()` + +Assert that two sets contain the same elements, regardless of order. + +```javascript +expect( setOf( 1, 2, 3 ) ).toEqualSet( setOf( 3, 2, 1 ) ) +expect( setOf( 'a', 'b' ) ).toEqualSet( setOf( 'b', 'a' ) ) +expect( setOf( 1, 'a', true ) ).toEqualSet( setOf( true, 'a', 1 ) ) +``` + +### `toBeSubsetOf()` / `notToBeSubsetOf()` + +Assert that all elements of the actual set are contained in the expected set. + +```javascript +expect( setOf( 1, 2 ) ).toBeSubsetOf( setOf( 1, 2, 3, 4, 5 ) ) +expect( setOf( 1, 2, 3, 4, 5 ) ).notToBeSubsetOf( setOf( 1, 2 ) ) +``` + +### `toBeSupersetOf()` / `notToBeSupersetOf()` + +Assert that the actual set contains all elements of the expected set. + +```javascript +expect( setOf( 1, 2, 3, 4, 5 ) ).toBeSupersetOf( setOf( 1, 2, 3 ) ) +expect( setOf( 1, 2, 3 ) ).notToBeSupersetOf( setOf( 1, 2, 3, 4, 5 ) ) +``` + +### `toBeDisjointFrom()` + +Assert that two sets share no common elements. + +```javascript +expect( setOf( 1, 2 ) ).toBeDisjointFrom( setOf( 3, 4 ) ) +``` + +### `toHaveUnion()` / `notToHaveUnion()` + +Assert that the union of two sets equals an expected set. + +```javascript +expect( setOf( 1, 2 ) ).toHaveUnion( setOf( 3, 4 ), setOf( 1, 2, 3, 4 ) ) +``` + +### `toHaveIntersection()` / `notToHaveIntersection()` + +Assert that the intersection of two sets equals an expected set. + +```javascript +expect( setOf( 1, 2, 3 ) ).toHaveIntersection( setOf( 3, 4, 5 ), setOf( 3 ) ) +``` + +### `toHaveDifference()` / `notToHaveDifference()` + +Assert that the set difference (actual - expected) equals an expected result. + +```javascript +expect( setOf( 1, 2, 3 ) ).toHaveDifference( setOf( 3, 4, 5 ), setOf( 1, 2 ) ) +``` + +### `toHaveSymmetricDifference()` / `notToHaveSymmetricDifference()` + +Assert that the symmetric difference (elements in either set but not both) equals an expected result. + +```javascript +expect( setOf( 1, 2, 3 ) ).toHaveSymmetricDifference( setOf( 3, 4, 5 ), setOf( 1, 2, 4, 5 ) ) +``` + +### Real-World Example: Menu Selection + +```javascript +it( "validates menu selection", () => { + var fruits = setOf( 'apple', 'banana', 'cherry' ) + var selected = setOf( 'apple', 'banana' ) + + expect( selected ).toBeSubsetOf( fruits ) + expect( selected ).toHaveIntersection( fruits, setOf( 'apple', 'banana' ) ) +} ) +``` + +* * * + +## Range Expectations (BoxLang) + +TestBox now includes a full set of matchers for BoxLang `Range` objects, including containment, ordering, bounds, and step/clamp assertions. + +{% hint style="info" %} +**BoxLang only.** Range features depend on BoxLang range support. On CFML engines these expectations are guarded and report unsupported behavior cleanly. + +Ranges are created with the `..` operator (for example `1..10`, `..10`, `1..`, `..`) and stepped via `.step( n )`. There is no `rangeNew()` BIF. +{% endhint %} + +### Core Range Matchers + +- `toBeRange()` +- `toContainValue( value )` +- `toContainRange( range )` +- `toBeInRange( range )` +- `toBeBeforeRange( range )` +- `toBeAfterRange( range )` + +### Range Shape And Direction + +- `toBeBounded()` +- `toBeUnbounded()` +- `toBeHalfBounded()` +- `toBeIterable()` +- `toBeAscending()` +- `toBeDescending()` + +### Step And Clamp + +- `toHaveStep( step )` +- `toClampTo( value, expected )` + +```javascript +var base = 1..10 +var stepped = (0..100).step( 5 ) +var chars = "a".."z" +var dates = createDate( 2024, 1, 1 )..createDate( 2024, 1, 31 ) + +expect( base ).toBeRange() +expect( base ).toContainValue( 5 ) +expect( base ).toContainRange( 3..7 ) +expect( 8 ).toBeInRange( base ) + +expect( stepped ).toHaveStep( 5 ) +expect( base ).toClampTo( 15, 10 ) +expect( dates ).toContainValue( "2024-01-15" ) + +expect( chars ).toBeAscending() +expect( chars ).toContainValue( "m" ) +``` + +Equivalent native Range API examples used under these matchers: + +```javascript +base.contains( 5 ) +base.contains( 3..7 ) +stepped.getStep() // 5 +base.clamp( 15 ) // 10 +``` + +* * * + +## Data Navigator Expectations + +TestBox now provides a suite of matchers that leverage BoxLang's built-in `dataNavigate()` BIF to safely navigate and assert against values in nested data structures. These matchers support dot-notation, array indexes, wildcards, filters, recursive descent, and all other JSONPath-style expressions. + +{% hint style="info" %} +**BoxLang only.** Data navigator features require the BoxLang runtime and are guarded at the matcher level. On CFML engines they throw `TestBox.BoxLangFeatureNotAvailable`. +{% endhint %} + +### `toHavePath()` / `notToHavePath()` + +Assert that a path exists (or does not exist) in a nested data structure. + +```javascript +var data = { + "app" = { "name" = "TestApp", "settings" = { "debug" = true, "port" = 8080 } }, + "users" = [ { "name" = "Alice", "age" = 30 } ] +} + +expect( data ).toHavePath( "app.name" ) +expect( data ).toHavePath( "app.settings.debug" ) +expect( data ).toHavePath( "users[1].name" ) +expect( data ).notToHavePath( "nonexistent" ) +expect( data ).notToHavePath( "app.nonexistent" ) +``` + +### `toHavePathValue()` / `notToHavePathValue()` + +Assert that the value at a path matches an expected value. + +```javascript +expect( data ).toHavePathValue( "app.name", "TestApp" ) +expect( data ).toHavePathValue( "app.settings.port", 8080 ) +expect( data ).toHavePathValue( "app.settings.debug", true ) +expect( data ).toHavePathValue( "users[1].name", "Alice" ) +expect( data ).notToHavePathValue( "app.name", "WrongApp" ) +``` + +### `toHavePathType()` / `notToHavePathType()` + +Assert the type of the value at a path. Supports standard types (`string`, `numeric`, `boolean`, `struct`, `array`) and common aliases (`str`, `num`, `bool`, `arr`, `obj`, `map`). + +```javascript +expect( data ).toHavePathType( "app.name", "string" ) +expect( data ).toHavePathType( "app.settings.port", "numeric" ) +expect( data ).toHavePathType( "app.settings.debug", "boolean" ) +expect( data ).toHavePathType( "app.settings", "struct" ) +expect( data ).toHavePathType( "users", "array" ) +expect( data ).toHavePathType( "app.settings.port", "num" ) // alias +expect( data ).notToHavePathType( "app.name", "numeric" ) +``` + +### `toHavePathSatisfying()` / `notToHavePathSatisfying()` + +Assert that the value at a path satisfies a predicate closure. + +```javascript +expect( data ).toHavePathSatisfying( "app.name", name -> name == "TestApp" ) +expect( data ).toHavePathSatisfying( "app.settings.port", port -> port > 1000 ) +expect( data ).toHavePathSatisfying( "app.settings", settings -> settings.keyExists( "debug" ) ) +expect( data ).notToHavePathSatisfying( "app.name", name -> name == "WrongName" ) +``` + +### `path()` + +Navigate to a path and return a normal `Expectation` on the value at that path. Supports chaining any matcher on the result. + +```javascript +expect( data ).path( "app.name" ).toBe( "TestApp" ) +expect( data ).path( "app.settings.port" ).toBeGT( 8000 ) +expect( data ).path( "users" ).toHaveLength( 1 ) +expect( data ).path( "nonexistent" ).toBeNull() +``` + +### `queryPath()` + +Navigate to a path and return an `Expectation` on an array of all matching values. Fans out at wildcards, filters, and recursive descent segments. + +```javascript +expect( data ).queryPath( "users[*].name" ).toHaveLength( 1 ) +expect( data ).queryPath( "users[*].name" ).toInclude( "Alice" ) +expect( data ).queryPath( "nonexistent" ).toBeEmpty() +``` + +### Real-World Example: API Response Validation + +```javascript +it( "validates an API response", () => { + var response = { + "success" = true, + "data" = { + "users" = [ + { "name" = "Alice", "role" = "admin", "active" = true }, + { "name" = "Bob", "role" = "user", "active" = false } + ], + "metadata" = { "total" = 2, "page" = 1 } + } + } + + // Path existence + expect( response ).toHavePath( "data.users" ) + expect( response ).notToHavePath( "data.errors" ) + + // Path values + expect( response ).toHavePathValue( "success", true ) + expect( response ).toHavePathValue( "data.metadata.total", 2 ) + + // Path types + expect( response ).toHavePathType( "data.users", "array" ) + expect( response ).toHavePathType( "data.metadata.total", "numeric" ) + + // Predicate + expect( response ).toHavePathSatisfying( "data.users[1].role", role -> role == "admin" ) + + // Path extraction + expect( response ).path( "data.users[1].name" ).toBe( "Alice" ) + expect( response ).queryPath( "data.users[*].name" ).toInclude( "Bob" ) +} ) +``` + +## Class-Level `skip` Annotation + +BDD test classes now support a class-level `skip` annotation, so an entire class can be skipped without touching each `describe()` or editing the runner's filters. + +```js +/** + * @skip + */ +component extends="testbox.system.BaseSpec" { + + function run(){ + describe( "Payment gateway", function(){ + // none of this runs while @skip is present + } ); + } + +} +``` + +Skipped classes are reported as skipped rather than silently omitted, so the count stays honest. + +You can also pass a reason: + +```js +/** + * @skip Waiting on the sandbox credentials + */ +``` + +## MockBox `$args()` Improvements + +`$args()` previously built its argument hash in a way that depended on struct iteration order. Two structurally identical structs built in a different order could hash differently, so a mock could fail to match arguments it should have matched and would return `null` instead. + +This was always latent, but Lucee 7.1 changed its underlying map implementation and made it reproducible. + +`$args()` now normalizes nested structures deterministically, so structurally-equal arguments match regardless of how they were built: + +```js +var mock = createMock( "PaymentService" ) + .$( "charge" ) + .$args( { amount : 100, currency : "USD" } ) + .$results( true ); + +// matches, even though the keys were supplied in a different order +mock.charge( { currency : "USD", amount : 100 } ); +``` + +`$args()` also now understands BoxLang `Set` and `Range` objects when matching. + +## Summary + +| Feature | Type | Example | +| --- | --- | --- | +| `withContext()` | Expectation | `expect( v ).withContext( "label" ).toBe( x )` | +| `expectAny()` | Collection | `expectAny( arr ).toBeGT( 2 )` | +| `expectSome()` | Collection | `expectSome( arr, 2, 5 ).toBeGT( 2 )` | +| `expectNone()` | Collection | `expectNone( arr ).toBeEmpty()` | +| `assertAll()` | Grouped | `assertAll( closures, "heading" )` | +| `toBeTruthy()` | Matcher | `expect( v ).toBeTruthy()` | +| `toBeFalsy()` | Matcher | `expect( v ).toBeFalsy()` | +| `toBeSameInstanceAs()` | Matcher | `expect( a ).toBeSameInstanceAs( b )` | +| `toHaveSize()` | Matcher | `expect( v ).toHaveSize( 3 )` | +| `toThrowMatching()` | Matcher | `expect( fn ).toThrowMatching( p )` | +| `toIncludeAll()` | Matcher | `expect( v ).toIncludeAll( needles )` | +| `toIncludeAny()` | Matcher | `expect( v ).toIncludeAny( needles )` | +| `toIncludeNone()` | Matcher | `expect( v ).toIncludeNone( needles )` | +| `toBeASet()` / `nottoBeASet()` | Set | `expect( setOf( 1, 2 ) ).toBeASet()` | +| `toEqualSet()` / `notToEqualSet()` | Set | `expect( setOf( 1, 2 ) ).toEqualSet( setOf( 2, 1 ) )` | +| `toBeSubsetOf()` / `notToBeSubsetOf()` | Set | `expect( setOf( 1 ) ).toBeSubsetOf( setOf( 1, 2 ) )` | +| `toBeSupersetOf()` / `notToBeSupersetOf()` | Set | `expect( setOf( 1, 2 ) ).toBeSupersetOf( setOf( 1 ) )` | +| `toBeDisjointFrom()` | Set | `expect( setOf( 1 ) ).toBeDisjointFrom( setOf( 2 ) )` | +| `toHaveUnion()` / `notToHaveUnion()` | Set | `expect( setOf( 1 ) ).toHaveUnion( setOf( 2 ), setOf( 1, 2 ) )` | +| `toHaveIntersection()` / `notToHaveIntersection()` | Set | `expect( setOf( 1, 2 ) ).toHaveIntersection( setOf( 2, 3 ), setOf( 2 ) )` | +| `toHaveDifference()` / `notToHaveDifference()` | Set | `expect( setOf( 1, 2 ) ).toHaveDifference( setOf( 2 ), setOf( 1 ) )` | +| `toHaveSymmetricDifference()` / `notToHaveSymmetricDifference()` | Set | `expect( setOf( 1, 2 ) ).toHaveSymmetricDifference( setOf( 2, 3 ), setOf( 1, 3 ) )` | +| `toBeRange()` | Range | `expect( 1..10 ).toBeRange()` | +| `toContainValue()` | Range | `expect( 1..10 ).toContainValue( 5 )` | +| `toContainRange()` | Range | `expect( 1..10 ).toContainRange( 2..5 )` | +| `toBeInRange()` | Range | `expect( 5 ).toBeInRange( 1..10 )` | +| `toBeBeforeRange()` | Range | `expect( 0 ).toBeBeforeRange( 1..10 )` | +| `toBeAfterRange()` | Range | `expect( 11 ).toBeAfterRange( 1..10 )` | +| `toBeBounded()` | Range | `expect( 1..10 ).toBeBounded()` | +| `toBeUnbounded()` | Range | `expect( r ).toBeUnbounded()` | +| `toBeHalfBounded()` | Range | `expect( r ).toBeHalfBounded()` | +| `toBeIterable()` | Range | `expect( 1..10 ).toBeIterable()` | +| `toBeAscending()` | Range | `expect( 1..10 ).toBeAscending()` | +| `toBeDescending()` | Range | `expect( 10..1 ).toBeDescending()` | +| `toHaveStep()` | Range | `expect( r ).toHaveStep( 2 )` | +| `toClampTo()` | Range | `expect( 1..10 ).toClampTo( 20, 10 )` | +| `toHavePath()` / `notToHavePath()` | Navigator | `expect( data ).toHavePath( "app.name" )` | +| `toHavePathValue()` / `notToHavePathValue()` | Navigator | `expect( data ).toHavePathValue( "app.name", "TestApp" )` | +| `toHavePathType()` / `notToHavePathType()` | Navigator | `expect( data ).toHavePathType( "app.settings.port", "numeric" )` | +| `toHavePathSatisfying()` / `notToHavePathSatisfying()` | Navigator | `expect( data ).toHavePathSatisfying( "app.name", p -> p == "TestApp" )` | +| `path()` | Navigator | `expect( data ).path( "app.name" ).toBe( "TestApp" )` | +| `queryPath()` | Navigator | `expect( data ).queryPath( "users[*].name" ).toInclude( "Alice" )` | + +## Bug Fixes and Improvements + +### Code Coverage Is Now Opt-In + +The `coverageEnabled` URL parameter in the CFML test runner now defaults to `false` instead of `true`. + +Code coverage requires [FusionReactor](https://www.fusion-reactor.com/), so defaulting it on meant every plain runner hit paid for a feature most runs did not want. If you rely on coverage, enable it explicitly: + +``` +/tests/runner.cfm?coverageEnabled=true +``` + +{% hint style="warning" %} +This is a behavioral change. If you were relying on the implicit default to collect coverage, you must now pass `coverageEnabled=true` or set it in your runner options. +{% endhint %} + +### Date Equality In Assertions + +Equalize assertions now compare date and date/time objects by their instant rather than calling `actual.equals()` blindly. Comparing a `java.util.Date` against a CFML date string, or two date objects of different concrete types, now behaves as expected instead of failing on type identity. + +### Engine Detection + +`isLucee()` returned `true` under BoxLang, because BoxLang registers a `lucee` server scope key for compatibility. Any spec branching or skipping on engine took the Lucee path when running on BoxLang. `isLucee()` now requires the absence of the `boxlang` key. + +### Simple Reporter HTML Escaping + +Bundle and spec names are now HTML-encoded in the Simple reporter, so a test name containing markup no longer breaks the report layout. + +### Full Null Support + +TestBox internals assumed a non-null runtime in several places, producing spurious null-reference errors on engines configured with full null support. Null handling was audited across the runners, coverage service and mock generator, and the configuration is now covered in CI. + +### BoxLang CLI Runner + +Several fixes land for the BoxLang CLI runner: + +- The runner no longer misreads its own script path as a positional bundle argument. +- `KeyNotFoundException [url]` no longer crashes every CLI run on BoxLang 1.17 and later. The `url` scope is not registered in CLI mode, and the runner now params it rather than referencing it directly. +- `GetPageContextResponse()` no longer errors when running BoxLang in Adobe compatibility mode.