Skip to main content

Checks

Checks​

Checks are Starlark objects built in to the smart_test module for writing assertions. Checks have names which identify them and can generate errors. In test results, any check without any associated errors is considered a pass and any check with associated errors is considered a fail.

Creating Checks​

When a Check is created, it should be named, for example

ck = smart_test.check("check-name")

The name of ck is the string "check-name" and this serves as the identity of the check.

Additionally, the smart_test.check function has the same signature as the Starlark string .format method, allowing the generation of different check names dynamically:

# generate checks dynamically using formatting
fields = ["f1", "f2"]
for field in fields:
    ck = smart_test.check("check-{}", field)

Check Errors​

Checks have an error method which records an error associated with the check's name.

In Test results, Checks which did not generate any errors are considered to have passed, while those which did generate at least one error are considered to have failed.

ckPass = smart_test.check("pass-check")
ckFail = smart_test.check("fail-check")
if False:
    ckPass.error("never fail")
else:
    ckFail.error("always fail")

Formatting Errors​

The .error method of a check accepts arguments in the style of Starlark's string .format method, for example

ck.error("something went wrong")
ck.error("{} not present", user)
ck.error("{user} not present, user="Jane")

Check Error Attributes​

A check carries with it a set of error attributes which can be useful for sharing attributes across calls to .error().

In the example below, both call sites to .error will be associated with the user passed into f.


ck = smart_test.check("user-ok")
def f(ck, user):
    ck = ck.errorAttrs(user: user)
    resp = http.get("http://app.namespace.svc:8080/users" + user)
    if resp.status_code != 200:
        ck.error("bad status code: {}", resp.status_code)
        return
    j = resp.json()
    if j["name"] != user:
        ck.error("unexpected name {}", j["name"])

Additionally, attributes can take any value which converts to json.

Check Name Identity​

Two distinct check objects in the script with the same name will be considered identical in the Test results.

ck1 = smart_test.check("check-name")
ck2 = smart_test.check("check-name")

def f(ck):
    ck.error("f was called")

f(ck1) # results are identical to calling f(ck2)