Skip to contents

List the units a coding run failed on

qlm_failures.Rd

Reports which units of a qlm_coded object produced no usable coding, and why. A run over a real corpus rarely comes back complete: requests fail, providers refuse a text or reject it on length, and an endpoint can accept a schema and then ignore it. The object records all of this, in an .error list-column and as NA values, but nothing about its shape says how many units were affected, and for an array-valued property the obvious check does not work (see below). print() uses the same test to report a count.

Usage

qlm_failures(x)

Arguments

x

A qlm_coded object.

Value

A tibble with one row per failed unit and columns .id, reason (a character description) and .error (the recorded condition, or NULL for a unit that failed by returning NA for every required property). Zero rows when every unit was coded.

Details

A unit counts as failed when either of two things holds:

  • it carries an .error. qlm_code() records one when the request failed, when the provider cut the response off or withheld it (see the Truncated responses section of qlm_code()), when the response held no JSON or JSON that did not parse, and when its JSON did not match the codebook schema, naming the offending path; or

  • every required scalar property of the codebook schema is NA for it. That is how an object coded before every response was validated shows a response the endpoint sent without honouring the schema; a run coded since records such a unit under the first rule.

Array and nested-object properties are not consulted. After conversion, a missing array and a schema-valid empty one are the same zero-length list-column cell, so neither is.na() nor a row count on such a column can tell failure from a unit to which nothing applied. For a codebook whose required properties are all arrays or nested objects, only .error identifies failed units.

See also

qlm_backfill() to re-code the failed units; qlm_code(), whose default on_error = "continue" attempts every unit and leaves the failed ones in the object rather than stopping the run; accessors for the other accessor functions.

Examples

examples <- readRDS(system.file("extdata", "example_objects.rds", package = "quallmer"))

# A complete run: zero rows
qlm_failures(examples$example_coded_sentiment)
#> # A tibble: 0 × 3
#> # ℹ 3 variables: .id <int>, reason <chr>, .error <list>

# A run that came back incomplete: a request that timed out, and responses
# cut off at max_tokens
qlm_failures(examples$example_coded_incomplete)
#> # A tibble: 4 × 3
#>   .id          reason                                                 .error    
#>   <chr>        <chr>                                                  <list>    
#> 1 3150_1.txt   "The response used the whole max_tokens limit of 90 a… <qllmr_t_>
#> 2 3918_1.txt   "The response used the whole max_tokens limit of 90 a… <qllmr_t_>
#> 3 7227_7.txt   "Failed to perform HTTP request.\nCaused by error:\n!… <httr2_fl>
#> 4 11413_10.txt "The response used the whole max_tokens limit of 90 a… <qllmr_t_>

# The same run after qlm_backfill(): the timed-out unit recovered, the
# cut-off ones left alone, since re-sending the request cannot fix them
qlm_failures(examples$example_coded_backfilled)
#> # A tibble: 3 × 3
#>   .id          reason                                                 .error    
#>   <chr>        <chr>                                                  <list>    
#> 1 3150_1.txt   The response used the whole max_tokens limit of 90 an… <qllmr_t_>
#> 2 3918_1.txt   The response used the whole max_tokens limit of 90 an… <qllmr_t_>
#> 3 11413_10.txt The response used the whole max_tokens limit of 90 an… <qllmr_t_>