This function filters and pools, i.e., row-binds, qualified clients/groups from different sources with an option to summarize by client. Unlike bind_source(), no need to supply variable names; the function will guess what should be included and their names from the supplied definition from build_def(). Whether a client is qualified relies on the flag variables set by define_case(). Therefore, this function is intended to be used only with the built-in define_case() as def_fn in build_def().
Arguments
- data
A list of data.frame or remote table which should be output from
execute_def().- def
A tibble of case definition generated by
build_def().- output_lvl
Either:
"raw" - output all records (default),
or "clnt" - output one record per client with summaries including the date and source of the earliest record ('first_entry_date/src'), the first valid record ('first_valid_date/src'), and the latest record ('last_entry_date/src'). Source-specific record counts are also provided (see the return section).
- include_src
Character. It determines records from which sources should be included. This matters when clients were identified only from, not all, but some of the sources. This choice will not impact the number of client that would be identified but has impact on the number of records and the first/latest entry date. The options are one of:
"all" - records from all sources are included;
"has_valid" - for each client, records from sources that contain at least one valid record are included;
"n_per_clnt" - for each client, if they had fewer than
n_per_clntrecords in a source (seerestrict_n()), then records from that source are removed.
- ...
Additional arguments passing to
bind_source()
Value
A data.frame or remote table with clients that satisfied the predefined case definition. The columns depend on output_lvl.
With output_lvl = "raw", the output is the row-bound records of the qualified clients. Note that these are the records define_case() returned for each source, i.e., only those matching the codes in vals, further subset by include_src; they are referred to as the client's "included records" below. The variables that build_def() named in the definition are kept and renamed to a common set of names:
def- the definition label (def_labinbuild_def()) the record was identified by.src- the source label (src_labsinbuild_def()) the record came from.clnt_id,uid,date_var- the client id, unique record id, and date variable, taken from theclnt_id,uid, anddate_vararguments ofdefine_case().uidanddate_varare present only if they were supplied in the definition.flag_restrict_n,flag_restrict_date- the flags produced byrestrict_n()andrestrict_date(), present only if the definition usedn_per_clntgreater than 1, orapart/within, respectively.flag_valid_record- 1 if the record satisfied all the criteria of the definition, and 0 otherwise. It is taken fromflag_restrict_dateif present, otherwise fromflag_restrict_n, and is filled with 1 for the sources that had no such flag (i.e., sources whose criteria were applied by filtering instead of flagging).src_No- the position of the source in the input list. This is added bybind_source()when binding local data.frames only, and is absent when all inputs are remote tables.
With output_lvl = "clnt", the output has one row per def and clnt_id summarizing that client's included records. The date columns are present only if date_var was supplied for every source in the definition; otherwise they are omitted with a warning.
first_entry_date,first_entry_src- the date of the client's earliest included record, and the source it came from, regardless of whether that record was valid.first_valid_date,first_valid_src- the date of the client's earliest record withflag_valid_recordequal to 1, and the source it came from. Note thatfirst_valid_datecan be later thanfirst_entry_date: the earliest record of a client is often not one of the records that made them a case.last_entry_date,last_entry_src- the date of the client's latest included record, and the source it came from, regardless of whether that record was valid.raw_in_[src]- one column per source, counting the client's included records in that source.valid_in_[src]- one column per source, counting the client's valid records (i.e., the number of flags) in that source.
The source of the first/last record is determined by comparing dates, and ties are broken by the alphabetical order of the source labels, e.g., if a client's earliest date is present in both "dad" and "msp", first_entry_src is "dad". Records with a missing date never contribute to the first/last date or source.
Examples
# toy data
df1 <- make_test_dat()
df2 <- make_test_dat()
# use build_def to make a toy definition
sud_def <- build_def("SUD", # usually a disease name
src_lab = c("src1", "src2"), # identify from multiple sources, e.g., hospitalization, ED visits.
# functions that filter the data with some criteria
def_fn = define_case,
fn_args = list(
vars = starts_with("diagx"),
match = "start", # "start" will be applied to all sources as length = 1
vals = list(c("304"), c("305")),
clnt_id = "clnt_id", # list()/c() could be omitted for single element
# c() can be used in place of list
# if this argument only takes one value for each source
n_per_clnt = c(2, 3)
)
)
# save the definition for re-use
# saveRDS(sud_def, file = some_path)
# execute definition
sud_by_src <- sud_def %>% execute_def(with_data = list(src1 = df1, src2 = df2))
#>
#> Actions for definition SUD using source df1:
#> → --------------Inclusion step--------------
#> ℹ Identify records with condition(s):
#> • where at least one of the diagx, diagx_1, diagx_2 column(s) in each record
#> • contains a value satisfied regular expression: ^304
#> • ignoring case. Use ignore_case = FALSE for a case-sensitive match, which may run faster
#>
#> All unique value(s) and frequency in the result (as the conditions require just one of the columns containing target values; irrelevant values may come from other vars columns):
#> 304 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054
#> 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
#> 3055 3056 3057 3058 3059 999 NAs
#> 1 1 1 1 1 1 1
#> → --------------No. rows restriction--------------
#> ℹ Of the 25 clients in the input, 19 were flagged as 0 by restricting that each client must have at least 2 records
#> → -------------- Output all records--------------
#>
#> Actions for definition SUD using source df2:
#> → --------------Inclusion step--------------
#> ℹ Identify records with condition(s):
#> • where at least one of the diagx, diagx_1, diagx_2 column(s) in each record
#> • contains a value satisfied regular expression: ^305
#> • ignoring case. Use ignore_case = FALSE for a case-sensitive match, which may run faster
#>
#> All unique value(s) and frequency in the result (as the conditions require just one of the columns containing target values; irrelevant values may come from other vars columns):
#> 304 3040 3041 3042 3043 3044 3045 3046 3047 3049 305 3050 3051 3052 3053 3054
#> 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
#> 3055 3056 3057 3058 3059 999 NAs
#> 1 1 1 1 1 1 1
#> → --------------No. rows restriction--------------
#> ℹ Of the 28 clients in the input, 27 were flagged as 0 by restricting that each client must have at least 3 records
#> → -------------- Output all records--------------
# pool results from src1 and src2 together at client level
pool_case(sud_by_src, sud_def, output_lvl = "clnt")
#> # A tibble: 7 × 6
#> def clnt_id raw_in_src1 raw_in_src2 valid_in_src1 valid_in_src2
#> <chr> <int> <dbl> <dbl> <int> <int>
#> 1 SUD 1 1 0 1 0
#> 2 SUD 6 1 1 1 0
#> 3 SUD 13 0 1 0 1
#> 4 SUD 16 1 0 1 0
#> 5 SUD 23 1 1 1 0
#> 6 SUD 38 1 1 1 0
#> 7 SUD 40 1 0 1 0