Exclude source codes

[oleg-odysseus]

User story

As a concept-set user

I want to be able to exclude source codes from a given concept-set, which contains Standard concept mapped to this particular source code

So that no events for the excluded source codes make their way into cohort

Comments:

• Problem is that currently included Standard code, would still bring in events, for the excluded source code, even if the source code is explicitly added to the expression with Exclude option enabled.

• Often users will exclude source codes and download a CSV of the set for use in a study and do not realize that those codes are being retained in the included source codes.

Example:

What users typically try to do:

192278 Non-standard code is one of the 164 mapped non-standard codes to 201826

Users try to exclude 192278, but still keep all of the remaining mapped non-standard codes.

What they should be doing is using "Mapped" option

Snomed "Type 2 diabetes mellitus" with mapped option brings in 164 Non-standard concepts + Itself included

If we want to exclude one of the non-standard code, for example

We can see that the resulting Included Concepts are now 1 less → 164 and does not contain the excluded code

However the Standard code is still there, and it will still trigger persons to enter cohort with the Excluded Non-standard code, as it's Standard one will trigger it.

That is a problem.

Here is why:

The resulting concept set contains both standard concepts and non-standard concepts therefore it has to be used in following way in cohort definitions:

Same concept set has to be specified in two instances of:

1. condition occurrence of "mixed concept set"
2. condition occurrence of "Any Condition" (yes, keep it blank here and add additional attribute for source concept)
   condition source concept is "mixed concept set"
   JSON view:

Entry events criteria are combined with logical OR statement and the underlying cohort SQL code looks like this:

SQL equivalent (pseudocode)

select ... from condition_occurrence where condition_concept_id in (Included Concepts: 201826, 44836914, ....)
union all
select from condition_occurrence where condition_source_concept_id in (Included Concepts: 201826, 44836914, ....)

Key items to understand here.

1. condition_concept_id column will never have non-standard concepts, thus they can be ignored
2. condition_source_concept_id column can contain both Non-Standard and Standard concepts (imagine case when Snomed vocabulary is used by source data)
3. Included concepts will NOT contain the excluded 192278 code
   4. There might be records for which the condition_concept_id = 201826 and condition_source_concept_id = 192278 (the excluded non-standard concept)
   In this case according to the above SQL , the record will get selected by it's condition_concept_id and person will be included in the cohort

Implementation options

Option A: Reworking the currently existing "Mapped" option, so that it would no longer include the Standard code itself in the Included codes. Only the mapped Non-Standard codes.
And in addition utilizing the existing ability to explicitly Exclude the unwanted source codes.

To handle this case we can exclude explicitly the Standard concept like this:

Notice that now the included concepts count is 163, they no longer have the Standard code, but now have all the 164 mapped non-standard minus the excluded non-standard.

Problem is there could be multiple of these and it looks too excessive to additionally explicitly exclude the standard code.

Therefore the idea is to change how the current "Mapped" option works, so that it would no longer include the Standard code itself.

So that the goal could be achieved with 2 simple entries:
Standard code with Mapped option
+
Non-standard code with Exclude option

Option B: a feature to check all excluded non-standard codes, to see if they are included by any of the included Standard codes,
if so - completely replace the Included standard code with the mapped non-standard code (user guided process)
(creating a mixed concept-set, to be used twice in CohortBuilder: occurrence of ... + source_concept )

1. A new button "Exclude source concepts" (next to Optimize) would be added

2. Pressing pops up a dialog window, displaying excluded source codes and what they are mapped by (similar to Non-Standard to Standard conversion feature)

Note: here we have 2 option how to do the replacement

Option B.1 - remove from the expression the Standard code completely and add to the expression the non-excluded mapped Non-standards
(here additionally we can consider if we should remove the excluded Non-standards or keep them to explicitly tell the viewer that it was intentional)

Option B.2 - replace the Standard code (or simply add the "Mapped" option to it) plus add an additional copy of the standard code with Exclude option.

Option C (final version after more discussions):

Assumptions:

1. We do not want to change how current Expression options (Exclude, Descendant, Mapped) work.

2. We want a way in a concept set to identify Non-standard concepts that are marked with Exclude, still being brought into the set by a non-excluded Standard concepts

3. We want to allow automated resolution of such cases with one of the approaches, selected by user:

Implementation:

Introduce a new feature (similar to Non-Standard to Standard conversion, or Optimize feature), name TBD (for example "Check Excluded")

Inputs would be a concept set, with Non-Standard concepts marked with Exclude and an non-excluded Standard concept(s).

Note: Descendant and Mapped do not matter (should not affect triggering the condition)
User would see which excluded concept is being brought in by which standard code(s)

Note: in this example, if we selected/focused excluded code E11.1, which is mapped to 2 standard concepts, we would have to add 2 Exclude and 2 Mapped entries.

The user would select which cases and how would he wants to address the resolution.

3.1) Approach C.1 - enabling Mapped option on the standard concept in the expression AND adding the standard code with Exclude option to the expression
Example:

3.2) Approach C.2 - by replacing the standard concept in the expression with all the non-standard codes mapped to it

Example: