dataset: synthetic from PANTHER #25
Conversation
Co-Authored-By: Neha Talluri <[email protected]> Co-Authored-By: Oliver Faulkner Anderson <[email protected]> Co-Authored-By: Altaf Barelvi <[email protected]>
| @@ -0,0 +1,100 @@ | |||
| pathways = ["Apoptosis_signaling", "B_cell_activation", | |||
There was a problem hiding this comment.
Each of the files have this variable. I think we should have it only in the snakefile and send this list to each of the files that use this pathway list
| "FGF_signaling", "Interferon_gamma_signaling", | ||
| "JAK_STAT_signaling", "VEGF_signaling"] | ||
| # TODO: deduplicate this from thresholding scripts by passing it in? | ||
| thresholds = [1, 100, 200, 300, 400, 500, 600, 700, 800, 900] |
There was a problem hiding this comment.
similar thing for the thresholds
|
A question I’d appreciate feedback on: Currently, we generate separate source, target, and prize files for each pathway, but we combine all pathways into each thresholded interactome. Should we also create a combined list of sources, targets, and prizes? Should we also combine the gold standard as well? Or would it be better to keep separate interactomes for each individual pathway (keep it the way it is)? |
|
We should have separate gold standards. |
|
When this is reviewed (or before) we should do tests to see how connected the networks are after thresholding, adding back the pathway data, and removing proteins that don't have uniprot ids. |
|
Also there is a chance we can use more panther pathways, we should look to see what else we can use from pathway commons. |
|
@oliverfanderson @ctrlaltaf For the gold standard nodes (and potentially the edges), should we exclude source, target, and prize nodes when defining it? Currently, it looks like we’re including these nodes in the gold standard for each pathway. These nodes overlap with the gold standard, but that overlap should happen naturally, not by construction/being predefined. I’m concerned this could inflate our precision and recall metrics, because of a form of data leakage. |
Plan to keep all of them in the gold standard. But update the evaluation code to deal with the sources/targets/prizes being in the gold standard and shown as a different baseline where those are all set as frequency 1.0. |
|
Should we also consider how sparse an interactome becomes after applying a threshold to the STRING interactome? When we filter by size, we implicitly accounting for the decrease in graph density as well. Would it make more sense to treat size and density as separate variables when evaluating performance? However, does testing for density even matter in this context; are there any interactomes that aren’t already highly connected? I’m thinking we should first threshold the interactomes, then select only those that are highly connected (e.g., density ≥ 0.85). From that subset, we could choose a few to represent different size scales. |
This does not add anything to
config/*.yaml.Co-Authored-By: Neha Talluri [email protected]
Co-Authored-By: Oliver Faulkner Anderson [email protected]
Co-Authored-By: Altaf Barelvi [email protected]