We find and classify the normal epithelial, malignant epithelial and lymphocyte cells in subsections of H&E stained histology images taken at 20x magnification. Example with the malignant cells labelled in red:
Some visualizations of the data, as well as some problem areas are in exploring_data notebook.
Procedure: The general outline of our method is:
- Classifier - Train a classifier to detect if an image is centered on a normal, malignant or lymphocyte nuclei, or else not on any nuclei. We create a data set by selecting square regions centered at the nuclei centers. We test different fully convolutional and regular CNNs. Augmentation is with flips, rotations and cropping. (code)
- Heat Maps - Apply the classifier to the image, outputting a 4 dimensional heatmap of the probability of each cell class. (code)
- Cell Locations - Based on the heat maps, output locations and classifications for the cells using non-maximum supression
Some of the testing is included in this notebook
- This model is also tested on a public lymphocyte data set in this notebook, but without any fine tuning it is not effective.
- We also test training it on the lymphocyte dataset with better results in this notebook