Applying deep learning on histological images of lymph tissues



Table of content

  1. Abstract
  2. Related works
  3. Methodology
  4. Experiment Setup and Implementation
  5. Results and Analysis
  6. Conclusion
  7. Publications
  8. Links


Histopathology whole slide images contain valuable information for many medical applications such as cancer diagnosis. The recent advancement of Deep Learning (particularly Transfer Learning) allowed using the computer systems for efficient and effective diagnosis in histopathology imaging. However, these approaches have some limitations:
a) the pre-trained transfer learning models lack medical imaging features.
b) These algorithms show reasonably high accurate predictions but lack transparency.
c) limited labeled images in the histopathology images.

In this study, we address the issues mentioned above. First, we briefly discuss two histopathology imaging approaches. We mainly focus on transfer-learning approaches in medical imaging, particularly the need for an efficient transfer-learning approach for histopathology imaging. We also address explainable AI approaches and the need to ensure the reliability of AI in computational histopathology. A brief detail of self-supervised learning approaches is included to provide the details of handling unlabelled datasets. Finally, a discussion is provided at the end of the study

The literature review provides a comprehensive overview of existing research relevant to histopathology, DL, and associated challenges. Key topics covered include whole slide images (WSI), patch-based analysis, transfer learning (TL) in medical imaging, and explainable AI (XAI). Notable studies are referenced to support the proposed approach, including research showcasing the potential of DL algorithms in detecting metastases, the effectiveness of TL in medical imaging, and the importance of XAI in enhancing model interpretability. The related works section highlights the significance of features learned from medical images, the limitations of traditional staining techniques, and the need for domain-specific datasets in transfer learning. Additionally, the review emphasizes the importance of XAI in medical imaging, providing interpretable explanations and insights into DL model decisions. Overall, the related works section sets the stage for the proposed research by summarizing key findings and gaps in existing literature.


The proposed methodology outlines a multi-stage approach to enhance the accuracy of DL models in histopathological image analysis. The research focuses on addressing challenges associated with traditional staining techniques (H&E) and the cost implications of resorting to immunohistochemistry (IHC). The key elements of the methodology include:

Methodology of the study

  1. Collection of Diverse Medical Image Datasets:

    • Compilation of various medical image datasets, including retinal images, histopathology photos, and other relevant images from the same field.
    • Ensuring diversity in datasets to represent a wide range of properties and attributes.
  2. Model Creation and Pre-training:

    • Training a DL model using the gathered medical image datasets. The specific architecture and classification layer used in this initial training phase are not crucial as they will be adjusted in subsequent stages.
    • This phase serves as pre-training to provide a foundation for further fine-tuning.
  3. Fine-tuning for IHC Stained Histopathology:

    • Utilizing a labeled dataset specific to IHC-stained histopathology images to fine-tune the pre-trained model from the previous stage.
    • Customizing the model to specialize in IHC-stained histopathology categorization using knowledge gained from various medical picture datasets.
  4. Fine-tuning for H&E Images:

    • Using a labeled dataset of H&E (Hematoxylin and Eosin) stained histopathology images to further fine-tune the model.
    • This additional fine-tuning step aims to improve the model’s performance specifically for H&E images, commonly used in histopathology analysis.
  5. Comparison of Model Accuracies:

    • Evaluating and comparing the accuracies achieved by each model fine-tuned for IHC-stained and H&E-stained histopathology images separately.
    • Providing insights into the effectiveness of the two fine-tuning approaches for different histopathology image types.

Experiment Setup and Implementation

Dataset Selection:

Model Selection and Pre-training:

Fine-tuning for IHC Stained Histopathology:

Fine-tuning for H&E Images:

Model Evaluation and Comparison:

Integration of Explainable AI (XAI):

Explained AI in cancer cell detection

Patching Mechanisms and Ensemble Methods:

Ablation Study:

Results and Analysis

  1. Benchmark Results Using VGG16, VGG19, ResNet, and EfficientNet:

    • Conducted benchmark experiments to evaluate the performance of VGG16, VGG19, ResNet, and EfficientNet on the histopathology image dataset.
    • Analyzed key metrics, including accuracy, precision, recall, and F1 score, to assess the effectiveness of each model.
    • Identified the most promising DL model based on overall performance, sensitivity to metastatic tissues, and computational efficiency.
    • The comparative analysis provides insights into the strengths and weaknesses of each model, guiding further experimentation in the proposed methodology. Result for various model accuracy for transfer learning analysis
  2. SHAP and LIME Test Results for Model Interpretability:

    • Applied SHAP (Shapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) to interpret and explain the decisions made by the selected DL model.
    • Analyzed the results of SHAP and LIME tests to enhance the interpretability of the model’s predictions.
    • Examined specific instances where SHAP and LIME provided insights into the features influencing the model’s decision-making process.
    • The interpretability achieved through SHAP and LIME contributes to user trust and understanding of DL model decisions in the context of histopathological image analysis. SHAP positive image result SHAP Negative image result LIME positive image result LIME Negative image result


The proposed study addresses critical challenges in histopathological image analysis, specifically focusing on the detection of metastatic tissues using deep learning (DL) algorithms. The key findings and conclusions drawn from the research are as follows:

Significance of DL in Histopathology:

Transfer Learning and Domain Specificity:

XAI for Model Interpretability:

Proposed Methodology:

Future Directions:

In conclusion, the proposed study positions DL algorithms as valuable tools in histopathological image analysis, emphasizing the importance of domain-specific pre-training and XAI for transparency. The methodology offers a comprehensive approach to address challenges in cancer diagnosis, paving the way for advancements in the field.