Where & Why: Interpreting Convolutional Neural Networks Trained to Identify Strong Gravitational Lenses
Talk, National Astronomy Meeting, Coventry, UK
In the near future Euclid will discover ~10^5 strong gravitational lenses from the ~10^9 objects it will detect during its mission. This will be far too much data for humans to process alone. In recent years machine learning models have been trained to find gravitational lenses (Metcalf et al. 2019). These models have performed well and have detected new gravitational lenses (e.g. Huang et al. 2019).
The problem is the question “What is being detected?”. Machine learning approaches such as CNNs have been described as ‘Black Boxes’. It is not clear what CNNs respond to in the images. We simply do not know if we are detecting the ‘low hanging fruit’ of gravitational lenses. Do these approaches only detected certain types of gravitational lenses such as blue Einstein rings and miss rare lenses such as red Einstein rings?
Recently, there has been an attempt to understand how CNNs respond to features in the data, using techniques such as Deep Dream (Mordvintsev et al 2015), Grad-CAM (Selvaraju et al. 2017), and Occlusion maps (Zeiler & Fergus 2014). I have therefore applied these advanced interpretability techniques to a Convolutional Neural Network (CNN) I have created to identify strong gravitational lenses in simulated data, in order to understand the features within the image that the CNN has learnt to associate with strong gravitational lenses (Wilde et al 2022).
I demonstrated for the first time that such CNNs do not select against compound lenses (i.e. double source plane systems, or Jackpot lenses), obtaining recall scores as high as 76% for compound arcs and 52% for double rings. I verified this performance using Hubble Space Telescope (HST) and Hyper Suprime-Cam (HSC) data of all known compound lens systems. This implies that existing lens finding methodologies should be able to recover compound lensing systems even without further training, albeit not with 100% recall. With the help of the interpretability tools, it is clear that the network is responding to arcs and colour in the case of lensing systems (in agreement with Jacobs et al. 2022) and isolated systems in the case of non-lenses.
These interpretability techniques have allowed me to infer positional information about the lens detection in the image. Using post-training interpretability methods can help us understand what the CNN has learnt but cannot allow us to influence what the CNN learns. I have therefore also trained a machine learning model to identify if an image contains a gravitational lens and output the position of the lens within the image. The next step is to build this interpretability directly into our machine learning models, because having a single lens classification value between 0 and 1 will not be enough when applied to future surveys. We need to consider adding a positional output to these models to identify if these models have made a sensible classification or if an unexpected feature in the image has triggered a misclassification.