concept heatmap in category deep learning

This is an excerpt from Manning's book Deep Learning with R. Login to get full access to this book.

Visualizing heatmaps of class activation in an image—Useful for understanding which parts of an image were identified as belonging to a given class, thus allowing you to localize objects in images.

This general category of techniques is called class activation map (CAM) visualization, and it consists of producing heatmaps of class activation over input images. A class activation heatmap is a 2D grid of scores associated with a specific output class, computed for every location in any input image, indicating how important each location is with respect to the class under consideration. For instance, given an image fed into a dogs-versus-cats convnet, CAM visualization allows you to generate a heatmap for the class “cat,” indicating how cat-like different parts of the image are, and also a heatmap for the class “dog,” indicating how dog-like parts of the image are.

2 Function to write a heatmap to a PNG

3 Writes the heatmap

Finally, you’ll use the magick package to generate an image that superimposes the original image with the heatmap you just obtained (see figure 5.32).

Figure 5.32. Superimposing the class activation heatmap on the original picture

Listing 5.44. Superimposing the heatmap with the original picture

library(magick) library(viridis) image <- image_read(img_path) 1 info <- image_info(image) geometry <- sprintf("%dx%d!", info$width, info$height) pal <- col2rgb(viridis(20), alpha = TRUE) 2 alpha <- floor(seq(0, 255, length = ncol(pal))) pal_col <- rgb(t(pal), alpha = alpha, maxColorValue = 255) write_heatmap(heatmap, "elephant_overlay.png", width = 14, height = 14, bg = NA, col = pal_col) image_read("elephant_overlay.png") %>% 3 image_resize(geometry, filter = "quadratic") %>% image_composite(image, operator = "blend", compose_args = "20") %>% plot()

This is an excerpt from Manning's book Deep Learning with Python. Login to get full access to this book.

Visualizing heatmaps of class activation in an image— Useful for understanding which parts of an image were identified as belonging to a given class, thus allowing you to localize objects in images.

This general category of techniques is called class activation map (CAM) visualization, and it consists of producing heatmaps of class activation over input images. A class activation heatmap is a 2D grid of scores associated with a specific output class, computed for every location in any input image, indicating how important each location is with respect to the class under consideration. For instance, given an image fed into a dogs-versus-cats convnet, CAM visualization allows you to generate a heatmap for the class “cat,” indicating how cat-like different parts of the image are, and also a heatmap for the class “dog,” indicating how dog-like parts of the image are.

Finally, you’ll use OpenCV to generate an image that superimposes the original image on the heatmap you just obtained (see figure 5.36).

Figure 5.36. Superimposing the class activation heatmap on the original picture

Listing 5.44. Superimposing the heatmap with the original picture

import cv2 img = cv2.imread(img_path) #1 heatmap = cv2.resize(heatmap, (img.shape[1], img.shape[0])) #2 heatmap = np.uint8(255 * heatmap) #3 heatmap = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET) #4 superimposed_img = heatmap * 0.4 + img #5 cv2.imwrite('/Users/fchollet/Downloads/elephant_cam.jpg', superimposed_img)#6 #1 - Uses cv2 to load the original image #2 - Resizes the heatmap to be the same size as the original image #3 - Converts the heatmap to RGB #4 - Applies the heatmap to the original image #5 - 0.4 here is a heatmap intensity factor. #6 - Saves the image to disk