Visualization of model evaluation (image, true mask, predicted mask)

[HripsimeS]

Hello. I am trying to do a visualization after the evaluation on original images/masks. For example I have 10 images/masks that I would like the test my model and then would like to plot results of all of them with images, true masks and predicted masks. The solution below does not work efficiently as it is plotting only one case (image, true mask and predicted mask) multiple times. Let me know if you have more efficient solution for this visualization. Look forward to hearing from you.

loader = LoadImage()
original_image = loader(test_data[0]["image_meta_dict"]["filename_or_obj"])[0]

with torch.no_grad():
for i, test_data in enumerate(test_loader):

    plt.figure("check", (18, 6))
    
    plt.subplot(1, 3, 1)
    plt.title(f"image {i}")
    plt.imshow(original_image[:, :, 250], cmap="gray")
    
    plt.subplot(1, 3, 2)
    plt.title(f"mask {i}")
    plt.imshow(test_labels[0].detach().cpu()[1, :, :, 250])
    
    plt.subplot(1, 3, 3)
    plt.title(f"prediction {i}")
    plt.imshow(test_outputs[0].detach().cpu()[1, :, :, 250])
    
    plt.show()
    if i == 2:
        break

P.S. If for evaluation of model I am using 10 images/masks, the metric_org is returning the average of all 10 scores? If yes, then how I can check dice score of each case individually? This is the target part:

dice_metric(y_pred=test_outputs, y=test_labels)
metric_org = dice_metric.aggregate().item()
dice_metric.reset()

[yiheng-wang-nv]

Hi @HripsimeS , please put original_image = loader(test_data[0]["image_meta_dict"]["filename_or_obj"])[0] within your for loop:

for i, test_data in enumerate(test_loader):
    original_image = loader(test_data[0]["image_meta_dict"]["filename_or_obj"])[0]
   ...

[HripsimeS]

@yiheng-wang-nv thank you! And what is wrong with data structure, why test_labels[0] and test_outputs[0] are not working efficiently inside of loop in "51st cell", eventually plotting results of the last case?

[yiheng-wang-nv]

In the 51st cell, test_labels and test_outputs are not defined. What you did is just to load the data:

with torch.no_grad():
    for i, test_data in enumerate(test_loader):
        loader = LoadImage()
        original_image = loader(test_data["image_meta_dict"]["filename_or_obj"])[0]

[HripsimeS]

@yiheng-wang-nv ok thanks. So I need to define them inside of the loop with the following code?

test_data = [post_transforms(i) for i in decollate_batch(test_data)]
test_outputs, test_labels = from_engine(["pred", "mask"])(test_data)

If I need to use this code inside of loop, then I received following error.
KeyError: 'Key pred of transform EnsureTyped was missing in the data and allow_missing_keys==False.'

Please let me know how would be better to define test_labels and test_outputs inside of loop to be able to see unique true and predicted masks of each case separately. Thanks a lot in advance for your help!

[yiheng-wang-nv]

HI @HripsimeS , I think the tutorial https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/spleen_segmentation_3d.ipynb is clear. Please check the last few cells carefully.

[HripsimeS]

@yiheng-wang-nv thank you very much for useful tips! I could succeed to plot all unique test cases with their image, true mask and predicted mask. I really appreciate a professional Q&A support of MONAI and if there is a place where I can leave my feedback, I will do with a great pleasure :) For other users, if you want to see a solution to do a visualization of all test cases you can find below:

model.load_state_dict(torch.load(os.path.join(root_dir, "best_metric_model.pth")))
model.eval()
with torch.no_grad():
for i, test_data in enumerate(test_loader):
test_inputs = test_data["image"].to(device)
roi_size = (160, 160, 160)
sw_batch_size = 4
test_data["pred"] = sliding_window_inference(test_inputs, roi_size, sw_batch_size, model)
test_data = [post_transforms(i) for i in decollate_batch(test_data)]
test_outputs, test_labels = from_engine(["pred", "mask"])(test_data)
original_image = loader(test_data[0]["image_meta_dict"]["filename_or_obj"])[0]
plt.figure("check", (18, 6))
plt.subplot(1, 3, 1)
plt.title(f"image {i}")
plt.imshow(original_image[:, :, 250], cmap="gray")
plt.subplot(1, 3, 2)
plt.title(f"mask {i}")
plt.imshow(test_labels[0].detach().cpu()[2, :, :, 250])
plt.subplot(1, 3, 3)
plt.title(f"prediction {i}")
plt.imshow(test_outputs[0].detach().cpu()[2, :, :, 250])
plt.show()
if i == 4:
break