update doc

Signed-off-by: Yiheng Wang <[email protected]>

Author: yiheng-wang-nv

acceleration/fast_inference_tutorial/fast_inference_tutorial.ipynb

@@ -203,7 +203,7 @@
     "loader = LoadImaged(keys=\"image\", reader=\"NibabelReader\", to_gpu=True)\n",
     "```\n",
     "\n",
-    "Please note that only NIfTI (.nii, for compressed \".nii.gz\" files, this feature also supports but the acceleration is not significant) and DICOM (.dcm) files are supported for direct GPU data loading.\n"
+    "Please note that only NIfTI (`.nii`, for compressed `.nii.gz` files, this feature also supports but the acceleration is not guaranteed) and DICOM (`.dcm`) files are supported for direct GPU data loading.\n"
    ]
   },
   {
@@ -265,27 +265,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Test data already exists at ./Task03_Liver/imagesTs_nii\n",
-      "Weights already exists at ./model.pt\n",
-      "TensorRT model already exists at ./model_trt.ts\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "root_dir = \".\"\n",
     "torch.backends.cudnn.benchmark = True\n",
     "torch_tensorrt.runtime.set_multi_device_safe_mode(True)\n",
     "device = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
     "train_files = prepare_test_datalist(root_dir)\n",
-    "# since the dataset is too large, the smallest 31 files are used for warm up (1 file) and benchmarking (30 files)\n",
-    "train_files = sorted(train_files, key=lambda x: os.path.getsize(x), reverse=False)[:31]\n",
     "weights_path = prepare_model_weights(root_dir=root_dir, bundle_name=\"wholeBody_ct_segmentation\")\n",
     "trt_model_name = \"model_trt.ts\"\n",
     "trt_model_path = prepare_tensorrt_model(root_dir, weights_path, trt_model_name)"
@@ -609,13 +597,36 @@
     "plt.legend()\n",
     "plt.show()"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Limitations\n",
+    "\n",
+    "Although the optimizations have shown significant improvements in inference time, there are still some limitations to consider:\n",
+    "\n",
+    "1. **TensorRT**: \n",
+    "   - **Model Compatibility**: Not all models are compatible with TensorRT. Models with unsupported layers or operations may not benefit from TensorRT acceleration.\n",
+    "   - **Batch Size**: TensorRT is optimized for larger batch sizes. For very small batch sizes, the overhead of conversion and execution might outweigh the performance gains.\n",
+    "   - **Precision**: While using lower precision (e.g., FP16) can speed up inference, it may lead to a loss in model accuracy, which is critical in medical imaging applications.\n",
+    "\n",
+    "2. **GPU-Based Preprocessing**:\n",
+    "   - **Memory Usage**: The GPU-based preprocessing requires additional GPU memory. This can be a limitation if the available GPU memory is limited.\n",
+    "\n",
+    "3. **GPU Direct Storage (GDS)**:\n",
+    "   - **File Format Support**: Currently, only specific file formats like NIfTI (for compressed `.nii.gz` NIFTI files, this feature also supports but the acceleration is not guaranteed) and DICOM are supported for direct GPU data loading. Other formats may not benefit from this feature.\n",
+    "   - **Small File Acceleration**: For small files, the overhead of conversion and execution might outweigh the performance gains.\n",
+    "\n",
+    "By understanding these limitations, users can better assess when and how to apply these acceleration features effectively in their workflows."
+   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "kvikio_env",
+   "display_name": "monai_tutorial",
    "language": "python",
-   "name": "kvikio_env"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {