UDMT tutorial

GUI tutorial

1. Start UDMT in the terminal

Once you have UDMT installed, start by opening a terminal. Activate the environment and download the codes with:
```
$ source activate udmt
$ git clone https://github.com/cabooster/UDMT
$ cd UDMT/
```
To launch the GUI, simply enter in the terminal:
```
$ python -m udmt.gui.launch_script
```

Pre-trained models will be downloaded automatically before launching the GUI. Alternatively, you can manually download the model and place it in the specified location.

Model name	Location
trdimp_net_ep.pth.tar	`./udmt/gui/pretrained`
XMem.pth	`./udmt/gui/tabs/xmem/saves`
sam_vit_b_01ec64.pth	`./udmt/gui/tabs/xmem/sam_model`

2. Create a new project

Click the Create New Project button.
Set the Project Path and Project Name.
Add videos by clicking the Browse Videos button, then select the folders containing your videos. By default, the selected videos will be copied to the project path.
Finally, click the Create button to complete this process. The project you created will open automatically.

Open an existing project

To reopen a project, click the Load Project button on the home page.
Navigate to your project folder and select the config.yaml file.

Directory structure

Click to unfold the directory tree


UDMT
|-- config.yaml
|-- videos
    |-- video1.mp4
    |-- video2.mp4
|-- training-datasets
    |-- video1
        |-- img
        	|-- ...
        |-- label
        	|-- ...
|-- models
    |-- video1
        |-- DiMPnet_ep0018.pth.tar
        |-- DiMPnet_ep0019.pth.tar
        |-- DiMPnet_ep0020.pth.tar
|-- tracking-results
    |-- video1
        |-- video1-whole-filter5.mp4
        |-- video1-whole-filter5.npy
        |-- tmp-videos
        	|-- ...
|-- tmp
    |-- video1
        |-- evaluation_metric.json
        |-- evaluation_metric_for_test.json
        |-- evaluation_metric_for_train.json
        |-- extracted-images
            |-- start_pos_array.txt
            |-- 00000.jpg
			|-- ...
        |-- images
        	|-- ...
        |-- masks
        	|-- ...
        |-- test_set_results
        	|-- ...
        |-- train_set_results
        	|-- ...

Folder description

project_name/videos: Path to save the videos to be processed.

project_name/training-datasets: Path to save the training datasets.

project_name/models: Path to save the models.

project_name/tracking-results: Path to save the tracking results, including .npy files and .MP4 files showing the tracking trajectories.

project_name/tmp: Path to save temporary files, including images used during tracking initialization, extracted frames, and files generated during automatic parameter tuning.

3. Tracking initialization

Click the Select Videos button to choose a video you want to process (you can select videos from the videos folder in your project file).
After selecting the video, click the Launch Tracking Initialization GUI button to open the sub-GUI.
In the sub-GUI, click on the centroids of all the animals you want to track.
Click Forward Propagate to start the foreground extraction process.
Wait until foreground extraction is completed for all frames. Once finished, close the sub-GUI to proceed to the next tab.

4. Create training dataset

Click the Select Videos button to choose a video you want to process (if you have already selected videos in the previous tab, you don’t need to select again).
After selecting the video, choose the parameters for preprocessing the videos.
Click the Create Training Dataset button to start the creation process.
During the process, you can click the Show Video button at any time to visualize the creation progress.
Once the creation is complete, a popup message will appear, notifying you to proceed to the next tab.

5. Train network

Click the Select Videos button to choose a video you want to process (if you have already selected videos in the previous tab, you don’t need to select them again).
Set the training parameters:

Batch Size: Select a larger batch size if you have multiple GPUs. If you encounter a “CUDA out of memory” error, reduce the batch size.

Number of Workers: Set to 0 on Windows; on Linux, you can set it to 8.

Max Training Epochs: The default is 20 epochs. This value doesn’t need to be changed unless necessary.
Click the Train Network button to start the training process.

6. Analyze videos

Select a video you want to process.
Choose the parameters for preprocessing the videos and set the Filter Size for post-processing smooth trajectory filtering.
Click the Analyze Videos button to start the tracking process.
During the tracking, you can click the Show Video button at any time to visualize the tracking process. (This includes automatic parameter tuning and the final tracking process.)
Once the tracking is complete, the result files will be automatically saved in the tracking-results folder.
A popup message will appear, notifying you to proceed to the next tab.

7. Visualize tracks

Select a video you want to process.
Choose the parameters for post-processing the videos.
Click the Launch Track Visualization GUI button to start the visualization.
In the pop-up sub-GUI: The left side shows the current frame and the right side displays the trajectory XY coordinates that change with each frame.
You can click the Frame bar to visualize each frame.
Adjust the Dot Size bar to change the size of the dots marking positions in the image.