SenNet + HOA - Hacking the Human Vasculature in 3D

maybe i missed it, but any reason just normal unet2d would be suffice? any thoughts on 3D approach?

you can google for fast transformer. i use nextvit, which was used in kaggle rnsa breast mamnographs before. it takes 3to4 hr and can be speedup further by tensort rt to 3 hr

Try fp16 + 2xT4 + nn.DataParallel, for me it's faster than using single P100

The model I choose is very slow [maxvit-small],
and scoring single checkpoint with xy/yz/xz + 5tta takes 8 hours…

should try Next-ViT I think :D

another tip:
my previous two winning kaggle solution has huge shake up. I could win because i kept to the follwing principles. instead of the best parameter (e.g. threshold) for the solution:

• try to improve your solution so that it is not sensitive to thresholds, i.e. performance is optimal over a range of threshold. e.g. best AP verus MAP
• try to improve your solution so that it is not sensitive to data, i.e. there is low variance in accuracy for different validation samples

in summary, in addition to best results, think of ways to

• measure variance
• reduce variance

I'm using a combination of 0.5Dice loss and 0.5BCE loss. The loss is continuously decreasing, but it seems that the surface dice and LB are also increasing. Perhaps Dice loss is more suitable?

Is the model trained on xy xz yz or only inferred on xy xz yz?

Hello, may I ask how you do multi-view training during the training process, is it cycling three axes training in one epoch? I put the data of the three views into a dataset, set the batchsize to 1, and shuffle it, but it doesn't seem to work well. Could you please make a reply? Thank you very much

for iteration in range(100):
 # repeat grow up, grow down,   grow up, grow down, ....
 for t in range(750,0,-1): 
    print(t)
    prev_mask  = dense[t+1]
    prev_image = image[t+1]
    curr_image = image[t]

    m = curr_image[1:-1,1:-1]
    prev = prev_mask*prev_image

    th=2
    diff = 0
    diff += np.abs(m-prev[1:-1,1:-1])<th #[ 0, 0]
    diff += np.abs(m-prev[2:  ,1:-1])<th #[ 1, 0]
    diff += np.abs(m-prev[0:-2,1:-1])<th #[-1, 0]
    diff += np.abs(m-prev[1:-1,0:-2])<th #[ 0,-1]
    diff += np.abs(m-prev[2:  ,0:-2])<th #[ 1,-1]
    diff += np.abs(m-prev[0:-2,2:  ])<th #[-1,-1]
    diff += np.abs(m-prev[1:-1,2:  ])<th #[ 0, 1]
    diff += np.abs(m-prev[2:  ,2:  ])<th #[ 1, 1]
    diff += np.abs(m-prev[0:-2,2:  ])<th #[-1, 1]

    grow = diff>1
    #todo: grow across plane here ...

    predict += grow
    #image_show_norm('predict',predict)
    image_show_norm('predict',(predict>0).astype(np.float32))
    cv2.waitKey(0)
copy

how to properly design 3d solution.

if you cannot fit a single 3d net work (because of memory GPU constraint),
you properly need at least 2 net (of different hierarchy scale).

    def forward(self, batch):
        x = batch['image']
        x = x.expand(-1,3,-1,-1) 
        B, C, H, W = x.shape

        encode = []
        xx = self.stem0(x); encode.append(xx)
        xx = F.avg_pool2d(xx,kernel_size=2,stride=2)
        xx = self.stem1(xx); encode.append(xx)

        e = self.encoder #convnext
        x = e.stem(x);

        x = e.stages[0](x); encode.append(x)
        x = e.stages[1](x); encode.append(x)
        x = e.stages[2](x); encode.append(x)
        x = e.stages[3](x); encode.append(x)
        ##[print(f'encode_{i}', e.shape) for i,e in enumerate(encode)]

        last, decode = self.decoder(
            feature=encode[-1], skip=encode[:-1][::-1]
        )
        ##[print(f'decode_{i}', e.shape) for i,e in enumerate(decode)]
        ##print('last', last.shape)

        vessel = self.vessel(last)
copy

HINT: it is easier to check if this is kidney 5 or 6

@hengck23 - is this the right way to read 3D volume because i get negative stride error

class BuildDataset(torch.utils.data.Dataset):
def __init__(self, img_voxel_paths, msk_voxel_paths=[], transforms=None):
    self.img_voxel = create_3d_voxel(img_voxel_paths)  # Load the 3D voxel for images
    self.msk_voxel = create_3d_voxel_msk(msk_voxel_paths) if msk_voxel_paths else None  # Load the 3D voxel for masks
    self.transforms = transforms

def __len__(self):
    return self.img_voxel.shape[0]  # Number of slices in the voxel

def __getitem__(self, index):
    img = self.img_voxel[index, :, :].copy()  # Extract the 2D slice from the image voxel
    img = np.expand_dims(img, axis=0)  # Add channel dimension

    if self.msk_voxel is not None:
        msk = self.msk_voxel[index, :, :].copy()  # Extract the 2D slice from the mask voxel
        msk = np.expand_dims(msk, axis=0)  # Add channel dimension

        if self.transforms:
            data = self.transforms(image=img, mask=msk)
            img = data['image']
            msk = data['mask']

        return torch.tensor(img, dtype=torch.float32), torch.tensor(msk, dtype=torch.float32)
    else:
        if self.transforms:
            data = self.transforms(image=img)
            img = data['image']

        return torch.tensor(img, dtype=torch.float32)
copy

good idea👍

" you generate the kidney's mask?"
i label a few and train a model to label the rest.

you should try the kaggle metric (surface dice) on your validation set as well.
iou is not good enough to capture cv/lb correlation

try lower threshold as well

this reminds me of 3d SAM (segment anything model)
if so, we can use self-supervised learning to train the encoder

the seed will be prompt.

the decoder is to perform floodfill segmentation

i think there is offset bug in annotion for kidney2

Thanks for putting this together, I was just looking at doing something like this yesterday before getting pulled in to holiday shenanigans. I re-scaled the data back to the full image size, RLE encoded it, and saved it back to the training csv for anyone interested in using it without resizing on load: https://www.kaggle.com/datasets/squidinator/sennet-hoa-kidney-13-dense-full-kidney-masks

are you planning to apply a patch based approach? or taking the whole image into account? will we be able to fit the whole image into memory with the kaggle infrastructure?

i am surprised that the deep CNN nework can somhow differentate between venous and arterial (target) vessel from one imgae.

can anyone explain why?
(e.g. they have different apperance due to dye/contrast when the Hip CT scan is taken ??? or arterial vessel have thicker wall ???)

I trained initially on kidney 1, by adding also kidney 3 improved my LB from 0.65 -> 0.7. I do not know how to compare their resolutions but looks like they have pretty similar ones with 50um and 50.16um. The question would be if LB is also 50um and private something completly different