SenNet + HOA - Hacking the Human Vasculature in 3D

update on 06-JAN
trick to get lb 0.87+

• make a base model in the range of 0.864 (e.g. either seresnext50_32x4d or nextVIT base 2d unet)

• ensemble of the two will give 0.87+

• ensemble improve lb score:

1. detect more instance
2. for an already detected instance improve its boudary to improve the dice score for that instance
3. reduce fp

@hengck23 How do you compute your FPR ? Your definition probably differs from the one I'm used to regarding normalization.

Do you use (pred * (1 - truth)).sum() / truth.sum() ?
Same for TPR, you are using (pred * truth).sum() / truth.sum(), right ?

Thanks!

From my understanding the surface dice metric requires thresholding of predictions hence we cannot directly optimize it.
Applying the metric on probabilities does not seem to yield meaningful results but I did not dig very deeply nor look into the math.

GJ. I believe both sparse and dense have labeling issues (imho, cuz I`m not a doctor). Looked through and found even pretty big vessels missed. Thx for sharing.

maybe i missed it, but any reason just normal unet2d would be suffice? any thoughts on 3D approach?

Hi, I don't understand what does xy,zx,zy mean，could you please explain it, thanks!

did you try using a combination of these strats, or did you get excellent results trying each of them individually?

you can google for fast transformer. i use nextvit, which was used in kaggle rnsa breast mamnographs before. it takes 3to4 hr and can be speedup further by tensort rt to 3 hr

Try fp16 + 2xT4 + nn.DataParallel, for me it's faster than using single P100

The model I choose is very slow [maxvit-small],
and scoring single checkpoint with xy/yz/xz + 5tta takes 8 hours…

should try Next-ViT I think :D

another tip:
my previous two winning kaggle solution has huge shake up. I could win because i kept to the follwing principles. instead of the best parameter (e.g. threshold) for the solution:

• try to improve your solution so that it is not sensitive to thresholds, i.e. performance is optimal over a range of threshold. e.g. best AP verus MAP
• try to improve your solution so that it is not sensitive to data, i.e. there is low variance in accuracy for different validation samples

in summary, in addition to best results, think of ways to

• measure variance
• reduce variance

I'm using a combination of 0.5Dice loss and 0.5BCE loss. The loss is continuously decreasing, but it seems that the surface dice and LB are also increasing. Perhaps Dice loss is more suitable?

Is the model trained on xy xz yz or only inferred on xy xz yz?

Hello, may I ask how you do multi-view training during the training process, is it cycling three axes training in one epoch? I put the data of the three views into a dataset, set the batchsize to 1, and shuffle it, but it doesn't seem to work well. Could you please make a reply? Thank you very much

    def forward(self, batch):
        x = batch['image']
        x = x.expand(-1,3,-1,-1) 
        B, C, H, W = x.shape

        encode = []
        xx = self.stem0(x); encode.append(xx)
        xx = F.avg_pool2d(xx,kernel_size=2,stride=2)
        xx = self.stem1(xx); encode.append(xx)

        e = self.encoder #convnext
        x = e.stem(x);

        x = e.stages[0](x); encode.append(x)
        x = e.stages[1](x); encode.append(x)
        x = e.stages[2](x); encode.append(x)
        x = e.stages[3](x); encode.append(x)
        ##[print(f'encode_{i}', e.shape) for i,e in enumerate(encode)]

        last, decode = self.decoder(
            feature=encode[-1], skip=encode[:-1][::-1]
        )
        ##[print(f'decode_{i}', e.shape) for i,e in enumerate(decode)]
        ##print('last', last.shape)

        vessel = self.vessel(last)
copy

for iteration in range(100):
 # repeat grow up, grow down,   grow up, grow down, ....
 for t in range(750,0,-1): 
    print(t)
    prev_mask  = dense[t+1]
    prev_image = image[t+1]
    curr_image = image[t]

    m = curr_image[1:-1,1:-1]
    prev = prev_mask*prev_image

    th=2
    diff = 0
    diff += np.abs(m-prev[1:-1,1:-1])<th #[ 0, 0]
    diff += np.abs(m-prev[2:  ,1:-1])<th #[ 1, 0]
    diff += np.abs(m-prev[0:-2,1:-1])<th #[-1, 0]
    diff += np.abs(m-prev[1:-1,0:-2])<th #[ 0,-1]
    diff += np.abs(m-prev[2:  ,0:-2])<th #[ 1,-1]
    diff += np.abs(m-prev[0:-2,2:  ])<th #[-1,-1]
    diff += np.abs(m-prev[1:-1,2:  ])<th #[ 0, 1]
    diff += np.abs(m-prev[2:  ,2:  ])<th #[ 1, 1]
    diff += np.abs(m-prev[0:-2,2:  ])<th #[-1, 1]

    grow = diff>1
    #todo: grow across plane here ...

    predict += grow
    #image_show_norm('predict',predict)
    image_show_norm('predict',(predict>0).astype(np.float32))
    cv2.waitKey(0)
copy

@hengck23 - is this the right way to read 3D volume because i get negative stride error

class BuildDataset(torch.utils.data.Dataset):
def __init__(self, img_voxel_paths, msk_voxel_paths=[], transforms=None):
    self.img_voxel = create_3d_voxel(img_voxel_paths)  # Load the 3D voxel for images
    self.msk_voxel = create_3d_voxel_msk(msk_voxel_paths) if msk_voxel_paths else None  # Load the 3D voxel for masks
    self.transforms = transforms

def __len__(self):
    return self.img_voxel.shape[0]  # Number of slices in the voxel

def __getitem__(self, index):
    img = self.img_voxel[index, :, :].copy()  # Extract the 2D slice from the image voxel
    img = np.expand_dims(img, axis=0)  # Add channel dimension

    if self.msk_voxel is not None:
        msk = self.msk_voxel[index, :, :].copy()  # Extract the 2D slice from the mask voxel
        msk = np.expand_dims(msk, axis=0)  # Add channel dimension

        if self.transforms:
            data = self.transforms(image=img, mask=msk)
            img = data['image']
            msk = data['mask']

        return torch.tensor(img, dtype=torch.float32), torch.tensor(msk, dtype=torch.float32)
    else:
        if self.transforms:
            data = self.transforms(image=img)
            img = data['image']

        return torch.tensor(img, dtype=torch.float32)
copy

how to properly design 3d solution.

if you cannot fit a single 3d net work (because of memory GPU constraint),
you properly need at least 2 net (of different hierarchy scale).

you should try these pretrain 3d unet
https://github.com/Tencent/MedicalNet

i don't train in kaggle. i use my local machine

take a look at this thread: https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/453986

@hengck23 The vessels only look like a hole when viewed by slices, I suppose we actually won't see the faded/broken vessels in the slice (your image on the left), is that right? And the stuff inside, did you find that's for broken vessels only or it's generic?

this reminds me of 3d SAM (segment anything model)
if so, we can use self-supervised learning to train the encoder

the seed will be prompt.

the decoder is to perform floodfill segmentation

Thanks for putting this together, I was just looking at doing something like this yesterday before getting pulled in to holiday shenanigans. I re-scaled the data back to the full image size, RLE encoded it, and saved it back to the training csv for anyone interested in using it without resizing on load: https://www.kaggle.com/datasets/squidinator/sennet-hoa-kidney-13-dense-full-kidney-masks

HINT: it is easier to check if this is kidney 5 or 6

i think there is offset bug in annotion for kidney2

good idea👍

" you generate the kidney's mask?"
i label a few and train a model to label the rest.

you should try the kaggle metric (surface dice) on your validation set as well.
iou is not good enough to capture cv/lb correlation

try lower threshold as well