How to use my own dataset to train my model?

[hefei1019]

Hi, I am a student who learning incremental learning. I have use this code to train cifar-100 dataset, now I want to train my own dataset by this model. I meet some problem, can you tell me how to change this code to make it to be suitable for my dataset which is .mat form and it is necessary to divide the dataset to train and test file? Thanks for you reading, and I will expect you reply.

[lrzpellegrini]

Hi @hefei1019, I never worked with .mat files, but it should be easy to load and convert them to PyTorch Tensors (see https://discuss.pytorch.org/t/how-to-read-a-dataset-in-mat-form-in-pytorch/71668/2).

The dataset must be split in train and test sets before creating the NCProtocol:

icarl-pytorch/main_icarl.py

Line 98 in b9ad40d

protocol = NCProtocol(CIFAR100('./data/cifar100', train=True, download=True, transform=transform),

Another thing you may want to change is the model (https://github.com/lrzpellegrini/icarl-pytorch/blob/master/models/icarl_net.py), which currently comes with some hard-coded dimensions which are specific to CIFAR.

Consider that this code is very specific for the CIFAR dataset and may not work with other datasets in general. I recommend using the implementation found in the Avalanche framework which is more stable and flexible: https://github.com/ContinualAI/avalanche/blob/master/examples/icarl.py

[hefei1019]

Thank you for your detailed guidance.I wish you success in your work and study

…

---Original--- From: "Lorenzo ***@***.***> Date: 2021/6/16 18:14:11 To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [lrzpellegrini/icarl-pytorch] How to use my own dataset to train my model? (#1) Hi @hefei1019, I never worked with .mat files, but it should be easy to load and convert them to PyTorch Tensors (see https://discuss.pytorch.org/t/how-to-read-a-dataset-in-mat-form-in-pytorch/71668/2). The dataset must be split in train and test sets before creating the NCProtocol (https://github.com/lrzpellegrini/icarl-pytorch/blob/b9ad40d86c3b47ba852fc70e87269c78d2fb2741/main_icarl.py#L98). Another thing you may want to change is the model (https://github.com/lrzpellegrini/icarl-pytorch/blob/master/models/icarl_net.py), which currently comes with some hard-coded dimensions which are specific to CIFAR. Consider that this code is very specific for the CIFAR dataset and may not work with other datasets in general. I recommend using the implementation found in the Avalanche framework which is more stable and flexible: https://github.com/ContinualAI/avalanche/blob/master/examples/icarl.py — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.