reloading trained model #1088
Comments
|
the model.net has been modified in the train function and should be equivalent to the call with the trained model path above. are you receiving an error otherwise? indeed if you want to use the model again after reloading the notebook then you need to load the model from the saved path |
|
@carsen-stringer hi, it is not an error. It is a strange message saying
I comprehensively read through all your previous Issues. I found that If we want to load the saved fine-tuned/re-trained model for eval, we have to set the I'm still a little doubted when you said they are equivalent. I tried some times. One instance to drive my doubt is that the evaluation results vary for these two model re-loading methods. If we fine-tuned the initial model with RGB images, and eval outputs by loading the fine-tuned model from path are RGB masks, since model architecture was modified during the training. However, the eval outputs by directly calling the model are still gray-scale masks (gray scale outputs are same as the initial model's outputs). This difference shows that calling the trained model from path is different from directly calling the model. |
|
oh it means that the model didn't load - they are not equivalent because you are loading the default model. what I meant was that you don't have to load a model if you've just trained the model. I think the bug in the code above is that you're missing a slash in the path: "/content/human_in_the_loop/train/models/CP_tissuenet" but you can check your paths in the notebook and see if that's where it is. Sorry can you clarify though, the model only works if gpu is False, and what other issue showed this? |
@carsen-stringer, I just added the slah, and the reloading works now. So you meant, within a same workflow pipeline, if I just trained the model and then I can directly call the model, no need to re-load it again? If so, why do we still assign a new name to the trained model? for the gpu issue: I trained the cyto3 model by setting the gpu is true
Then, I run the gpu for training The training log info looks good as below:
Aftewards, I reloaded the model for evaluation with the following codes. Unfortunately, the model cannot detect any masks. However, when I shift the gpu to |
|
can you please post the output of the last block of code you posted? regarding the model training, we always still save the model so you can use it later, but it is modified in the workspace you are currently in (and thanks for all the detailed info!) |
@carsen-stringer Pls see the output log info below. You can see
|
|
@carsen-stringer if I use gpu = false, then the eval outputs log info would be like below. Cee pixels can be found and masks are predicted and created
|
|
thanks, can you please include the log output when the model is made? |
|
btw I think you want to run with "do_3D=False" because you are using "stitch_threshold" |
do you mean the log info of training the model? I set |
|
@carsen-stringer Hello, just to check back this model re-loading issue. Setting
Below, I compared these two methods to reload the trained model. Firstly, my model training is like below: Secondly, method No.1 to reload the trained model: Then, method No.2 to reload the trained model: If I understood your previous response correctly, model reloading method No.1 is the same as the method No.2, but actually they are not although I just run testing immediately after training the model within a workflow. for testing the model, as you suggested, I just removed the Unfortunately, the model testing performance based on these two reloading methods are significantly different. The first loading method has IOU of around 0.96 while the second loading method has IOU of around 0.54. Could you please help check if my codes and model implementation are somehow wrong? I would appreciate your help. |
|
what I mean is you do not have to load a model, since it has been trained in your notebook. you are loading different models with those commands, you can see in the logger output which model is loaded. I would guess that means your finetuned model is bad, which would make sense if you keep retraining it over and over - you should not start training from an already finetuned model, please follow the cellpose2 notebook, which starts with a built-in cellpose model to retrain from. |
@carsen-stringer thank you for the clarification. I will check out the cellpose2 notebook. |
Hello, I have a question about the run_cellpose_2.ipynb. For the section of Train new model, the initial model was trained and saved to model_path = train_dir + 'models/'. And then, for the following codes from the section of Evaluation on test data below,
masks = model.eval(test_data, channels=[chan, chan2], diameter=diam_labels)[0]When you run the model on test images, why don't
you load the trained model "CP_tissuenet" from the path and then pass the trained model for evaluation? why do you still directly call the model.eval()? It seems themodel.eval()` still calls the initial cyto default model, not the trained model' for evaluation. I would think of replacing this code block by below:trained_model_path = "content/human_in_the_loop/train/models/CP_tissuenet"
model = models.CellposeModel(gpu=True, pretrained_model= trained_model_path )
masks = model.eval(test_data,
channels=[chan, chan2],
diameter=diam_labels)[0]
Could you pls check?
The text was updated successfully, but these errors were encountered: