Training the CPPD recognition network with larger max_text_ length

[aliiafkari]

Hello everyone,
I am currently working on training the CPPD recognition network with a larger max_text_length value. However, I encountered the following error while running the model.
Could anyone kindly assist me in resolving this issue?
Thank you in advance for your help!

[GreatV]

Based on your question about training the CPPD recognition network with a larger max_text_length and the provided documents, here’s a detailed response:

---

Problem Overview

When increasing the max_text_length parameter for the CPPD recognition network, you encountered an error. This issue could stem from multiple causes, such as mismatched configurations, insufficient image resizing, or memory limitations.

From the discussion and issue logs, the following points are relevant:

1. Increasing max_text_length requires corresponding adjustments in the image width to maintain clarity.
2. Pretrained model weights may not align if the architecture or configuration is significantly altered (e.g., changing dictionary size or max text length).
3. Augmented datasets or specific configurations (e.g., RecAug, MultiScaleSampler) may unintentionally distort the input data or cause compatibility issues.

---

Potential Causes and Suggestions

1. Mismatch Between max_text_length and Image Dimensions

• Increasing max_text_length requires adjusting the image width to avoid making the text too blurry for recognition. In your configuration:

  scales: [[640, 32], [640, 48], [640, 64]]

  These scales may not be sufficient for very long strings (e.g., 135 characters). You could try increasing the width to accommodate the longer text:

  scales: [[1024, 32], [1024, 48], [1024, 64]]

  Why? Longer text requires more horizontal space for the network to learn meaningful features.

2. Dataset Augmentation Issues

• Augmentations like RecAug or RecConAug can sometimes warp text, especially for long strings.
• You’ve disabled RecAug and RecConAug, which is good for long-text datasets. However, you should ensure that the dataset itself is clear and not overly distorted by other augmentation techniques.

3. Pretrained Model Compatibility

• If you’ve modified the max_text_length or dictionary, the pretrained model might not fully align with the new configuration. This could explain why the model struggles to improve.
• Ensure the dictionary (character_dict_path) and max_text_length in the configuration match the dataset and task requirements.

4. Dataset Size and Diversity

• With only ~450 original images (even with augmentation), the dataset may not provide sufficient diversity for generalization. If possible, increase the dataset size by collecting more varied samples of long-text images.

5. Memory Constraints

• Training on long text with large images can quickly exhaust GPU memory. To mitigate this:
  • Reduce batch_size_per_card to a smaller value (e.g., batch_size_per_card: 12).
  • Use mixed precision training (amp in PaddleOCR).

---

Actionable Steps

Step 1: Adjust Image Dimensions and max_text_length

• Increase the image width in the scales section:

  scales: [[1024, 32], [1024, 48], [1024, 64]]

• Ensure the image_shape matches the new width in both the Train and Eval transforms:

  RecResizeImg:
    image_shape: [3, 48, 1024]

Step 2: Verify Dataset Label Format

• Ensure all labels in your dataset are valid and do not exceed the max_text_length. Any label longer than max_text_length will cause errors during training.

Step 3: Reinitialize Training if Necessary

• If changing max_text_length or the dictionary, avoid loading pretrained weights unless you are fine-tuning. Either:
  • Remove the pretrained_model path from the configuration or
  • Ensure that the pretrained model aligns with the new max_text_length and dictionary.

Step 4: Monitor For Errors

• If you encounter specific errors (such as "Illegal Instruction" or "RecursionError"), check:
  • GPU compatibility (e.g., CUDA version matching PaddlePaddle).
  • Dataset issues (e.g., corrupted images or mismatched labels).

Step 5: Debugging Training Progress

• Log detailed metrics during training, especially accuracy and loss trends for both CTCLoss and NRTRLoss. If accuracy stagnates (~50%), it might indicate:
  • Overfitting to a small dataset.
  • Misalignment between the configuration and dataset.

---

References

From the documents provided:

1. Increasing max_text_length and image width:

   • Refer to Issue #13938, where increasing the image width was suggested for long-text datasets.
   • Ensure RecAug and RecConAug are disabled for long-text training.

2. Troubleshooting Model Errors:

   • Discussions like RecursionError suggest dataset corruption or augmentation issues as potential causes.

3. Fine-tuning and Pretrained Model Loading:

   • In Discussion #13897, loading pretrained models with modified dictionaries can cause poor performance if not handled carefully.

---

Summary

To train the CPPD recognition network with a larger max_text_length, you should:

1. Increase image width proportionally.
2. Ensure dataset quality and alignment with the configuration.
3. Avoid incompatible pretrained weights if the dictionary or max_text_length changes.
4. Monitor training metrics and debug errors related to GPU or dataset inconsistencies.

If you continue to face issues, please share detailed logs or error messages for further assistance.

---

Response generated by 🤖 feifei-bot | chatgpt-4o-latest