Filtering strings with SimPO Dataset yields subpar results

[holchan]

I’ve extracted tokens (like class names, variables) from Unreal Engine 5 using regex and am cleaning them for a model fine-tuning task. I’m using a SimPO dataset with 150 manually crafted entries, designed to filter important tokens and return them in JSON format. The model processes 100 tokens at a time, but the results are not as expected.

My dataset contains chosen and rejected responses, but I’m unsure if 150 entries are enough to improve results. Should I expand the dataset? A smaller model for this task is enough? Since i can run it with greater precision? Should i use a base or a instruct one?

Dataset Entries:

{
  "instruction": "Filter out irrelevant tokens.",
  "chosen_response": "{ \"entities\": [\"ValidToken1\", \"ValidToken2\"] }",
  "rejected_response": "{ \"entities\": [\"InvalidToken1\", \"InvalidToken2\"] }"
}