Researched and wrote a paper evaluating the impact of fine-tuning reasoning models on entity tracking, using a T5-base model with a focus on mathematical and computational reasoning tasks. The study compares performance across various datasets: general knowledge, coding, and math. Results indicate that models trained on coding datasets excel in entity tracking, while those on math datasets struggle with unfamiliar symbols.
This paper explores the impact of fine-tuning models equipped with reasoning capabilities on entity tracking. Leveraging a T5-base model, we evaluate the effects of fine- tuning via two distinct avenues: mathematical reasoning using mathematical question-answer pairs and computational reasoning with coding- related question-answer pairs. The study in- vestigates the models’ performances across in- dividual datasets—general knowledge, code, and math—as well as their combinations. Re- sults demonstrate that models trained on code- only datasets exhibit superior entity tracking capabilities compared to general knowledge- based models, while those trained on math- ematical reasoning exhibit challenges due to out-of-vocabulary symbols 
Artwork