| C. Troussas, A. Krouska, Ph. Mylonas, C. Sgouropoulou, I. Voyiatzis |
| A Comparative Study of Large Language Model Fine-Tuning Strategies for Low-Resource Text Classification |
| 11th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM 2026), Thessaloniki, Greece, September 18-20, 2026 |
|
ABSTRACT
|
| Large language models (LLMs) have displayed extraordinary abilities in diverse NLP tasks. However, a research hurdle that still exists is how to adapt them well to domain-specific low-resource text classification tasks. This paper introduces a systematic comparative study on LLMs fine-tuning techniques: full fine-tuning, LoRA, prompt tuning and prompt learning on three low-resource text classification datasets from the categories of medical triage, legal document classification, and social media hate speech detection. Based on extensive experiments over datasets with 200-2000 labeled examples, parameter-efficient fine-tuning techniques (especially LoRA) reach full fine-tuning level accuracy while only use less than 3% trainable parameters; and prompt-based techniques also provide reasonable zero-shot and few-shot accuracy with chain-of-thought reasoning. Our findings offer actionable guidelines for practitioners to apply LLMs to low-resource text classification tasks.
|
| 18 September, 2026 |
| C. Troussas, A. Krouska, Ph. Mylonas, C. Sgouropoulou, I. Voyiatzis, "A Comparative Study of Large Language Model Fine-Tuning Strategies for Low-Resource Text Classification", 11th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM 2026), Thessaloniki, Greece, September 18-20, 2026 |
[
BibTex] [
Print] [
 Back] |
