QualSynth Uses Deterministic Validation for LLM Minority-Class Oversampling
Researchers have unveiled a training-free framework named QualSynth in which a large language model is instructed to apply sparse modifications to genuine minority-class rows within feature constraints derived from the data and a machine-readable output contract, while a deterministic validation layer vets every candidate before it enters the training set.
The Tech TL;DR:
- QualSynth uses a large language model, guided by feature constraints drawn from the data and a machine-readable output contract, to apply sparse modifications to genuine minority-class rows.
- Across eight benchmark datasets with ten seeds, the framework matched SMOTE within prespecified margins of 0.03 in F1 score and 0.02 in ROC-AUC.
- Class weighting by itself earned the strongest overall ranks, and no augmentation method raised aggregate ROC-AUC above the unweighted baseline.
Imbalanced tabular classification tasks often involve synthetic oversampling producing rows that violate feature constraints or duplicate existing observations. QualSynth tackles this as a training-free framework in which a large language model is instructed to apply sparse modifications to genuine minority-class rows within feature constraints derived from the data and a machine-readable output contract.
Each candidate passes through a deterministic validation layer that inspects its structure, values, and duplication before it is admitted to the training set.
The validation layer is the architectural difference.
# Pseudocode for the admit/reject gate
for row in llm_candidates:
if not matches_schema(row, feature_constraints):
reject(row, reason="schema violation")
elif violates_domain(row, column_ranges):
reject(row, reason="value out of range")
elif is_duplicate(row, existing_records):
reject(row, reason="duplicate observation")
else:
training_set.append(row)
We ran QualSynth on eight benchmark datasets with ten seeds, comparing it to four synthetic oversamplers—including the widely used SMOTE—and to class-weighted learning without augmentation. On four datasets, QualSynth posted the top F1 score of the synthetic oversamplers, while SMOTE held the best average ranks.
Equivalence tests showed QualSynth and SMOTE varying by less than prespecified margins of 0.03 in F1 score and 0.02 in the area under the receiver operating characteristic curve (ROC-AUC). Once multiple comparisons were corrected for, the single pairwise difference that remained was a higher ROC-AUC for QualSynth than for the diffusion model TabDDPM.
| Method | Result reported |
|---|---|
| SMOTE | Best average ranks |
| QualSynth vs. SMOTE | Differed by less than prespecified margins of 0.03 in F1 score and 0.02 in ROC-AUC |
| QualSynth vs. TabDDPM | Higher ROC-AUC for QualSynth after correction for multiple comparisons |
| Class weighting alone | Best overall ranks |
Class weighting by itself earned the strongest overall ranks, and no augmentation method raised aggregate ROC-AUC above the unweighted baseline.
QualSynth thus stands as a controlled, inspectable route to minority-class augmentation, offering predictive utility on par with SMOTE’s.
QualSynth is a training-free framework that uses a large language model, guided by feature constraints drawn from the data and a machine-readable output contract, to apply sparse modifications to genuine minority-class rows.
Each candidate passes through a deterministic validation layer that inspects its structure, values, and duplication before it is admitted to the training set.
QualSynth thus stands as a controlled, inspectable route to minority-class augmentation, offering predictive utility on par with SMOTE’s.