Why Is Training Accuracy High but Validation Accuracy Low?
Clarify overfitting, leakage and unfair comparisons with a reproducible baseline exercise.
The doubt: does a high training score prove a good model?
No. Training data has already influenced the model. The practical question is whether the model generalizes to appropriately held-out observations. A gap between training and validation scores is a signal to investigate, not a complete diagnosis.
Possible explanations include overfitting, a small or unrepresentative validation sample, inconsistent preprocessing, data leakage or a change in the underlying population. Start by reproducing the split and checking the task rather than immediately adding more model complexity.
A simple baseline before tuning
This Python example calculates a constant-prediction baseline without external libraries. It was reviewed for arithmetic and syntax; this machine has no available Python command, so Python execution is not claimed.
train_labels = [0, 0, 0, 0, 1]
validation_labels = [0, 1, 1, 1, 1]
prediction = 0
train_score = sum(y == prediction for y in train_labels) / len(train_labels)
validation_score = sum(y == prediction for y in validation_labels) / len(validation_labels)
print(train_score, validation_score)Expected output is 0.8 and 0.2. No complicated model is involved: different class proportions alone create the gap. The exercise shows why interpreting a score without inspecting the data can lead to a wrong explanation.
Should I change the split until the score improves?
Do not select a split solely because it gives a pleasing result. Choose it to reflect use. Observations from the same person or device may require grouping; future predictions may require time-based evaluation. Keep the decision documented and assess uncertainty with a suitable validation approach.
Where does preprocessing belong?
Learn preprocessing parameters from training data and apply the learned transformation to validation data. A pipeline helps keep this sequence consistent. Looking at held-out data to choose features, thresholds or transformations can contaminate the evaluation even when the model fitting call itself uses training rows only.
What evidence would support overfitting?
Compare performance as training sample size and model complexity change, using a consistent evaluation setup. Check a simple baseline and inspect examples of errors. A persistent gap may support the hypothesis, but rule out implementation mistakes and changed data distributions before presenting it as fact.
Practice and expected result
Replace the validation labels with [0,0,0,1,1]. The constant-zero validation accuracy becomes 0.6. Explain why this improves without any model learning. Then list three checks for a real classifier: split validity, preprocessing fitted only on training observations, and metrics that represent the operational goal. Avoid treating accuracy as sufficient for a rare positive class.
Data Science course.
scikit-learn common pitfalls.
Prepared by AI Leaf Global Learn for classroom learners in Mudichur/Tambaram, Chennai, and online learners across India. Confirm current course and batch details. Area tags describe audiences, not additional offices.