dataset-split-review

Dataset Split Review

A random split is often the wrong split. Incorrect splitting causes massive overestimation of model performance.

Time-Series Data: If the data has a time component, train_test_split is strictly forbidden. You must use a chronological split to prevent the model from learning the future.
Group Leakage: If the dataset has multiple rows for a single user/patient/session, a standard split will put rows from the same user in both train and test. You must use GroupKFold or group-based splitting.
Stratification: For imbalanced datasets, verify that stratification is used to maintain the target distribution across all splits.
Action: Review the splitting code and explicitly verify Time, Group, and Stratification safety.

A random split is often the wrong split. Incorrect splitting causes massive overestimation of model performance.

Time-Series Data: If the data has a time component, train_test_split is strictly forbidden. You must use a chronological split to prevent the model from learning the future.
Group Leakage: If the dataset has multiple rows for a single user/patient/session, a standard split will put rows from the same user in both train and test. You must use GroupKFold or group-based splitting.
Stratification: For imbalanced datasets, verify that stratification is used to maintain the target distribution across all splits.
Action: Review the splitting code and explicitly verify Time, Group, and Stratification safety.