Feature engineering

Feature engineering turns raw data into inputs that make a useful signal easier for a model to learn.

Tiny example

Raw transaction records contain timestamps and amounts. A fraud model may benefit from derived features:

transactions in the last hour
amount ÷ the account's normal amount
days since the account was opened
country changed since the previous transaction

Changing random forest to a neural network cannot recover information that the inputs never represent.

Before changing the algorithm again

1. Verify labels, missing values, class balance, and the evaluation split.
2. Inspect errors by user, time period, and important segment.
3. Add only features available at the real prediction time.
4. Fit transformations on training data, then apply them to validation and test data.
5. Compare against the same baseline and check latency, drift, fairness, and maintainability.

scikit-learn’s pipeline guidance warns that fitting preprocessing or feature selection on test data leaks information and creates an optimistic score.¹

FDE note

Feature engineering is an experiment, not a guaranteed fix. Watch for label leakage, future information, unstable identifiers, and proxy variables that encode sensitive attributes.

Footnotes

1. scikit-learn, common pitfalls, recommends splitting before preprocessing and fitting transformations only on training data. ↩