Feature engineering is the process of transforming raw data into input variables, called features, that represent the underlying problem in a form a model can learn from. In data processing, it follows data cleaning, deduplication and the handling of missing data, and it decides what information a predictive model can actually see.
Good features often matter more than the choice of algorithm. A model predicting delivery delays learns little from a raw timestamp, but much more from derived features such as hour of day, day of week, distance to depot and the number of parcels already queued for the same route.
How feature engineering works
Feature engineering works by encoding domain knowledge and data structure into numeric or categorical columns aligned to a single unit of prediction, such as one customer, one transaction or one day. The process usually iterates: create candidate features, evaluate their effect on validation performance, keep the useful ones and discard noise.
- Define the prediction unit and timestamp, the moment at which features must be known.
- Clean and align sources, resolving duplicates, missing values and inconsistent units.
- Transform, encode and aggregate raw fields into candidate features.
- Evaluate candidates with validation scores and feature importance.
- Package the final logic in a reproducible pipeline used identically for training and scoring.
The last step is critical. When training features are computed in a notebook and production features in separate code, small differences create training-serving skew, a frequent reason why models perform worse after deployment than in testing.
Organizations with many models often centralize this logic in a feature store, which keeps feature definitions, computes them on a schedule or in streaming mode and serves the same values to training jobs and live applications. A feature store also provides point-in-time correct joins, so historical training rows receive feature values exactly as they existed at that moment. Automated feature generation tools can enumerate aggregations and transforms at scale, but they produce many redundant candidates and still need domain review.
Common feature engineering techniques
Most techniques fall into a few families, each addressing a specific property of the raw data.
| Technique | What it does | Typical example |
|---|---|---|
| Scaling and normalization | Puts numeric values on comparable ranges | Standardize income to zero mean and unit variance |
| Mathematical transforms | Reduces skew or captures nonlinearity | Log of purchase amount, squared distance |
| Binning | Groups continuous values into intervals | Age bands such as 18-24 and 25-34 |
| Categorical encoding | Turns labels into numbers | One-hot, target or frequency encoding |
| Date and time features | Extracts cycles and durations | Weekday, month, days since last order |
| Aggregations | Summarizes history per entity | Orders in the last 30 days per customer |
| Interactions and ratios | Combines variables | Debt-to-income ratio, price per square meter |
| Text and embeddings | Converts unstructured content to vectors | Word counts, sentence embeddings |
Cyclical encoding deserves special mention: hour 23 and hour 0 are adjacent, so representing time with sine and cosine components lets a model see that closeness, which a plain integer cannot express.
Feature engineering compared with related data processing steps
Feature engineering is distinct from but dependent on the cleaning steps that precede it. Cleaning makes data correct, while feature engineering makes it informative for a specific prediction task.
| Step | Goal | Output |
|---|---|---|
| Data cleaning | Fix errors and inconsistencies | Correct, standardized records |
| Deduplication | Remove repeated entities or events | One record per real-world item |
| Imputation | Fill missing values plausibly | Complete columns, often with flags |
| Feature engineering | Represent the problem for a model | Informative input variables |
| Feature selection | Keep the most useful features | Smaller, more robust feature set |
The boundaries overlap. Adding a missing-value indicator next to an imputed column is both imputation and feature engineering, because the fact that a value was missing is often predictive in itself.
Choosing features for different models
The right amount of feature engineering depends on the model family. Some algorithms need carefully prepared inputs, while others discover structure themselves when given enough data.
| Model family | Needs scaling | Handles interactions itself | Feature engineering effort |
|---|---|---|---|
| Linear and logistic regression | Yes | No | High: transforms and interactions matter |
| Distance-based models | Yes | Partly | High: scale and relevance matter |
| Tree ensembles | No | Yes | Medium: aggregations and encodings matter |
| Deep neural networks | Yes | Yes | Lower for raw text, images and audio |
Even with deep learning, tabular business data benefits from hand-crafted aggregations such as recency, frequency and monetary value, because those summaries capture behavior across many raw events that a model cannot easily reconstruct.
Pitfalls in feature engineering
The most serious pitfall is target leakage, where a feature uses information unavailable at prediction time. Aggregations computed over the whole dataset, including future rows, and target encoding without out-of-fold estimation are common sources.
| Pitfall | Consequence | Prevention |
|---|---|---|
| Target leakage | Excellent tests, poor production results | Compute features strictly as of the prediction time |
| Fitting transforms on all data | Optimistic validation scores | Fit scalers and encoders on training folds only |
| High-cardinality one-hot encoding | Huge sparse matrices, overfitting | Use target, hashing or frequency encoding |
| Training-serving skew | Silent accuracy loss after launch | Share one pipeline or a feature store |
| Feature explosion | Slow training, unstable models | Select features and regularize |
Features can also encode sensitive attributes indirectly, for example postal code standing in for ethnicity or income. Reviewing features for fairness and stability belongs in the process, not after deployment.
Stripped to the essentials
A computer model only learns from the columns it is given. Feature engineering is the work of turning messy raw records into clear, useful clues, much as a doctor turns a long patient history into a few key facts such as age, blood pressure and recent symptoms.
The raw data may contain everything, but not in a helpful form. Well-chosen clues make the model smarter without changing the model at all.
Where it is applied
A bank’s fraud team trains a model on card transactions. The raw data holds only the amount, merchant, timestamp, card number and location. The team engineers features as of each transaction: number of transactions on the card in the last hour, ratio of the amount to the card’s 90-day median, distance from the previous transaction divided by elapsed time, whether the merchant category is new for the cardholder and the hour of day encoded cyclically.
All aggregations use only earlier transactions, and the same feature code runs in training and in the real-time scoring service. Compared with a model trained on raw fields alone, the engineered model flags fraudulent transactions with noticeably higher precision at the same alert volume, so analysts review fewer false alarms while catching more real fraud.