Dev

Practical Automated Feature Engineering in Python: Validating py-evoFE for Tabular ML

Back to Tutorials

What Automated Evolutionary Feature Engineering Solves

Tabular machine-learning performance is often constrained less by the classifier or regressor than by the representation of the raw columns. A model may receive amount, quantity, event_time, customer_id, and several categorical fields, while the useful predictive signals are interactions and transformations such as:

  • normalized amounts, ratios, differences, and conditional rates;
  • non-linear transforms such as logarithms, powers, clipping, or bins;
  • calendar components extracted from timestamps;
  • group-level aggregates such as a customer's historical mean amount;
  • nested relationships, such as a value relative to a segment aggregate;
  • interactions between numeric and categorical variables.

Hand-building these features is slow and highly dependent on domain knowledge. Automated feature engineering attempts to search this space systematically. py-evoFE is designed around evolutionary search: candidate feature recipes are treated as individuals in a population, evaluated with a machine-learning objective, then recombined and mutated across generations.

This is different from merely generating every pairwise interaction. The search can represent chained transformations and non-linear projections, so it can explore structures that would be tedious to enumerate manually. For example, a candidate could conceptually express a normalized transaction measure, then apply a non-linear transform, rather than only creating a fixed library of first-order features.

The trade-off is substantial: evolutionary search is an optimization process over your validation metric. If you do not isolate validation correctly, the system can discover features that exploit sampling quirks rather than a signal that survives deployment. Treat it as a model-selection component with computational cost and leakage risk, not as a one-click preprocessing step.

Prerequisites and Project Decisions

Before evaluating py-evoFE, establish a baseline and decide what a valid prediction-time feature is. This matters more than the search settings.

You need:

  • A supervised tabular dataset with a clearly defined target.
  • A split strategy consistent with production. Use a temporal split for forecasting or event prediction; use group-aware splits when records from an entity must not cross train and test.
  • A baseline model and metric. Keep the baseline simple enough to debug.
  • A feature contract stating which columns are available at prediction time.
  • A reproducible environment: pinned Python dependencies, fixed split definitions, fixed random seeds where supported, and saved experiment outputs.
  • Enough compute for repeated candidate evaluation. Evolutionary methods multiply fit cost by the number of candidate recipes and generations.

Polars is useful when raw table operations or preprocessing are a bottleneck, particularly for wide data or datasets that do not fit comfortably in a conventional eager workflow. scikit-learn remains useful for estimator selection, preprocessing, cross-validation, and metric calculation. The important operational question is the data boundary: determine exactly where a Polars frame becomes the matrix consumed by the estimator and by the feature-engineering library.

Do not assume that a library accepting Polars input also preserves Polars objects throughout training, inference, serialization, and estimator integration. Verify this in the installed package documentation and with a small smoke test before committing to a large search.

Define the prediction-time contract

For every prospective feature, answer these questions:

  1. Is every source column known when the prediction is requested?
  2. If the feature uses an aggregate, is that aggregate computed only from data available before that event?
  3. Can the same feature state be reproduced later for a single batch or one record?
  4. Does the feature depend on row ordering, an external lookup, or a mutable population statistic?

A feature can look statistically valid during offline training yet be impossible to compute online. For example, a global average computed across the full dataset includes future information in a time-based problem. Likewise, a customer-level aggregate must be built from historical observations only if it is used for an event occurring today.

Your baseline is the control group. Without it, an improved score after feature search is not evidence that the genetic search helped; the difference may come from a changed split, preprocessing change, metric mistake, or random variation.

Start with a single immutable split definition. For a binary classification task, the following is a deliberately ordinary baseline pattern. Adapt the column selection and preprocessing to your schema.

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_columns = ["amount", "quantity", "account_age_days"]
categorical_columns = ["country", "channel"]

numeric_pipeline = Pipeline(
    steps=[
        ("impute", SimpleImputer(strategy="median")),
        ("scale", StandardScaler()),
    ]
)

categorical_pipeline = Pipeline(
    steps=[
        ("impute", SimpleImputer(strategy="most_frequent")),
        ("encode", OneHotEncoder(handle_unknown="ignore")),
    ]
)

preprocessor = ColumnTransformer(
    transformers=[
        ("numeric", numeric_pipeline, numeric_columns),
        ("categorical", categorical_pipeline, categorical_columns),
    ]
)

baseline = Pipeline(
    steps=[
        ("preprocess", preprocessor),
        ("model", LogisticRegression(max_iter=1000)),
    ]
)

baseline.fit(X_train, y_train)
validation_probability = baseline.predict_proba(X_validation)[:, 1]
validation_auc = roc_auc_score(y_validation, validation_probability)
print(f"Baseline validation ROC AUC: {validation_auc:.5f}")

This example is not intended to prescribe logistic regression as the final model. Its purpose is to produce a reproducible reference. Use the same target definition, split boundary, metric, and prediction-time inputs for every later comparison.

For regression, use a regression metric appropriate to the business loss, such as MAE, RMSE, or a transformed-target metric. For imbalanced classification, accuracy is usually insufficient; prefer metrics such as PR AUC, ROC AUC, recall at a fixed precision, or a thresholded cost metric.

Record more than one number:

  • training metric;
  • validation metric;
  • fit duration;
  • number of raw features and transformed features;
  • random seed or run identifier;
  • estimator parameters;
  • split identifier.

Feature search must occur inside the training process, not against a test set. The safest evaluation design uses three levels of data:

  1. Training data for evolving and fitting candidate transformations.
  2. Validation data for choosing settings and comparing candidate systems.
  3. Final test data held untouched until one final evaluation.

For smaller datasets, nested cross-validation is stronger: an outer split estimates final generalization, while an inner split drives the evolutionary search and hyperparameter selection. It is expensive, but that cost reflects the fact that you are selecting among many feature recipes.

Time and entity boundaries are non-negotiable

A random row split is invalid when related rows appear on both sides of the split. Common examples include multiple claims per policyholder, multiple events per account, repeated measurements per device, or transactions per merchant.

For temporal data, train only on the past and validate on later records. This rule applies to transformations as well as labels. Date parts derived directly from the event timestamp are usually safe. A category frequency calculated over the whole dataset is not safe if the future validation records contributed to it.

For entity aggregation, calculate aggregate state from the training period, then map that state into validation. Decide a fallback for unseen entities: a global training statistic, a missing indicator, or a domain-specific default. Never compute validation aggregates using validation targets.

Step 3: Integrate py-evoFE Without Inventing an API

py-evoFE's public interface, supported input types, configuration names, result objects, and export methods must be confirmed against the installed release. Do not copy guessed constructor names from examples written for another package or version. Evolutionary ML packages often differ materially in how they specify primitive operations, fitness functions, population controls, and transformed-output retrieval.

Use this implementation sequence instead.

Run a tiny smoke test first

Create a reduced training dataset with a limited number of numeric, categorical, and timestamp columns. Configure a deliberately small search budget. The purpose is to answer operational questions, not to maximize the metric:

  • Does the package accept your preferred table type directly?
  • Are nulls, strings, categorical values, infinities, and timestamps accepted?
  • Does it expect a separate target vector?
  • Does it return an engineered table, a fitted transformer, selected recipe metadata, or all three?
  • Can the fitted result transform an unseen validation batch without re-fitting?
  • Does the output integrate with your selected scikit-learn estimator?
  • Are the discovered recipes serializable and inspectable?

Use this conceptual adapter boundary in your codebase. The function body is where you place the exact, verified py-evoFE calls for the installed version.

from dataclasses import dataclass
from typing import Any

@dataclass
class EvolutionResult:
    fitted_feature_engineer: Any
    train_features: Any
    validation_features: Any
    recipe_report: Any

def evolve_features(train_table, train_target, validation_table) -> EvolutionResult:
    """Implement with the verified py-evoFE API for the pinned package version.

    Required behavior:
    - fit feature search using training data only;
    - transform validation data using fitted training state;
    - retain a human-readable recipe/export report;
    - never pass validation labels into feature discovery.
    """
    raise NotImplementedError("Add calls from the installed py-evoFE documentation.")

This boundary prevents package-specific calls from leaking across your training and serving code. It also makes it feasible to replace the library, disable the search, or compare it with manually engineered features.

After the smoke test works, use the transformed outputs in a normal estimator pipeline:

result = evolve_features(X_train, y_train, X_validation)

final_estimator.fit(result.train_features, y_train)
validation_prediction = final_estimator.predict(result.validation_features)
validation_score = metric_function(y_validation, validation_prediction)

The code is intentionally interface-neutral. final_estimator, metric_function, and the exact matrix conversion depend on the estimator and verified library output. What must not change is the fit/transform order: fit on training data, then transform validation data with frozen learned state.

Step 4: Configure the Search as a Budgeted Experiment

Evolutionary algorithms commonly expose controls equivalent to population size, number of generations, mutation or crossover behavior, allowed transformation primitives, maximum expression depth, complexity penalties, and random seeds. Exact names vary, but their effects are familiar.

Start small. A large primitive set combined with deep chains makes the search space explode. It also raises the likelihood of mathematically invalid expressions and uninterpretable artifacts.

A sensible progression is:

  1. Start with a limited numeric-only subset and a short budget.
  2. Permit simple arithmetic and carefully protected non-linear transforms.
  3. Add timestamps or categorical representations only after checking their semantics.
  4. Add group aggregates only when historical availability is explicit.
  5. Increase search budget only when the small experiment produces stable improvement across repeated splits or folds.

Complexity control is critical. Candidate recipes should be penalized or constrained when they create excessive columns, deep expression trees, unstable divisions, or transforms that cannot be served. A tiny validation gain is not worth a feature graph that requires expensive stateful computation at inference.

Keep a strict transformation allowlist. In high-stakes domains, avoid primitives that make sense mathematically but create unsafe values, such as unrestricted division, logarithms of non-positive values, or exponentials likely to overflow. If the library protects these operations, understand exactly how: clipping, protected division, missing values, and arbitrary fallback constants each alter the feature distribution.

Step 5: Evaluate More Than the Winning Metric

The best candidate from a large search may win by chance. Require evidence beyond one validation score.

Evaluate the selected recipe on:

  • multiple resamples or temporal windows where possible;
  • a final untouched test split;
  • key cohorts such as geography, acquisition channel, customer tenure, or device type;
  • missing-data-heavy rows;
  • newly observed categories and entities;
  • inference-sized batches and single-row requests if applicable.

Compare at least four systems:

  1. Raw baseline features.
  2. Your existing manual features.
  3. Evolutionary features alone, if the output format supports it.
  4. Raw plus evolutionary features, subject to duplicate-column and multicollinearity checks.

Also inspect feature redundancy. Automated search can produce many near-duplicate transforms of a strong source column. More columns can increase memory use, slow training, and make model behavior harder to audit without materially improving generalization.

Common Errors and How to Fix Them

Error: Feature search reports an impressive validation gain but test performance drops

Likely cause: repeated tuning against the same validation set caused selection overfitting.

Fix: reserve a final test set, reduce the number of unconstrained experiments, and use nested cross-validation or repeated splits when the dataset size permits. Treat each run's configuration as an experiment artifact.

Error: Candidate features work offline but cannot be computed in production

Likely cause: recipes depend on future information, full-dataset aggregates, unavailable columns, or state that was never persisted.

Fix: add a prediction-time feature contract. Export and version the recipe, fitted state, aggregate lookup tables, null-handling rules, and fallback behavior. Test transformation on a production-shaped batch before accepting a candidate.

Error: Search crashes with division-by-zero, NaNs, infinite values, or type failures

Likely cause: raw values violate a primitive operation's mathematical domain, or mixed input types are being passed into numeric operations.

Fix: profile nulls, cardinality, ranges, zero counts, negatives, timestamps, and string columns before search. Restrict primitives, clean invalid values using training-derived rules, and add explicit indicators where missingness itself is useful.

Error: Runs are too slow or exhaust memory

Likely cause: too many candidate recipes, generations, input columns, expression depth, folds, or output columns.

Fix: reduce the initial search budget, preselect plausible raw columns, cap expression complexity, profile the transformation stage separately from model fitting, and use a smaller representative development sample. Scale only after demonstrating a stable gain.

Error: The result cannot be reproduced

Likely cause: random state, split definitions, package versions, primitive allowlists, or data snapshots were not recorded.

Fix: pin dependencies, persist split indices or temporal cutoffs, log search configuration and seed, preserve the selected recipe report, and version the feature schema with the model artifact.

Production Checklist

Before deploying an evolved feature set, require all of the following:

  • The final test metric materially exceeds the baseline under the same split policy.
  • The gain survives at least one alternate split, time window, or resampling procedure.
  • Every selected feature is available at prediction time.
  • Learned aggregate state is fitted only on allowed historical training data.
  • The fitted transformer or recipe can be persisted and reloaded.
  • Input schema validation catches missing, renamed, and unexpected columns.
  • Null, unseen-category, invalid-domain, and overflow behavior is documented.
  • Feature computation meets latency and memory budgets.
  • Raw inputs, engineered outputs, recipe version, and model version are observable in production.

The right outcome is not necessarily a massive generated feature table. A successful search may identify a small number of robust transformations that teach the team something useful about the data-generating process. Preserve those recipes, simplify them where possible, and prefer maintainable signal over a marginal offline score.

FAQ

Is automated feature engineering better than gradient-boosted trees?

Not automatically. Tree-based models already learn many thresholds and interactions from raw tabular columns, so a broad feature search can add little value or create redundant columns. Evolutionary features can still help when useful transformations are ratios, normalized measures, time-derived signals, aggregate relationships, or representations that make a stable signal easier for the chosen estimator to use. Measure against a strong baseline rather than assuming either approach wins.

Can py-evoFE safely create target encodings or group aggregates?

Only when the implementation and split design prevent target leakage. Any target-dependent encoding must be generated out of fold for training rows and derived solely from training history for validation or test rows. Group aggregates based only on feature values also need temporal discipline when future rows would otherwise influence historical predictions.

Should I use Polars throughout the pipeline?

Use Polars where it improves ingestion and table transformation for your workload, but verify each library boundary. The feature engineer and downstream scikit-learn estimator may require a different tabular or matrix representation. Standardize and test conversion at one explicit adapter layer rather than allowing silent conversions throughout the codebase.

How long should an evolutionary search run?

There is no universal duration. Begin with a small budget that completes quickly enough to debug and repeat. Increase population, generations, or allowed expression complexity only after a small run produces interpretable candidates and a stable validation benefit. The correct budget is bounded by generalization evidence and compute cost, not by the point at which the training metric stops improving.

How do I know whether a discovered feature is trustworthy?

Inspect its complete recipe, source columns, mathematical domains, learned state, and prediction-time availability. Then test its distribution across training, validation, and production-like data. A feature is not trustworthy merely because it scores well in one experiment; it must be reproducible, leakage-free, stable across relevant cohorts, and operationally computable.