Feature Engineering for Machine Learning: From Raw Data to Model-Ready Features

It's often said that data scientists spend 80% of their time on data preparation. Feature engineering—transforming raw data into representations that ML models can learn from effectively—is where models are won and lost. Even the best algorithm will underperform on poorly engineered features.

Handling Missing Data

import pandas as pd
from sklearn.impute import SimpleImputer, KNNImputer

# For numerical features: mean/median imputation
num_imputer = SimpleImputer(strategy="median")
df["reading_time_filled"] = num_imputer.fit_transform(df[["reading_time"]])

# For categorical features: most frequent value
cat_imputer = SimpleImputer(strategy="most_frequent")
df["category_filled"] = cat_imputer.fit_transform(df[["category"]])

# KNN imputation preserves correlations between features
knn_imputer = KNNImputer(n_neighbors=5)
df_imputed = pd.DataFrame(knn_imputer.fit_transform(df), columns=df.columns)

Encoding Categorical Variables

• One-hot encoding: For low-cardinality categoricals (category, status). Creates binary columns.
• Target encoding: Replace category with mean target value. Powerful but prone to leakage—use cross-validation folds.
• Embeddings: For high-cardinality categoricals (user_id, product_id). Learned representations capture semantic relationships.

Creating Interaction Features

# Polynomial features
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True)
X_poly = poly.fit_transform(X[["reading_time", "word_count"]])

# Domain-specific interactions
df["words_per_minute"] = df["word_count"] / (df["reading_time"] + 1)
df["recency_score"] = (df["publish_date"] - df["publish_date"].min()).dt.days

Automated Feature Selection

Not all features help—some add noise. Use mutual information, SHAP feature importance, or recursive feature elimination to identify which features actually contribute to model performance, then remove the rest to improve generalization.

Production Application Telemetry Wrapper

Here is an enterprise-grade telemetry decorator in Python to measure execution latency, record counts, and catch pipeline boundaries:

import time
import logging
from functools import wraps

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("MirahLabs.Telemetry")

def monitor_performance(operation_name: str):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            t0 = time.perf_counter()
            try:
                res = func(*args, **kwargs)
                dt = time.perf_counter() - t0
                logger.info(f"{operation_name} succeeded in {dt:.4f}s")
                return res
            except Exception as e:
                dt = time.perf_counter() - t0
                logger.error(f"{operation_name} failed after {dt:.4f}s: {str(e)}")
                raise e
        return wrapper
    return decorator

Model Performance & Retrieval Profiles

Below is the performance comparison profile for our processing pipeline tested in staging against sanitized validation datasets:

US & UK Regulatory Standards for Artificial Intelligence

Deploying machine learning models in the US and UK markets requires strict alignment with local regulatory frameworks. In the United States, applications must respect the guidelines set by the FTC regarding algorithmic transparency, alongside the Executive Order on Safe, Secure, and Trustworthy AI. In the United Kingdom, AI systems must comply with the UK General Data Protection Regulation (UK GDPR), which enforces strict rules on automated profiling (under Article 22). Conducting bias auditing and maintaining explainable decision paths is critical to avoiding compliance sanctions in both jurisdictions.