NYC Taxi Trip Duration Prediction

The Problem

Predicting how long a trip will take is a common real-world problem in logistics, ride-sharing, and delivery. This project uses a dataset of 10,000 NYC taxi trips (pickup/dropoff timestamps, coordinates, neighborhoods, distance, passenger count) to test how much feature engineering, done automatically rather than by hand, can improve prediction accuracy.

Approach

1. Used Featuretools' Deep Feature Synthesis to automatically generate features from the raw trip, pickup neighborhood, and dropoff neighborhood tables.
2. Built 3 versions of the feature set, from minimal to rich, to isolate how much each addition improves the model.
3. Trained a Gradient Boosting Regressor on each feature set version, predicting log(trip_duration + 1).
4. Compared R² across all 3 stages and ranked feature importance for the richest model.

Results

Trip duration is right-skewed: most trips take under 1,000 seconds (about 16 minutes), with a small tail of longer trips.

Across the three feature-engineering stages, R² went from 0.757 (basic trip attributes plus a simple weekend flag) to 0.808 (adding minute/hour/day/week/month features) to 0.799 (adding neighborhood-level aggregations on top). Interestingly, the richest feature set didn't perform best — the extra aggregation features added complexity without a clear payoff, a useful reminder that automated feature engineering doesn't guarantee improvement past a certain point.

The final model's predicted duration distribution closely mirrors the real one, and trip_distance dominates feature importance by far (0.867), followed by dropoff location and pickup hour, matching intuition: distance drives duration, location affects route/congestion, and hour affects traffic.

Predicted trip durations on the test set, closely tracking the shape of the real distribution above.

Tools