Data Wrangling·Lesson 32 of 67·intermediate
Time series basics
Parse timestamps properly, index by them, resample to a coarser grain, and use rolling windows to watch a model's latency and quality drift.
In this lesson · 18 min
- →Parse timestamps and set a DatetimeIndex
- →Slice by date with a string
- →Pull parts of a timestamp out with the .dt accessor
- →Resample to a coarser grain with a chosen aggregation
- →Smooth a noisy metric with a rolling window, and measure change with shift and diff
Almost everything an AI system emits is a time series: requests per minute, latency, cost, error rate, the fraction of answers a judge accepts. pandas has a whole layer for this, and it removes most of the date arithmetic you would otherwise get wrong.
Timestamps and the index
Parsing real timestamps means being explicit, and letting bad rows fail visibly:
Setting the timestamp as the index unlocks the time-aware behaviour:
The .dt accessor
On an ordinary timestamp column, .dt reaches the parts:
Resampling
resample is a group-by where the groups are time buckets. It is how a stream of individual
requests becomes a chart you can read:
The frequency strings you will use: min, h, D, W, ME (month end). Upsampling to a finer
grain leaves gaps, which you then fill deliberately:
Rolling windows
A rolling window smooths a noisy metric so a trend becomes visible:
The first five rolling values are NaN because a six-wide window needs six observations.
min_periods relaxes that when you would rather have an early, noisier number:
shift and diff
shift moves a column along the index, which is how you compare each point with the one before:
Try it yourself
Resample the request log to hourly buckets in hourly, with requests (count) and median_latency columns.
Add a rolling_3d column — the three-day rolling mean of cost — and a change column holding the day-on-day difference.
Check yourself
Why does a rolling(6) mean start with five NaN values?
Check yourself
You add a feature using .shift(-1) to a model trained on time-ordered data. What have you done?
Recap
- Parse with an explicit
formatandutc=True;errors="coerce"keeps bad rows visible. - A
DatetimeIndexlets you slice with a string:df.loc["2026-03-01"]. .dtreaches the parts of a timestamp column —hour,date,day_name,dayofweek.resample("1h").agg(...)is a group-by over time buckets;ffillorinterpolatefills the gaps upsampling leaves.rolling(n)smooths, and needsnobservations unless you setmin_periods.shift(1)anddiff()compare a point with the one before.shift(-1)looks into the future — and leaks it into any model you train.