9 Time series-related instance methods

9.1 Shifting / lagging

One may want to shift or lag the values in a time series back and forward in time. The method for this is shift, which is available on all of the pandas objects.

In [1]: ts = ts[:5]

In [2]: ts.shift(1)
Out[2]: 
2011-01-31         NaN
2011-02-28    0.469112
2011-03-31   -0.282863
2011-04-29   -1.509059
2011-05-31   -1.135632
Freq: BM, dtype: float64

The shift method accepts an freq argument which can accept a DateOffset class or other timedelta-like object or also a offset alias:

In [3]: ts.shift(5, freq=datetools.bday)
Out[3]: 
2011-02-07    0.469112
2011-03-07   -0.282863
2011-04-07   -1.509059
2011-05-06   -1.135632
2011-06-07    1.212112
dtype: float64

In [4]: ts.shift(5, freq='BM')
Out[4]: 
2011-06-30    0.469112
2011-07-29   -0.282863
2011-08-31   -1.509059
2011-09-30   -1.135632
2011-10-31    1.212112
Freq: BM, dtype: float64

Rather than changing the alignment of the data and the index, DataFrame and Series objects also have a tshift convenience method that changes all the dates in the index by a specified number of offsets:

In [5]: ts.tshift(5, freq='D')
Out[5]: 
2011-02-05    0.469112
2011-03-05   -0.282863
2011-04-05   -1.509059
2011-05-04   -1.135632
2011-06-05    1.212112
dtype: float64

Note that with tshift, the leading entry is no longer NaN because the data is not being realigned.

9.2 Frequency conversion

The primary function for changing frequencies is the asfreq function. For a DatetimeIndex, this is basically just a thin, but convenient wrapper around reindex which generates a date_range and calls reindex.

In [6]: dr = pd.date_range('1/1/2010', periods=3, freq=3 * datetools.bday)

In [7]: ts = pd.Series(randn(3), index=dr)

In [8]: ts
Out[8]: 
2010-01-01    0.469112
2010-01-06   -0.282863
2010-01-11   -1.509059
Freq: 3B, dtype: float64

In [9]: ts.asfreq(BDay())
Out[9]: 
2010-01-01    0.469112
2010-01-04         NaN
2010-01-05         NaN
2010-01-06   -0.282863
2010-01-07         NaN
2010-01-08         NaN
2010-01-11   -1.509059
Freq: B, dtype: float64

asfreq provides a further convenience so you can specify an interpolation method for any gaps that may appear after the frequency conversion

In [10]: ts.asfreq(BDay(), method='pad')
Out[10]: 
2010-01-01    0.469112
2010-01-04    0.469112
2010-01-05    0.469112
2010-01-06   -0.282863
2010-01-07   -0.282863
2010-01-08   -0.282863
2010-01-11   -1.509059
Freq: B, dtype: float64

9.3 Filling forward / backward

Related to asfreq and reindex is the fillna function documented in the missing data section.

9.4 Converting to Python datetimes

DatetimeIndex can be converted to an array of Python native datetime.datetime objects using the to_pydatetime method.