In our previous post we showed how to subtract 5 minutes from a pandas DataFrame:
pd.Timestamp('now') - pd.Timedelta(10, 'minutes')We can also use this knowledge in order to get the last 10 minutes of a pandas DataFrame. In our example, we assume that df[“Timestamp”] contains the timestamp. First, we get the last timestamp in the dataset using
# Use this if the timestamp is the index of the DataFrame last_ts = df.index.iloc[-1]
or
# ... or use this if the timestamp is in a colum last_ts = df["Timestamp"].iloc[-1]
Next, we define the first timestamp that shall be considered by subtracting 10 minutes from last_ts:
first_ts = last_ts - pd.Timedelta(10, 'minutes')
Now we can filter the DataFrame using
# Use this if the Timestamp is in a column filtered_df = df[df["Timestamp"] >= first_ts]
or
# Use this if the Timestamp is the index of the DataFrame filtered_df = df[df.index >= first_ts]
By filtering, we don’t need the DataFrame to be sorted and the original order will be maintained.
Full example:
This example loads our pre-built time series example dataset from our previous post How to create pandas time series DataFrame example dataset. The code loads that dataset (which is 1 second long) and takes the last 0.5 seconds from it.
import pandas as pd
# Load example dataset
df = pd.read_csv("https://techoverflow.net/datasets/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)
# Use this if the timestamp is the index of the DataFrame
last_ts = df.index[-1]
first_ts = last_ts - pd.Timedelta(0.5, 'seconds')
filtered_df = df[df.index >= first_ts]
# Plot the result
filtered_df.plot()