%%javascript
IPython.OutputArea.auto_scroll_threshold = 9999;
from IPython.core.display import display, HTML
display(HTML("<style>.rendered_html { font-size: 15px; }</style>"))
import logging
logging.basicConfig(level='INFO')
Timeseria is a time series processing library which aims at making it easy to handle time series data and to build statistical and machine learning models on top of it.
It comes with a built-in set of common operations (resampling, slotting, differencing etc.) as well as models (reconstruction, forecasting and anomaly detection), and both custom operations and models can be easily plugged in.
Timeseria also tries to address by design all those annoying things which are often left as an implementation detail but that actually cause wasting massive amounts of time - as handling data losses, non-uniform sampling rates, differences between time-slotted data and punctual observations, variable time units, timezones, DST changes and so on.
This is a (super) quickstart, if you are looking for a more structured introduction, have a look at the welcome notebook. Also the reference documentation might be useful.
Let's load some data: an indoor temperature winter dataset.
from timeseria import storages
DATASET_PATH = '/'.join(storages.__file__.split('/')[0:-1]) + '/tests/test_data/csv/'
temperature_timeseries = storages.CSVFileStorage(DATASET_PATH + 'temperature_winter.csv').get()
Let's have a look at the time series we just loaded:
temperature_timeseries
Now plot the data, using Timeseria built-in plotting engine:
temperature_timeseries.plot(aggregate=False)
If you zoom in, you will see that data has been aggregated and plotted as a line chart plus an area chart (which reprensents the minimum and maximum values boundaries before aggregating the data). In this way you can plot even millions of data points wihtout slowing down the plot or crashing your browser, without loosing much information about peaks and spikes.
Now resample the time series at one hour sampling interval, also making data uniform and equally spaced over time. Gaps are filled by linear interpolation (the default interpolation method for the resampling) and the "data loss" index is added as data point attribute.
temperature_timeseries = temperature_timeseries.resample('1h')
temperature_timeseries
temperature_timeseries[58].data_loss
temperature_timeseries.plot()
Use a simple, periodic average-based model for reconstructing missing data. Fit on about 80% of the data and test on the rest, with 6, 12, and 24 hours gaps. Limit the evaluation to 100 samples per step to speed up the process.
from timeseria.models import PeriodicAverageReconstructor
paverage_reconstructor = PeriodicAverageReconstructor()
paverage_reconstructor.fit(temperature_timeseries[0:2000])
paverage_reconstructor.evaluate(temperature_timeseries[2000:], steps=[6,12,24], limit=100)
Apply the reconstruction model. By default only full (100%) data losses are recontructed, as Timeseria considers data points with lower da losses (i.e. because in the interpolation process they were sitting in between existing and missing data points) as still representative. Also, a "data_reconstructed" index is added.
temperature_timeseries = paverage_reconstructor.apply(temperature_timeseries)
...and plot:
temperature_timeseries.plot()
temperature_timeseries[0].data_reconstructed
Use a LSTM neural network mdoel to to forecast three days of temperatures, but before fitting run a cross validation to get an idea about the accuracy you can expect:
from timeseria.models import LSTMForecaster
LSTM_forecaster = LSTMForecaster(window=12, neurons=64, features=['values', 'diffs', 'hours'])
LSTM_forecaster.cross_validate(temperature_timeseries, rounds=3)
Now fit, apply and plot:
LSTM_forecaster.fit(temperature_timeseries)
LSTM_forecaster.apply(temperature_timeseries,n=72).plot(indexes=['forecast'])
Use the periodic average anomaly detection model, which will consider a data point value an anomaly if too far from its periodic average. This will add an "anomaly" index as data points attibute.
from timeseria.models import PeriodicAverageAnomalyDetector
paverage_anomaly_detector = PeriodicAverageAnomalyDetector()
paverage_anomaly_detector.fit(temperature_timeseries, stdevs=5)
Apply and plot, but show only the "anomaly" index to improve plot readability:
paverage_anomaly_detector.apply(temperature_timeseries).plot()
Apply again both models, but this time on the same time series, and plot:
anoamaly_temperature_timeseries = paverage_anomaly_detector.apply(temperature_timeseries)
forecast_and_anomaly_temperature_timeseries = LSTM_forecaster.apply(anoamaly_temperature_timeseries, n=72)
forecast_and_anomaly_temperature_timeseries.plot()
Slot the time series into 1-day slots, also computing the min and max operations besideds the default average one, but change the timezone before to get the right daily aggregates and to properly take into account the DST change. Slotting in days is indeed different than slotting in 24 hours, they are just two different time units: the first is variable, the second is fixed.
forecast_and_anomaly_temperature_timeseries.change_timezone('Europe/Rome')
from timeseria.operations import min, max
forecast_and_anomaly_temperature_timeseries.slot(unit='1D', extra_operations=[min,max]).plot()
You can have a look at the welcome tutorial, which provides a more structired introduction on Timeseria datastructures and their philosophy, as well as practical examples on both buiilt-in and custom models and operations.
You can also have a look at the example repository (Timeseria-notebooks), which is also ready to be play wiht in Binder, together with this quickstart and the welcome tutorial.
Or, you can give it a try in your own projects:
pip install git+https://github.com/sarusso/Timeseria.git..or you can contribute! :)