Time Series Classification

Time Series data is the type of data that is recorded over specific time intervals. Any dataset that stores a separate timestamp, whether date or time, can be considered as a Time series dataset. Generally, the time intervals are equally spaced but the duration may vary in certain conditions. Time series classification is a field of supervised machine learning. In Time series classification, one or more features are measured over time. The aim is to classify the given data point under a correct label or class. In this article, we will explore Time Series Classification and How it can be performed in Machine Learning or Deep Learning.

What is Time Series and Time Series Data in ML ?

Time Series is a form of sequential data points that are recorded at successive points in time. Here, each sample or data point represents an observation corresponding to a particular instance of time. The interval span can either be uniform, or vary occasionally. It ranges from seconds to minutes, hours, days, months, etc.

In Machine Learning, Time Series Data refers to any dataset containing a specific timestamp, which can be in form of dates, months or specific hours. The temporal order matters here, and the main aim is to analyze the pattern or trend followed by data points over time. This helps in estimating and forecasting future events and making informed decisions. It can be used in assessing disease spread, trends, stock prices, etc.

Key Features of Time Series Classification

Some of the key features or characteristics of the time series data are:

1. The entire sequence has an assigned label or class. Individual steps are not classified
2. Considers Sequence or Temporal order of the data is essential and a dependency since future values depend on historical data
3. Common patterns, trends, and seasonality (periodic) can be extracted from analysis of Time Series data
4. Data points in Time series data are often correlated
5. Timestamps are used to index a data point
6. Used in real-time analysis and trend identification

Assumptions in Time Series Classification

• A key assumption is that the time intervals are equally spaced and order is preserved
• Assumes class-specific patterns existing in sequence. Linearity among features in many models, i.e. Linear relationship between variables is considered.
• Sequence length is assumed to be constant
• No missing timestamps or irregular sampling is assumed. This can cause inconsistency in data patterns and adversely impact the classification
• Assumes random errors with no autocorrelation

Time Series Classification Workflow

Time Series Classification is the process of assigning label or category to a time series sequence. Classification is different from Forecasting since it predicts the type of entire sequence and not the next value.

Step 1: Installing Dependencies and Data Collection of Time Series Data

A dataset with Time-stamped data is created or collected to perform classification. It must have data samples in an expected sequence. Each sequence is assigned a corresponding label. The Time Series Data can be classified as Univariate or Multivariate based on the number of features.

! pip install tslearn
# Import necessary libraries
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, confusion_matrix
from tslearn.datasets import CachedDatasets
from tslearn.neighbors import KNeighborsTimeSeriesClassifier
from sklearn.decomposition import PCA

# Load a sample time series classification dataset
X, y, _, _ = CachedDatasets().load_dataset("Trace") 

Step 2: Visualize the Time Series (Analyzing the pattern)

Different types of Data Visualization Techniques can be utilized for pattern identification in Time Series Data. Some of these Techniques are:

• Line Plots are used to understand the Trends, Seasonality, and Cyclicity in the Data
• Rolling Mean and Standard Deviation Analysis
• Class-wise Pattern Analysis

plt.figure(figsize=(10, 4))
for i in range(3):
    plt.plot(X[i], label=f"Class {y[i]}")
plt.title("Sample Time Series from Trace Dataset")
plt.xlabel("Time")
plt.ylabel("Value")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

Output

Dataset used: "Trace" Dataset from "CachedDatasets" in "tslearn" library.

Step 3: Decompose the Time Series and Check Stationarity (Optional)

• Decomposing of Time Series is done to separate Trend, Seasonality, and Residual (error)
• ADF or Augmented Dickey Fuller and KPSS test can be utilized for checking stationarity.
• If non-stationary, log transformation can be applied

# Decomposing Time Series
from statsmodels.tsa.seasonal import seasonal_decompose
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.stattools import adfuller

series = pd.Series(
    [0.1, 0.2, 0.3, 0.4, 0.5, 0.1, 0.2, 0.3, 0.4, 0.5, 0.1, 0.2, 0.3, 0.4, 0.5],
    index=pd.date_range("2021-01-01", periods=15) 
)

decomposition = seasonal_decompose(series, model='additive', period=5)
decomposition.plot()
plt.show() 

# Checking Stationarity
result = adfuller(series)
print("ADF Statistic:", result[0])
print("p-value:", result[1])

Output

ADF Statistic: -3496593705052619.0
p-value: 0.0

Step 4: Split the Data into Training and Testing Data

Splitting of data is an essential part in model training since it ensures that the data for training and testing is correctly distributed in appropriate proportion.

# Train-test split (chronological for time series)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)

Note: The Time Series Data must not be shuffled. Chronological splitting should be used to split the data into training and testing data.

Step 5: Build the Time Series Classification Model and Model Training

There are multiple techniques that can be utilized for Time Series Classification. Some of these techniques include:

• KNN + DTW
• CNN
• LSTM
• Transformers
• ROCKET or Mini ROCKET
• TS Fresh + RF/SVM

# Model training using K Neighbors Time Series Classifier
knn = KNeighborsTimeSeriesClassifier(n_neighbors=1, metric="dtw")
knn.fit(X_train, y_train)

# Predict
y_pred = knn.predict(X_test)

Step 6: Model Evaluation on Key Metrics - Accuracy, F1-Score, Confusion Matrix

Classification models are typically evaluated on a specific set of Evaluation Metrics. They are calculated based on True positives, False positives, True negatives, and False negatives. Let's look at some key metrics:

• Accuracy
• Precision
• Recall
• F1-score
• Sensitivity
• Confusion Matrix

report = classification_report(y_test, y_pred)
print(report)

Output

Classification Report:
precision recall f1-score support
1 1.00 1.00 1.00 7
2 1.00 1.00 1.00 3
3 1.00 1.00 1.00 5
4 1.00 1.00 1.00 5
accuracy 1.00 20
macro avg 1.00 1.00 1.00 20
weighted avg 1.00 1.00 1.00 20

Step 7: Visualizing the Results

# Flatten X for PCA
X_flat = X.reshape((X.shape[0], -1))
pca = PCA(n_components=2)
X_2D = pca.fit_transform(X_flat)

# Plot 2D PCA projection
plt.figure(figsize=(8, 6))
scatter = plt.scatter(X_2D[:, 0], X_2D[:, 1], c=y, cmap='tab10', edgecolor='k')
plt.title("PCA Projection of Time Series")
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.grid(True)
plt.legend(*scatter.legend_elements(), title="Classes")
plt.tight_layout()
plt.show()

Output

Techniques for Time Series Classification

Time Series Classification can be performed using various techniques. Some of these techniques are listed below:

1. Feature-based Classification: Converts time series into a feature vector (e.g., mean, std dev, trend, autocorrelation) and applies traditional classifiers like SVM, RF.
2. Shapelet-based Classification: This approach identifies small, unique subsequences (shapelets) that are highly representative of a class.
3. Distance-based Classification: This approach uses distance metrics like DTW to compare the similarity between sequences. Example: DTW + K-Nearest Neighbors.
4. Deep Learning Models: Hierarchical features are auto-learnt directly from raw time series data using DL Models. This approach is deal for capturing complex temporal dependencies.
5. Transform-based Classification: This approach aims at transforming time series data into symbolic form for enhanced pattern recognition. Examples: SAX and DWT.
6. Ensemble Methods: These methods utilize multiple classifiers and combine them to improve accuracy and make model more robust for multivariate or noisy time series data. Examples: TS-CHIEF, HIVE-COTE

Multiple different approaches are utilized for Time Series Forecasting. Some of these are:

1. ARIMA
2. SARIMA or Seasonal ARIMA
3. Exponential Smoothing
4. Prophet
5. LSTM
6. GRU for Multivariate Forecasting

To know more about Time Series Techniques, you can refer to Time Series Forecasting

Applications of Time Series Classification

1. Finance: Classifying Trends in Stock prices
2. Healthcare: ECG signals Classification
3. IOT: Equipment failure prediction
4. Manufacturing: Optimal Inventory management and Forecasting

Advantages

1. Supports Traditional and Deep Learning models
2. Works for univariate and multivariate data
3. Captures complex temporal relationships
4. Effective for pattern recognition in real-world applications

Disadvantages

1. Huge historical data required for efficient forecasting
2. Sensitive to outliers or missing data
3. Some assumptions are violated in real-life applications
4. Computationally expensive for scalable series
5. Hard to handle sudden rise or fall in trend
6. Low model interpretability for Deep Learning Models