Skip to main content

TensorFlow Time Series

Time series analysis is one of the most important applications of Recurrent Neural Networks (RNNs). In this tutorial, we'll learn how to use TensorFlow to build powerful models for analyzing and forecasting time-dependent data.

Introduction to Time Series​

A time series is a sequence of data points collected or recorded at specific time intervals. Examples include:

  • Stock prices
  • Weather measurements
  • Energy consumption
  • Website traffic
  • Sensor readings

Time series analysis is valuable for understanding patterns, seasonality, trends, and making predictions about future values.

Why Use RNNs for Time Series?​

Traditional time series methods (like ARIMA) work well for many problems, but RNNs offer several advantages:

  1. Automatic feature learning: RNNs can learn complex patterns without manual feature engineering
  2. Handling nonlinearity: They capture complex nonlinear dependencies
  3. Long-term memory: LSTMs and GRUs are designed to remember patterns over long sequences
  4. Multivariate inputs: Can handle multiple variables easily

Time Series Components with TensorFlow​

Before diving into code, let's understand the key components of time series analysis:

  1. Trend: Overall direction (upward/downward)
  2. Seasonality: Regular patterns that repeat at predictable intervals
  3. Cyclical patterns: Irregular fluctuations
  4. Noise: Random variation in the data

Preparing Time Series Data for TensorFlow​

Data preparation is critical for time series modeling. Let's walk through the process:

python
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.preprocessing import MinMaxScaler

# Sample time series data generation (simulating monthly sales)
time = np.arange(0, 36, 1)
series = 10 + time * 0.1 + np.sin(time/2) * 5 + np.random.randn(36) * 2

# Convert to DataFrame for easy manipulation
df = pd.DataFrame({'time': time, 'value': series})
print(df.head())

This will output:

   time     value
0     0  10.550808
1     1  12.154174
2     2  10.823429
3     3  11.698584
4     4  14.950007

Data Normalization​

Always normalize time series data:

python
# Scale the data
scaler = MinMaxScaler(feature_range=(0, 1))
series_scaled = scaler.fit_transform(df['value'].values.reshape(-1, 1))

# Visualize the data
plt.figure(figsize=(12, 6))
plt.plot(time, series)
plt.title('Simulated Time Series Data')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True)
plt.show()

Creating Windowed Datasets​

For time series forecasting, we typically use a "window" of past values to predict future values:

python
def create_time_series_dataset(data, window_size):
    X, y = [], []
    for i in range(len(data) - window_size):
        X.append(data[i:i+window_size])
        y.append(data[i+window_size])
    return np.array(X), np.array(y)

# Create windowed dataset
window_size = 5  # Look back 5 time steps to predict the next value
X, y = create_time_series_dataset(series_scaled, window_size)

print(f"X shape: {X.shape}, y shape: {y.shape}")

This produces:

X shape: (31, 5, 1), y shape: (31, 1)

Train-Test Split​

Always split your time series data, remembering that the order matters:

python
# Split data into train and test sets (80% train, 20% test)
train_size = int(len(X) * 0.8)
X_train, X_test = X[:train_size], X[train_size:]
y_train, y_test = y[:train_size], y[train_size:]

Building RNN Models for Time Series​

Let's implement three different models to compare their performance:

1. Simple RNN Model​

python
def build_simple_rnn_model(window_size):
    model = tf.keras.models.Sequential([
        tf.keras.layers.SimpleRNN(20, activation='relu', input_shape=(window_size, 1)),
        tf.keras.layers.Dense(1)
    ])
    model.compile(optimizer='adam', loss='mse')
    return model

# Build and train simple RNN model
simple_rnn_model = build_simple_rnn_model(window_size)
simple_rnn_history = simple_rnn_model.fit(
    X_train, y_train, 
    epochs=50, 
    batch_size=4, 
    validation_data=(X_test, y_test),
    verbose=0  # Set to 1 to see progress
)

2. LSTM Model​

python
def build_lstm_model(window_size):
    model = tf.keras.models.Sequential([
        tf.keras.layers.LSTM(32, activation='relu', input_shape=(window_size, 1)),
        tf.keras.layers.Dense(1)
    ])
    model.compile(optimizer='adam', loss='mse')
    return model

# Build and train LSTM model
lstm_model = build_lstm_model(window_size)
lstm_history = lstm_model.fit(
    X_train, y_train, 
    epochs=50, 
    batch_size=4, 
    validation_data=(X_test, y_test),
    verbose=0
)

3. Stacked LSTM Model​

python
def build_stacked_lstm_model(window_size):
    model = tf.keras.models.Sequential([
        tf.keras.layers.LSTM(32, activation='relu', return_sequences=True, input_shape=(window_size, 1)),
        tf.keras.layers.LSTM(16, activation='relu'),
        tf.keras.layers.Dense(1)
    ])
    model.compile(optimizer='adam', loss='mse')
    return model

# Build and train stacked LSTM model
stacked_lstm_model = build_stacked_lstm_model(window_size)
stacked_lstm_history = stacked_lstm_model.fit(
    X_train, y_train, 
    epochs=50, 
    batch_size=4, 
    validation_data=(X_test, y_test),
    verbose=0
)

Making Predictions​

Now let's make predictions with our models:

python
# Make predictions
simple_rnn_pred = simple_rnn_model.predict(X_test)
lstm_pred = lstm_model.predict(X_test)
stacked_lstm_pred = stacked_lstm_model.predict(X_test)

# Inverse transform to original scale
simple_rnn_pred = scaler.inverse_transform(simple_rnn_pred)
lstm_pred = scaler.inverse_transform(lstm_pred)
stacked_lstm_pred = scaler.inverse_transform(stacked_lstm_pred)
y_test_orig = scaler.inverse_transform(y_test)

# Plot predictions
plt.figure(figsize=(12, 6))
plt.plot(y_test_orig, label='Actual')
plt.plot(simple_rnn_pred, label='Simple RNN')
plt.plot(lstm_pred, label='LSTM')
plt.plot(stacked_lstm_pred, label='Stacked LSTM')
plt.title('Time Series Prediction Comparison')
plt.xlabel('Time Steps')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()

Multi-step Forecasting​

So far, we've done one-step forecasting. But what if we want to predict multiple steps ahead?

python
def forecast_multi_step(model, initial_sequence, steps=10):
    """Generate multi-step forecasts"""
    forecast = []
    current_sequence = initial_sequence.copy()
    
    for _ in range(steps):
        # Get prediction for next step (reshape for model input)
        next_pred = model.predict(current_sequence.reshape(1, window_size, 1))
        forecast.append(next_pred[0, 0])
        
        # Update sequence by removing first element and adding prediction
        current_sequence = np.append(current_sequence[1:], next_pred)
    
    return np.array(forecast).reshape(-1, 1)

# Use the last window from training data as initial sequence
initial_sequence = X_train[-1]

# Generate multi-step forecasts
steps_ahead = 10
rnn_forecast = forecast_multi_step(simple_rnn_model, initial_sequence, steps_ahead)
lstm_forecast = forecast_multi_step(lstm_model, initial_sequence, steps_ahead)
stacked_forecast = forecast_multi_step(stacked_lstm_model, initial_sequence, steps_ahead)

# Inverse transform forecasts
rnn_forecast = scaler.inverse_transform(rnn_forecast)
lstm_forecast = scaler.inverse_transform(lstm_forecast)
stacked_forecast = scaler.inverse_transform(stacked_forecast)

# Plot results
plt.figure(figsize=(12, 6))
plt.plot(range(len(series)), series, label='Original Series')
plt.plot(range(len(series), len(series) + steps_ahead), rnn_forecast, label='RNN Forecast', linestyle='--')
plt.plot(range(len(series), len(series) + steps_ahead), lstm_forecast, label='LSTM Forecast', linestyle='--')
plt.plot(range(len(series), len(series) + steps_ahead), stacked_forecast, label='Stacked LSTM Forecast', linestyle='--')
plt.title('Multi-step Time Series Forecasting')
plt.xlabel('Time Steps')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()

Real-world Example: Stock Price Prediction​

Let's apply our knowledge to a real-world example: predicting stock prices. We'll use the Yahoo Finance API to get recent data:

python
import yfinance as yf

# Download stock data
ticker = 'AAPL'  # Apple Inc.
stock_data = yf.download(ticker, start='2020-01-01', end='2022-01-01')

# Use closing prices for prediction
closing_prices = stock_data['Close'].values.reshape(-1, 1)

# Normalize data
stock_scaler = MinMaxScaler(feature_range=(0, 1))
closing_prices_scaled = stock_scaler.fit_transform(closing_prices)

# Create dataset with window
window_size = 20  # Use 20 days of data to predict the next day
X_stock, y_stock = create_time_series_dataset(closing_prices_scaled, window_size)

# Train-test split (80-20)
train_size = int(len(X_stock) * 0.8)
X_train_stock = X_stock[:train_size]
X_test_stock = X_stock[train_size:]
y_train_stock = y_stock[:train_size]
y_test_stock = y_stock[train_size:]

# Build LSTM model for stock prediction
stock_model = tf.keras.models.Sequential([
    tf.keras.layers.LSTM(50, return_sequences=True, input_shape=(window_size, 1)),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.LSTM(50),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(1)
])

stock_model.compile(optimizer='adam', loss='mse')
stock_model.fit(X_train_stock, y_train_stock, epochs=50, batch_size=32, verbose=1)

# Make predictions
stock_predictions = stock_model.predict(X_test_stock)

# Inverse transform
stock_predictions = stock_scaler.inverse_transform(stock_predictions)
y_test_orig = stock_scaler.inverse_transform(y_test_stock)

# Plot results
plt.figure(figsize=(12, 6))
plt.plot(y_test_orig, label='Actual Stock Price')
plt.plot(stock_predictions, label='Predicted Stock Price')
plt.title(f'{ticker} Stock Price Prediction')
plt.xlabel('Trading Days')
plt.ylabel('Price (USD)')
plt.legend()
plt.grid(True)
plt.show()

Advanced Techniques​

1. Using Bidirectional RNNs for Time Series​

Bidirectional RNNs can capture patterns from both past and future context:

python
def build_bidirectional_lstm_model(window_size):
    model = tf.keras.models.Sequential([
        tf.keras.layers.Bidirectional(
            tf.keras.layers.LSTM(32, activation='relu'),
            input_shape=(window_size, 1)
        ),
        tf.keras.layers.Dense(1)
    ])
    model.compile(optimizer='adam', loss='mse')
    return model

2. Handling Multiple Features (Multivariate Time Series)​

Real-world time series often have multiple features:

python
def prepare_multivariate_data(df, target_col, feature_cols, window_size):
    features = df[feature_cols].values
    target = df[target_col].values
    
    # Scale features and target separately
    feature_scaler = MinMaxScaler(feature_range=(0, 1))
    features_scaled = feature_scaler.fit_transform(features)
    
    target_scaler = MinMaxScaler(feature_range=(0, 1))
    target_scaled = target_scaler.fit_transform(target.reshape(-1, 1))
    
    X, y = [], []
    for i in range(len(features) - window_size):
        X.append(features_scaled[i:i+window_size])
        y.append(target_scaled[i+window_size])
    
    return np.array(X), np.array(y), feature_scaler, target_scaler

3. Attention Mechanisms for Time Series​

Attention helps the model focus on important parts of the sequence:

python
from tensorflow.keras.layers import Dense, LSTM, Attention, Concatenate, Input
from tensorflow.keras.models import Model

def build_lstm_attention_model(window_size, features):
    inputs = Input(shape=(window_size, features))
    lstm_out = LSTM(32, return_sequences=True)(inputs)
    
    # Apply attention
    attention_layer = Attention()([lstm_out, lstm_out])
    
    # Combine attention output with original LSTM output
    concat = Concatenate()([lstm_out, attention_layer])
    
    # Final processing
    x = LSTM(16)(concat)
    outputs = Dense(1)(x)
    
    model = Model(inputs=inputs, outputs=outputs)
    model.compile(optimizer='adam', loss='mse')
    return model

Best Practices for Time Series Forecasting​

  1. Normalize your data: Always normalize time series data to a similar scale.
  2. Handle seasonality: Some time series have daily, weekly, or yearly patterns.
  3. Feature engineering: Add relevant features like day of week, month, holidays, etc.
  4. Evaluate properly: Use appropriate metrics like RMSE, MAE, or MAPE.
  5. Validate carefully: Consider using time-based cross-validation rather than random sampling.
  6. Watch for data leakage: Ensure future data doesn't leak into your training set.
  7. Consider domain knowledge: Incorporate domain expertise into your features or model design.

Summary​

In this tutorial, we've covered:

  • How to prepare time series data for TensorFlow models
  • Creating windowed datasets for sequence prediction
  • Building various RNN models (Simple RNN, LSTM, Stacked LSTM)
  • Making one-step and multi-step forecasts
  • Applying these techniques to real-world data
  • Advanced techniques and best practices

Time series analysis with TensorFlow opens up powerful possibilities for forecasting and understanding sequential data. Whether you're predicting stock prices, energy consumption, or sensor readings, these techniques provide a solid foundation for your projects.

Additional Resources​

Exercises​

  1. Try implementing a CNN-LSTM hybrid model for time series prediction
  2. Add more features to the stock prediction model (e.g., volume, moving averages)
  3. Experiment with different window sizes and compare performance
  4. Implement a model to forecast multiple time steps at once (sequence-to-sequence)
  5. Apply these techniques to a weather dataset to predict temperature or rainfall

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)