Statistics Assignment 2
Statistics Assignment 2
LAKSHMANAN R (MT24AAI044)
2. Weakly (or wide sense) stationary process How do these two definitions differ?
Provide examples of each from daily life (e.g., temperature variations, stock
prices).
A strictly stationary process is a time series where the joint probability distribution
does not change when shifted in time. This means that all statistical properties (mean,
variance, higher moments) are invariant to time shifts. Formally, a process X(t) is strictly
stationary if for any times t1,t2,…,tn, the joint distribution of (X(t1),X(t2),…,X(tn)) is the
same as that of (X(t1+τ),X(t2+τ),…,X(tn+τ)) for any time shift τ.
Example: Suppose the number of certain particles passing through a sensor each
minute is constant on average and randomly fluctuates around this mean, but the entire
distribution of counts remains unchanged over time. Such processes could be
encountered in controlled experiments in physics.
• The covariance Cov(X(t),X(t+τ)) depends only on the time difference τ, not on the
specific times t and t+τ.
Example: Daily temperatures can be approximately weakly stationary over short
periods (such as a month), where mean temperature and variance are stable, but they
may not hold strictly over a year because of seasonal effects.
Key Differences
• Weak Stationarity only requires mean and covariance stability over time, which
is often easier to satisfy in real-world data.
• Weakly Stationary: Stock prices may exhibit weak stationarity within a short
trading period if the average price and volatility remain stable. However, they are
not strictly stationary over long periods due to trends and external influences.
Q2. For a process to be weakly stationary, the mean and autocovariance must
satisfy specific properties. State these properties and explain their significance in
the context of random processes.
1. Constant Mean:
Significance: A constant mean indicates that the process does not have a long-term
trend. This stability in the mean ensures that any observed fluctuations are due to
random variations rather than systematic changes over time.
Significance: Constant variance implies that the process's fluctuations around the
mean are of similar magnitude over time. This prevents periods of unusually high or low
variability, making the process more predictable and stable.
3. Time-Invariant Autocovariance:
If the autocovariance is zero for a particular time lag τ, it implies that the values of the
process at those points are uncorrelated:
• For instance, if γ(τ)=0 when τ=1, this means that two consecutive values of the
process, separated by one time unit, are uncorrelated. This lack of correlation
suggests that knowing the value of the process at one time point gives no
predictive information about its value one time unit later.
• When autocovariance is zero at larger lags (e.g., γ(τ)=0 for τ>k), it indicates that
values spaced by τ>k are independent or at least uncorrelated over that interval,
which can be characteristic of some random (or white noise) processes.
• White Noise: A process with zero autocovariance at all non-zero lags (i.e., γ(τ)=0
for all τ>0) is called a white noise process. This means the process has no
memory of past values, and each value is independent of others.
• Short Memory: Processes with autocovariances that quickly drop to zero for
larger lags are said to have "short memory." This is typical in processes where
past values have minimal impact on future values, making them easier to
forecast over short intervals.
In summary, autocovariance reflects how much past values influence future values in a
stationary process. When autocovariance is zero at certain lags, it indicates
independence or lack of correlation, which often simplifies modelling and analysis.
Q4. Explain the relationship between Gaussian processes and stationary
processes. What additional conditions must hold for a Gaussian process to be
stationary?
A Gaussian process and a stationary process are closely related concepts in time
series analysis and stochastic processes, but they have distinct definitions.
Gaussian Processes
Stationary Processes
A Gaussian process can be stationary, but not all Gaussian processes are stationary.
For a Gaussian process to be stationary, it must satisfy the conditions of weak
stationarity, which involve additional constraints on the mean and covariance:
1. Constant Mean: The mean function E[X(t)] of the Gaussian process must be
constant over time. For stationarity, this mean should not vary with time;
otherwise, the process would exhibit a trend.
When these conditions hold, a Gaussian process is both Gaussian and stationary.
Because the Gaussian process is fully defined by its mean and covariance functions,
satisfying these two requirements is sufficient to ensure that a Gaussian process is
stationary. In other words, if a Gaussian process has a constant mean and a time-
invariant covariance function, it is weakly stationary, and in this case, weak
stationarity also implies strict stationarity due to the properties of the Gaussian
distribution.
Summary
• A stationary Gaussian process has both a constant mean and a covariance that
depends only on the time lag.
Example
• The covariance is zero for any non-zero lag (i.e., no autocorrelation), and the
variance is constant.
This process is stationery and Gaussian, as all requirements for weak stationarity are
met, and because it is Gaussian, it is also strictly stationary.
Q5. Describe two examples of strictly stationary processes and two examples of
weakly stationary processes. Justify why each process satisfies the required
conditions.
o Justification:
o Justification:
▪ For any interval of time, the number of events follows the same
Poisson distribution, independent of the specific time at which the
interval occurs.
o Justification:
o Justification:
▪ The mean daily return is typically close to zero over short periods,
and the variance, representing volatility, remains relatively
constant.
Summary
Q6. Write a Python program that simulates a weakly stationary Gaussian process
with a specified mean and autocovariance function. Visualize the generated
process and plot its autocovariance function.
import numpy as np
# Parameters
def autocovariance_function(tau):
plt.figure(figsize=(12, 6))
plt.xlabel("Time")
plt.ylabel("Value")
plt.legend()
plt.show()
lags = np.arange(tau_max)
plt.xlabel("Lag")
plt.ylabel("Autocovariance")
plt.show()
To simulate a weakly stationary Gaussian process, we can create a time series where
values are drawn from a Gaussian distribution with a specified mean and autocovariance
function. We will use Python libraries like numpy to generate the time series data and
matplotlib to visualize both the process and its autocovariance function.
1. Parameters: The process has a specified mean, variance, and number of time
points.
3. Covariance Matrix: The Toeplitz structure of the covariance matrix ensures that
each element depends only on the time difference (lag), making it consistent
with weak stationarity.
5. Visualization:
This approach provides both a simulation of a weakly stationary Gaussian process and
an empirical view of its autocovariance function, illustrating weak stationarity visually.
Q7. Choose a time series dataset (such as daily temperature or stock prices) and
check for weak stationarity by examining the mean and autocovariance. Implement
the Dickey-Fuller test to determine whether the time series is stationary.
To check for weak stationarity in a time series dataset, we can examine whether the
mean and autocovariance remain stable over time. Additionally, we can apply the
Dickey-Fuller test to formally assess stationarity. In this example, let's use Python with
the pandas, matplotlib, and stats models libraries to analyse a time series dataset.
The Dickey-Fuller test checks for the presence of a unit root in the series, where the null
hypothesis (H0) is that the time series is non-stationary. If the test statistic is less than
the critical value, we can reject the null hypothesis and conclude that the series is
stationary.
Let's work with an example dataset of daily temperature or stock prices. For
demonstration, I'll use a mock example of daily stock prices, but you can easily replace
this with any other time series dataset.
Code:
import numpy as np
import pandas as pd
# Load or generate a sample time series dataset (e.g., daily stock prices)
np.random.seed(42)
plt.figure(figsize=(12, 6))
plt.xlabel("Date")
plt.ylabel("Price")
plt.legend()
plt.show()
# Calculate and plot rolling mean and rolling variance
rolling_mean = time_series.rolling(window=30).mean()
rolling_std = time_series.rolling(window=30).std()
plt.figure(figsize=(12, 6))
plt.xlabel("Date")
plt.ylabel("Price")
plt.legend()
plt.show()
result = adfuller(time_series)
print(f"P-value: {result[1]}")
print("Critical Values:")
else:
print("\nThe p-value is greater than 0.05, so we fail to reject the null hypothesis.")
1. Generate or Load the Time Series Data: We create a synthetic time series with
a slight trend for illustration. Replace this with actual time series data (e.g., stock
prices or temperature) as needed.
2. Plotting the Time Series: This initial plot helps visualize any trend or seasonal
pattern.
4. Dickey-Fuller Test:
o The test statistic, p-value, and critical values are output, allowing us to
determine if the series is stationary.
o If the p-value is below 0.05, we reject the null hypothesis and conclude
that the series is stationary.
• If the rolling mean and variance are stable and the Dickey-Fuller test
indicates stationarity (p-value < 0.05), then the series is weakly stationary.
• If the rolling mean and variance are not stable or the Dickey-Fuller test does
not reject the null hypothesis (p-value > 0.05), the series is likely non-
stationary, possibly due to a trend or seasonality.
Q8. Explain why a random walk is not weakly stationary by analyzing its mean and
variance over time.
A random walk is a time series model in which each value is determined by adding a
random "step" to the previous value. Formally, a simple random walk XtX_tXt can be
defined as:
Xt=Xt−1+ϵt
where:
• ϵt is a white noise term with mean zero and constant variance σ2,
To analyse why a random walk is not weakly stationary, let's examine its mean and
variance over time.
However, this does not imply that the process has a stable mean over time, as the
values of Xt continue to fluctuate in a manner dependent on the cumulative sum of ϵt,
which is unbounded in the long term.
A key requirement for weak stationarity is that the variance of the process must be
constant over time. Let's compute the variance of XtX_tXt:
This shows that the variance of Xt increases linearly with time. As time ttt progresses,
the variance grows indefinitely, violating the requirement for weak stationarity that
variance remains constant.
The autocovariance function in a weakly stationary process should depend only on the
time difference (lag) between two points. In a random walk, however, the
autocovariance function does not satisfy this condition:
The autocovariance between Xt and Xt+τ increases as t grows, which means it depends
on the specific time t and not just the time lag τ. This dependency violates the
stationarity requirement that the autocovariance depends only on the lag, further
confirming that a random walk is not stationary.
Summary
• Mean: Although the expected mean might seem zero if X0=0, the cumulative
nature of random steps means the series will drift over time, and the mean is not
constant.
• Variance: The variance increases linearly with time, violating the stationarity
condition of a constant variance.
• Autocovariance: The autocovariance depends on the time index t and not just
the lag τ, failing the stationarity condition.
Thus, a random walk is not weakly stationary because it has a time-varying mean and an
unbounded variance, both of which disqualify it from being considered stationary.
Q9. Extend the concept of stationarity to multivariate processes. De- scribe the
conditions under which a multivariate process can be considered stationary.
Implement a simulation for a multivariate weakly stationary process.
1. Constant Mean Vector: The mean vector E[Xt]=μ does not depend on time t, i.e.,
E[Xt]=μ for all t.
Code:
import numpy as np
# Parameters
def cross_covariance_function(tau):
for i in range(n):
for j in range(n):
x_series = data[:n]
y_series = data[n:]
plt.figure(figsize=(14, 6))
plt.xlabel("Time")
plt.ylabel("Value")
plt.legend()
plt.show()
lags = np.arange(tau_max)
plt.figure(figsize=(12, 6))
plt.xlabel("Lag")
plt.ylabel("Cross-Covariance")
plt.show()
1. Define Parameters: Set the means, variances, and correlation between the two
series.
o This ensures the resulting data satisfies the weak stationarity conditions.
5. Plot the Simulated Data: Visualize each series to confirm the stationarity of the
multivariate process.
This code simulates a weakly stationary multivariate process and provides a visual
confirmation of the weak stationarity through its mean and cross-covariance properties.