Talk:Autoregressive moving-average model/Archive 1

This is an archive of past discussions about Autoregressive moving-average model. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 1

X_t or X_n

I am used to that t is used when function is continuous, while n is used when function is discrete. Any reason for keeping t in the discrete function? -- Heptor 01:54, 14 December 2005 (UTC)

Re: "Some constraints are necessary on the values of the parameters of this model in order that the model remains stationary. For example, processes in the AR(1) model with |φ1| > 1 are not stationary." Doesn't this describe stability rather than stationarity? A process is stationary so long as its parameters do not change with time. If φ1 is constant, then the process is stationary; however, |φ1| > 1 is not stable--i.e., the process will "blow up". --PLP Haire 20:49, 26 July 2006 (UTC)

I'm not sure how you are defining stationary here. With |φ1| > 1 the variance of the process will increase with time and therefore the process is not stationary (either strictly stationary or weakly stationary). See [Stationary Process]. --Richard Clegg 23:33, 26 July 2006 (UTC)

I think there should be an absolute value sign around the m-k in the Yule-Walker equations summation. --lextrounce 27 May 2007 (UTC)

read this?

Have any of the article authors read

http://www.ltrr.arizona.edu/~dmeko/notes_5.pdf

to my thinking it is far more approachable to the average encyclopedia reader. LetterRip 02:32, 5 November 2007 (UTC)

These are not the usual defintions of AR, MA and ARMA models that I come across in time-series analysis texts (e.g Harvey, A. (1981): Time Series Models.)

I am used to seeing them specified as follows:

AR(p) model:

X_{t}=\phi _{1}X_{t-1}+\phi _{2}X_{t-2}+\dots +\phi _{p}X_{t-p}+\epsilon _{t}

where ε_t is sampled from a normal distribution mean 0 and variance σ².

MA(q) model:

X_{t}=\theta _{1}\epsilon _{t-1}+\theta _{2}\epsilon _{t-2}+\dots +\theta _{q}\epsilon _{t-q}+\epsilon _{t}

where the ε_t are normal and independent (that is GWN) and an ARMA(p,q) model specified as:

X_{t}=\sum _{i=1}^{p}\phi _{i}X_{t-i}+\sum _{j=1}^{q}\theta _{j}\epsilon _{t-j}+\epsilon _{t}

Richard Clegg

Well, these defns are identical to what's in the article at present, with epsilon and X instead of X and Y, right? The article doesn't say anything about assumptions about the distribution of the noise term (X or epsilon); I guess it should. I'm sure Gaussian noise is by far the most common assumption although I'd be surprised if other noise models haven't been investigated. I guess I think we should steer away from identifying AR or MA models with a particular noise distribution. For what it's worth, Wile E. Heresiarch 03:48, 3 Feb 2005 (UTC)

Certainly I agree that what is given in the article is more general than what I wrote (with the brief fix -- thanks for that). But is it not sensible to stick to convention rather than give the most general model possible? When I hear talk of an AR model, I assume that the error sequence will be i.i.d. Gaussian zero mean. Any departure from that would be unusual would it not? I am very new to wikipedia so I don't know what is the common practice here. Do you agree that the notation above is more common (to use the epsilon rather than x) and to assume normally distributed i.i.d. errors? If so would you object to me rewriting the page slightly to use that notation and suggest that variants on the models could use different distributions for the innovations? To me, the ARIMA models imply normally distributed errors unless otherwise stated and certainly it would be very unusual not to have i.i.d. innovations (if the errors are not independent for example, the PACF for an AR will not be finite as discussed below.)

Another reason we might want to edit this is that the page for time series has a definition of an MA model on it specifying WN~(0,σ²) for the innovations so, currently the two pages are in conflict. However, as a newcomer, I don't want to press the point.

Richard Clegg

Richard, thanks for your comments, and thanks for taking the time to check in and test the waters -- always a useful habit I would say. I think you should go ahead and revise the article as you see fit to better address what is conventionally assumed about such models. I guess my only $0.02 would be to identify the conventional assumptions as such. Have at it! Regards & happy editing, Wile E. Heresiarch 05:22, 5 Feb 2005 (UTC)

Updated as discussed. Thanks for the advice. I hope it meets your approval.

--Richard Clegg 14:16, 6 Feb 2005 (UTC)

Could someone please add a discussion on the following topics?

how an MA model has a finite autocorrelation sequence but an infinite partial autocorrelation sequence.
how an AR model has an infinite autocorrelation sequence but a finite partial autocorrelation sequence.
how ARMA has both infinite

I'm not sure how these things work and would like to see the math behind it.

thanks,

Mark Wilde

I notice that recent edits to this page are including the possibility of a non-zero mean. This is, of course, not a major issue but I think that we should really leave this out because it just complicates the notation without adding much. At the very least, if we are going to include the possibility of a non-zero mean we should including it consistently on AR, MA and ARMA models. I certainly favour leaving it out or leaving in a section to discuss it as it does not add anything much of value in my opinion. --Richard Clegg 17:01, 14 Mar 2005 (UTC)

I agree. The proof for the Yule-Walker equations I just added would become unnecessary complicated (OK, I admit I don't know how to prove Yule-Walker with the c). Otherwise, feel free to elaborate my proof. --Heptor 00:35, 12 December 2005 (UTC)

$E[\varepsilon _{t}X_{t-m}]=0$ ?

I think

E[\varepsilon _{t}X_{t-m}]=0

only when m > 0, not less. Yet it seems to be required for the later steps in the proof. How is this resolved? After all,

E[\varepsilon _{t}X_{t-m}]=E\left[\varepsilon _{t}(\sum _{i=1}^{p}\varphi _{i}\,X_{t-i-m}+\varepsilon _{t})\right]=\sum _{i=1}^{p}\varphi _{i}\,E[\varepsilon _{t}\,X_{t-i-m}]+E[\varepsilon _{t}^{2}]\neq 0

since t-i-m>=0 for m<0. Yoderj 21:01, 21 July 2007 (UTC)

Indeed, for m>0, $E[\varepsilon _{t}X_{t+m}]\neq 0$ . The simplest way to see it is that by solving back the equation, or directly using the Wold theorem, $\varepsilon _{t}$ impacts $X_{t+m}$ (it appears in the infinite MA representation of $X_{t+m}$ ). However, your calculation is not a valid proof as it is circular - if I understand it right, basically you say that $E[\varepsilon _{t}X_{t+m}]\neq 0$ (m>0) as $E[\varepsilon _{t}X_{t+k}]\neq 0$ for 0<k<m. AdamSmithee 07:56, 5 November 2007 (UTC)

Normal or not ?

Currently the article allows the noise process $\varepsilon$ to be non-normal, as long as it has finite mean and variance. But the article also says that $X_{t}$ is normally distributed "by the central limit theorem". This is not true surely? If $\varepsilon$ is not normal then $X$ is not normal either. And if $\varepsilon$ is normal then so is $X$ (but not by the central limit theorem). Fathead99 (talk) 10:37, 13 February 2008 (UTC)

stationarity vs. invertability

stationarity & invertability should be explained in the article. Jackzhp (talk) 20:21, 11 March 2008 (UTC)

Box Jenkins method

Box-Jenkins method to identify the p & q of the ARMA(p,q) should be explained in the article. Jackzhp (talk) 23:40, 20 March 2008 (UTC)

Example for AR(1)

The part about the variance says :

${\textrm {var}}(X_{t})=E(X_{t}^{2})-\mu ^{2}={\frac {(c+\varphi \mu )^{2}+\sigma ^{2}-\mu ^{2}}{1-\varphi ^{2}}}.$

For c = 0, $\mu$ = 0 and its variance amounts to $\sigma ^{2}/(1-\varphi ^{2})$ .

But since

 $\mu ={\frac {c}{1-\varphi }}$

then $c+\varphi \mu =\mu$ so the variance equals $\sigma ^{2}/(1-\varphi ^{2})$ ,regardless of the value of c and $\mu$ So we should remove that condition. —Preceding unsigned comment added by 209.5.96.146 (talk • contribs) 15:44, 29 July 2008 UTC

You're absolutely right and thanks for pointing this out. I have corrected the article. In the future, feel free to make such corrections yourself, that's the whole point of Wikipedia. --Zvika (talk) 17:50, 29 July 2008 (UTC)

Matrix equation

Hello everyone. Great article! Now, in the calculation of the AR parameters section, we see a matrix equation with $\gamma _{i}$ terms in it. What do these gammas mean? Robinh (talk) 08:44, 1 August 2008 (UTC)

"

\gamma _{m}

is the autocorrelation function of X" (from the article) --Zvika (talk) 12:20, 1 August 2008 (UTC)

Autocorrelation or Autocovariance?

In the Derivation section it is stated:

Now, E[X_tX_{t − m}] = γ_m by definition of the autocorrelation function.

Is that not wrong? Isn't it the autocovariance function. Phunck (talk) 08:35, 27 June 2008 (UTC)

it is autocovariance strictly speaking but (unfortunately) in the literature the terms autocorrelation and autocovariance are alertanively used. EtudiantEco (talk) 09:17, 18 October 2008 (UTC)

Update on MATLAB's capabilities

Good article!

MATLAB's Econometrics Toolbox provides greatly expanded support for working with ARMA, ARMAX, and even VARMA models, much more than this article gives credit for. More info can be found here.

Please consider updating the Implementations in statistics packages section. I would do it myself, but since I work for The MathWorks, I would hate to be accused of spamming Wikipedia with our products. I'll leave the decision to update to the community, then. Mweidman (talk) 20:16, 16 October 2008 (UTC)

Thanks for the info (and the disclosure). Can you point more specifically to Matlab functions intended to estimate parameters for such models? --Zvika (talk) 06:54, 17 October 2008 (UTC)

Sure thing. The Econometrics Toolbox creates model objects that simultaneously handle both the conditional mean (AR / MA / ARMA / ARMAX) and conditional variance (ARCH / GARCH / GJR / EGARCH) aspects of volatility models. To create such an object, you would use the GARCHSET command. An ARMA model, for example, would hold the conditional variance specification to a constant. In order to estimate the various parameters for given sample data, you would use the GARCHFIT command. Other useful functions include GARCHSIM and GARCHPRED for Monte Carlo simulation and forecasting, respectively, and the various hypothesis tests that you can use to assess how well your model is fitting the data. Similar functions for VARMA models are also in the toolbox. Mweidman (talk) 18:00, 17 October 2008 (UTC)

Since this article is devoted to ARMA models, I don't believge that mentions of other models are relevant here. You are nevertheless free and welcome to contribute and write specific pages about these models and then add a mention to your software. EtudiantEco (talk) 09:25, 18 October 2008 (UTC)

Supplement material on non-english wiki

The wiki page fr:Processus autorégressif in french includes the features discussed here as well as some further developments and many proofs about the AR(p) process. If anyone is interested in writing a specifical page on AR(p) (instead of concentrating it in the ARMA page) much material is already available. EtudiantEco (talk) 09:25, 18 October 2008 (UTC)

Autoregressive moving average model

Should the model

X_{t}=\varepsilon _{t}+\sum _{i=1}^{p}\varphi _{i}X_{t-i}+\sum _{i=1}^{q}\theta _{i}\varepsilon _{t-i}.\,

include a c (constant) term, like the AR(p) model? Albmont (talk) 16:27, 5 November 2008 (UTC)

I believe that most definitions of both AR and ARMA don't use the constant term. However, to maintain consistency in the article, you are probably right that this should be changed. --Zvika (talk) 17:56, 6 November 2008 (UTC)

Split AR part

I think the AR part is long and pretty much independent of the rest of the article. I would suggest splitting it to Autoregressive model (which currently redirects here). Anybody have any thoughts about this? --Zvika (talk) 18:01, 6 November 2008 (UTC)

P.S. Possibly this should also be done for the MA part, which would then be moved to a (currently stub) article moving average model. --Zvika (talk) 18:02, 6 November 2008 (UTC)

OK, I've split the AR part into autoregressive model. Any comments are welcome. --Zvika (talk) 06:21, 10 November 2008 (UTC)

I think you are too hasty... Four days, two of whom were weekends, are not too much time to discuss a split (not that I disagree, but other people may). Albmont (talk) 17:26, 10 November 2008 (UTC)

I was just being a little BOLD. The article was long and technical, and there is no need for people interested in AR to read through the more complicated ARMA stuff. In any case, this page has few if any others watching it, as you can see from the lack of responses to the last few posts above. --Zvika (talk) 18:19, 10 November 2008 (UTC)

Where is $\mu$ ?

Dumb question I know, but the MA model in this article seems to lack the $\mu$ term that is included in the main article on the MA model ( $X_{t}=\mu +\varepsilon _{t}+\theta _{1}\varepsilon _{t-1}+\cdots +\theta _{q}\varepsilon _{t-q}\,$ vs. $X_{t}=\varepsilon _{t}+\sum _{i=1}^{q}\theta _{i}\varepsilon _{t-i}\,$ ). It's called a moving average model yet appears to lack the term for the average. It is entirely possible (probable actually) that I'm just mentally butchering the sigma notation, but an explanation would be great either wait. Executive Outcomes (talk) 19:48, 21 July 2009 (UTC)

You are right, it's just that the MA model is often (usually?) considered without the constant term. But I added it in for consistency with the MA article. --Zvika (talk) 12:10, 22 July 2009 (UTC)

Contradiction about fitting models

This article states "ARMA models in general can, after choosing p and q, be fitted by least squares regression to find the values of the parameters". This seems to contradict the following statement from the Moving_average_model article: "Fitting the MA estimates is more complicated than with autoregressive models because the error terms are not observable. This means that iterative non-linear fitting procedures need to be used in place of linear least squares." I suspect that the MA article is correct and the statement in this article is incorrect. Anyone know for sure? --Headlessplatter (talk) 20:00, 27 October 2009 (UTC)

stationarity & invertability

stationarity & invertability should be discussed. Jackzhp (talk) 18:08, 11 April 2010 (UTC)

So what is an autoregressive moving average model??

Even though Wikipedia is not a dictionary, articles should begin with a definition and description of a subject, they should provide other types of information about that subject as well. Maybe someone can add a few non-mathematical words that explain what this article is about. http://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#Wikipedia_is_not_a_manual.2C_guidebook.2C_or_textbook —Preceding unsigned comment added by 80.101.105.19 (talk) 14:01, 19 June 2008 (UTC)

ARMA is defined as being a Box-Jenkins model. A Box-Jenkins model is described in terms of being an ARMA model. An uncomfortable circular definition. Thangalin (talk) 21:06, 3 June 2010 (UTC)

URL (web address) of page contains a long hyphen / dash .. non-ASCII code

I find in inacceptable that the current URL of the page contains a non-ASCII code: A "long dash" aka "hyphen" between the 'Autoregressive' and 'moving-average'. If / When you consider changing (to replace the emdash with ASCII "-") I'd wonder (and vote for) simply using "ARMA" which the German and French versions of this wikipage use as well. Maechler (talk) 20:22, 1 December 2012 (UTC)

Many WP articles have titles that include en dash and other non-ASCII characters. See MOS:DASH. Dicklyon (talk) 02:34, 2 December 2012 (UTC)

Equation Missing mean

The equation under Autoregressive moving average model is missing the mean(mew symbol) — Preceding unsigned comment added by Osmankhalid2005 (talk • contribs) 08:37, 26 February 2013 (UTC)

The means of the current and lagged values of X are subsumed in the constant c. Duoduoduo (talk) 14:56, 26 February 2013 (UTC)

Xt or Xn