README.Rmd

---
title: "`sadj`"
author: "Chris Hansen, Steve White"
date: "`r Sys.Date()`"
output:
  md_document:
    variant: gfm
vignette: >
  %\VignetteIndexEntry{Quick Start}
  %\VignetteEngine{knitr::rmarkdown}
  %\usepackage[utf8]{inputenc}
---
  

  
```{r setup, include=FALSE, message = FALSE, warning = FALSE}
knitr::opts_chunk$set(echo = TRUE, cache = TRUE, warning = FALSE,
                      message = FALSE, comment = NA, 
                      fig.align = "center", 
                      fig.height = 4, fig.width = 5)
library(sadj)
library(kableExtra)
```

# sadj

The `sadj` package allows for relatively flexible use of the underlying
`X13-ARIMA-SEATS` program.  

__Copyright and Licensing__

The package is Crown copyright (c) 2021, Statistics New Zealand on behalf of the New Zealand Government, and is licensed under the MIT License (see LICENSE file).

<br /><a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This document is Crown copyright (c) 2021, Statistics New Zealand on behalf of the New Zealand Government, and is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

---


## Single Series

Results can be produced quickly and easily by
accepting default parameters.  Consider the following:

```{r simpleex}
ap <- X13Series(AirPassengers)
```

This produces an object, `ap`, of class `X13Series`.  This object is initialised
with the built-in `AirPassengers` object--a time series holding the classic Box
and Jenkins airline passengers data.  Once imported, the data is coerced into a
`data.frame` which looks as follows:

```r
head(ap, 10)
```
```{r simpleextab, echo = FALSE}
knitr::kable(head(ap, 10), row.names = FALSE, format = "markdown")
```

In fact, while we provide the ability to create an `X13Series` object directly
from a `ts` object because it is a commonly used type in R, we can also
create one from a `data.frame` like the one displayed above.

An `X13Series` object has a number of attributes stored interally, such as the
adjustment specifications required by the `X13-ARIMA-SEATS` program.  For
example, the specifications are held in an attribute called `SpecList`, and can
be retrieved via `getSpecList(ap)`:

```{r simpleex.spec}
getSpecList(ap)
```

The specification is actually an R list, but it has a print method that displays
a familiar string representation.  In this case, a default, minimal
specification was provided, but there are facilities for providing different
specifications when creating `X13Series` objects, or modifying specifications
for existing objects.  The details of working with specifications are outlined
in a vignette, `vignette("specifications", package = "sadj")`.

In this simple example, no adjustment specifications were explicitly provided.
In actual fact, though, a default specification was provided, and the original
call to `X13Series` could have been written in the following equivalent way:

```r
ap1 <- X13Series(AirPassengers, type = "x11")
```

If the SEATS adjustment method is preferred over X11, then a default SEATS
adjustment is also readily available:

```{r seats01}
ap2 <- X13Series(AirPassengers, type = "seats")
getSpecList(ap2)
```

Seasonal adjustment is carried out by calling `adjust` on an adjustable object:

```{r simpleex.adjust}
ap.res <- adjust(ap)
```

The result in this case is of class `X13SeriesResult`.  It is a data frame
containing the raw series and `X13-ARIMA-SEATS` output tables such as `d10`,
`d11`, and `d12` as columns:

```r
head(ap.res, 10)
```
```{r simpleex.res, echo = FALSE}
knitr::kable(head(ap.res, 10), row.names = FALSE, format = "markdown")
```

In fact, any output produced by `X13-ARIMA-SEATS` that is a time series is added
as a column to the output data frame.  The columns can vary depending on the
adjustement specification and, besides, are somewhat esoteric.  For this reason,
helper functions exist for extracting the more common components of an adjusted
series in a more uniform way.  For example, the raw, seasonally adjusted and
trend series can be accessed by calling `actual`, `seasadj`, and `trend`,
respectively.

```r
head(trend(ap.res), 10)
```
```{r simpleex.trend, echo = FALSE}
knitr::kable(head(trend(ap.res), 10), row.names = FALSE, format = "markdown")
```

Output that isn't a times series is stored as an attribute.  For example, the
'`UDG`' file, if present, is a file containing a range of summary statistics and
the like, and is read in as a list and stored as an attribute called `udg`.
Many of these attributes aren't directly useful, and helper functions are
instead provided which give the appropriate access.  For example, calling
`summary` on an adjusted series will cause the HTML version of the output to
display, or else it will cause a relatively pretty tabular version of the `udg`
attribute to display:

```r
summary(ap.res, html = TRUE)
```
![`summary(ap.res)`](img/htmlout.png)

or:

```r
summary(ap.res)
```
```{r simpleex.res.summary, echo = FALSE}
knitr::kable(summary(ap.res), row.names = FALSE, format = "markdown")
```

There are a range of built-in plotting functions provided to visualise the
results:

```{r simpleex.res.plot1}
plot(ap.res)
```

```{r simpleex.res.plot2}
plot(ap.res, type = "d10")
```






## Batch Series

The first thing you might want to do is run seasonal adjustment on your existing `X13-ARIMA-SEATS` file setup and evaluate the results. You might do something like the following:

1.) Read the series batch in from an `.mta` file:  

```r
mta_path <- "~/Network-Shares/corp-nas/seasadj/hlfs/hlfs.mta"
hlfs <- X13BatchFromMTA(mta_path)
```

2.) Seasonally adjust the series batch:

```r
hlfs_res <- adjust(hlfs)
```
3.) View summary information of the result:

```r
print(hlfs_res)
summary(hlfs_res)

# Print T-vals of regressors
tvals(hlfs_res)

```
### Interrogate  

4.) Interrogate specific series from the batch

```r
library(magrittr) # for pipes

# Use the snames from print/summary of the batch result to plug into `selectSeries`:
munemp_res <- hlfs_res %>% selectSeries("munemp")

# Spin up a Shiny to view plots and diagnostics
munemp_res %>% view()

# View summary diagnostic information
munemp_res %>% summary()

# View full html diagnostic report
munemp_res %>% summary(html=TRUE)

# What plots are there?
?sadj:::plot.X13SeriesResult

# Try a plot
munemp_res %>% plot(type="D10D8")

# How about an interactive plot?
munemp_res %>% plot(type="D10D8", interactive = TRUE)


```
5.) Interrogate an entire batch

```r
# Query Specs

# getParamVals.X13Batch
res <- getParamVals(hlfs,"series","span") %>% map( ~.x[[1]])

# getSpecParameter.X13Batch
# Same as getParamVals but it doesn't try to parse individual values
# Example of querying documenation
?sadj:::getSpecParameter.X13Batch 

```


### Update/Modify

6.) Update/Modify an entire batch

```r
# Modify Specs

# Add/remove outliers
# Add a Level shift to all series in a batch
# (Try querying to verify this)
addOutliers(hlfs) <- "LS2020.2"

# Remove the level shift
removeOutliers(hlfs) <- "LS2020.2"
# By default, removeOutliers does not clear out the save argument
# That is something to be fixed, but for now:
setSpecParameter(hlfs, "regression", "save") <- NULL

# Add outliers to a subset of series:
setSpecParameter(hlfs
                 , snames = c("munemp", "funemp")
                 ,"regression", "variables") <- c("LS1980.3")

```


## Sadj objects Summary
 

### Input Objects  

```{r input_objects, echo = FALSE}
data.frame(object=c("X13Batch", "X13SeriesGroup", "X13Series"),
           structure=c(
             "A list of X13SeriesGroup objects",
             "A list of X13Series objects",
             "A dataframe of time series data"
             ),
           comments = c(
             "A batch of X13Series objects as specified by an MTA file.",
             "X13Series object groupings which are delineated by empty spaces inside an MTA file (see note). Either 0 or 1 composites are allowed at the end of a group and nowhere else.",
             "Can have several attributes to represent the following X13 files that can be associated with a single series: spc, fac, regression.  Read the X13Series section to learn how to access and modify these attributes"
                        )
           ) %>% 
  knitr::kable(row.names = FALSE, format = "markdown", col.names = c("Object", "Structure", "Comments"))
```

### Output Objects  

```{r output_objects, echo = FALSE}
data.frame(object=c("X13SeriesResult","X13SeriesGroupResult", "X13BatchResult"),
           structure=c(
             "A dataframe of seasonally adjusted time series data",
             "A list of X13SeriesResult objects",
             "A list of X13SeriesGroupResult objects"),
           comments = c(
             "Has attributes that represent quality diagnostics and calculated modelling parameters",
             "X13SeriesResult object groupings which are delineated by empty spaces inside an MTA file (see note). Either 0 or 1 composites are allowed at the end of a group and nowhere else.",
             "A batch of X13SeriesResult objects as specified by an MTA file." 
                        )
           ) %>% 
  knitr::kable(row.names = FALSE, format = "markdown", col.names = c("Object", "Structure", "Comments"))
```

**Note: Empty lines in an MTA file do not mean anything to the X13 program.  They are a Statistics NZ convention and they do mean something to the sadj package - The groupings are submitted to X13 as their own batches.**

## Interacting with Objects

### Modifying Seasonal Adjustment Congfigurations (SPC files)
`X13Series` is a dataframe with time series input data.  It has attributes that are accessible/mutable with the following functions:  

```{r getsettab, echo = FALSE}
data.frame(get=c("getSpecList","getSpec","getSpecParameter", "getFacFile", "getRegFile"),
           set=c("setSpecList","setSpec","setSpecParameter", "setFacFile", "setRegFile"),
           comments=c("Method gets passed to the X13SpecList object which is attached to X13Series as an attribute",
                      "Same as comment above",
                      "Same as comment above",
                      "get/set the factor file dataframe attached to the the X13Series",
                      "get/set the regression file dataframe attached to the the X13Series")) %>% 
  knitr::kable(row.names = FALSE, format = "markdown")
```





## X13Batch Objects


## Vignettes