# Chapter 1 example dataset: `adcorr`

A simulated COA validation study, built for The Psychometrician's Handbook, Chapter 1
(Correlations). Every code example in the chapter runs on this one dataset.

The data are synthetic. No patient contributed to them. They were generated from a defined model so
that each teaching point in the chapter is a property of the data.

## Files

| File | Description |
|---|---|
| `adcorr.csv` | The dataset. Long CDISC ADaM BDS, 10,400 rows. |
| `01_simulate_coa_data.R` | The program that generates the dataset AND every code block from the chapter. |
| `README_data_dictionary.md` | This file. |

Run the R program to regenerate `adcorr.csv`. With `set.seed(2026)` you get exactly the
file same file.

## Structure of the dataset

One row per subject, per parameter, per visit. 200 subjects, 18 parameters, 52 rows each.

| Column | Description |
|---|---|
| `STUDYID` | Study identifier, always `PHB-CH1` |
| `USUBJID` | Unique subject identifier, `PHB-CH1-001` to `PHB-CH1-200` |
| `SUBJID` | Subject number, 1 to 200 |
| `PARAMCD` | Short parameter code |
| `PARAM` | Parameter label, including its statistical type |
| `PARAMN` | Parameter order, 1 to 18 |
| `AVISIT` | Visit label |
| `AVISITN` | Visit number |
| `AVAL` | The value |
| `BASE` | First-visit value, for parameters measured more than once |
| `CHG` | `AVAL − BASE`, for parameters measured more than once |

`BASE` and `CHG` are empty for the 12 parameters collected once. They are populated for `TARGTOT`,
`FINTOX`, `SYMBUR`, `DIARY1` and `DIARY2`.

## Parameters

### Continuous scales, for convergent and discriminant validity

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `TARGTOT` | continuous | Baseline, Retest, Week 12 | 0 to 100 |
| `RELATEDA` | continuous | Baseline | 0 to 100 |
| `RELATEDB` | continuous | Baseline | 0 to 100 |
| `UNRELATD` | continuous | Baseline | 0 to 100 |

### Ordinal items, one domain

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `ITEM01` to `ITEM04` | ordinal, 5-point Likert | Baseline | 1 to 5 |

Generated from a single latent factor, so they have a known underlying correlation.

### Skewed continuous and performance

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `FATVAS` | continuous, right-skewed | Baseline | 0 to 100 |
| `WALK6MIN` | continuous, approximately normal | Baseline | metres, mean ≈ 400 |

### Binary

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `SEVGRP` | True dichotomous variable | Baseline | 0, 1 |
| `DICHOTOM` | continuum cut at a clinical cutoff | Baseline | 0, 1 |

The two are different on purpose. `SEVGRP` is a real either/or with nothing continuous underneath.
`DICHOTOM` is a severity score cut at a threshold, so a continuum is hiding behind the 0 and the 1.
That difference decides which coefficient choice is correct (biserial/pointbiserial), which is the point of the slide that uses them.

### Anchors

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `PGIS` | ordinal anchor | Baseline | 1 to 5 |
| `PGIC` | ordinal anchor | Week 12 | 1 to 7 |

### Repeated measures

| `PARAMCD` | Type | Visits | Range |
|---|---|---|---|
| `FINTOX` | continuous, monthly | Month 1 to 4 | 0 to 100 |
| `SYMBUR` | continuous, monthly | Month 1 to 4 | 0 to 100 |
| `DIARY1` | continuous, daily | Day 1 to 14 | 0 to 10 |
| `DIARY2` | continuous, daily | Day 1 to 14 | 0 to 10 |

The two pairs behave in opposite ways (which is why there are two of them). For `FINTOX` and
`SYMBUR` the relationship lives between people: patients with worse financial toxicity have worse
symptom burden, but a given patient's month-to-month wobbles barely track each other. For `DIARY1`
and `DIARY2` the reverse holds. There is no between-person relationship at all, yet within a single
patient the two diary scores move together closely. the take home is that when you 
pool the rows and that relationship almost disappears.

## Working with the data

The data are long, so most analyses pivot to wide first. This is a deliberate choice because clinical trial data is long:

```r
library(dplyr); library(tidyr)

adcorr <- read.csv("adcorr.csv")

wide <- adcorr %>%
  filter(AVISIT == "Baseline") %>%
  pivot_wider(id_cols = USUBJID,
              names_from = PARAMCD,
              values_from = AVAL)

cor(wide$TARGTOT, wide$RELATEDA)
```

The repeated-measures parameters stay long, since the analyses that use them need the visit
structure.

## What the data were built to show

Values below are from the dataset. running the file as it is run will reproduce them exactly.

| Relationship | Result |
|---|---|
| `TARGTOT` and `RELATEDA`, convergent | r = 0.73, 95% CI 0.66 to 0.79 |
| `TARGTOT` and `UNRELATD`, divergent | r = 0.19 |
| `ITEM02` and `ITEM03`, ordinal | Pearson 0.65, Spearman 0.62, polychoric 0.71 |
| `PGIS` and `TARGTOT`, ordinal against continuous | Pearson 0.68, polyserial 0.72 |
| `TARGTOT` and `SEVGRP`, true dichotomy | point-biserial 0.46 |
| `TARGTOT` and `DICHOTOM`, cut continuum | point-biserial 0.37, biserial 0.48 |
| `TARGTOT` Baseline against Retest | ICC(2,1) 0.87, 95% CI 0.83 to 0.90 |
| `FATVAS` and `WALK6MIN` | Pearson −0.90, Spearman −0.94 |
| Change in `TARGTOT` against `PGIC` | r = 0.40 |
| `FINTOX` and `SYMBUR` | naive 0.54, within 0.23, between 0.64 |
| `DIARY1` and `DIARY2` | naive 0.24, within 0.54, between 0.14 |

In each of the last four rows the naive answer is the misleading one, which is the reason those
examples are in the chapter.

## Reproducing

```r
source("01_simulate_coa_data.R")   # rebuilds adcorr and writes adcorr.csv
```

Requires R 4.x and: dplyr, tidyr, ggplot2, psych, polycor, rmcorr, lme4, nlme, corrr, corrplot,
blandr, mirt, MASS. The script installs anything missing.