amj-data-analysis

1# Data Analysis & Validity (amj-data-analysis)
2 
3## When to trigger
4 
5- Data are collected and it is time to estimate and report
6- You are unsure whether your estimator matches your design (nested data, latent constructs, panel)
7- Reviewers will probe measurement validity, common-method bias, or endogeneity
8- Interaction/mediation effects need correct testing and reporting
9- A reviewer says "the analysis does not support the inference" or "validity is not established"
10 
11## Establish measurement before estimation
12 
13AMJ reviewers expect the measurement model to be defended first:
14 
15- **Reliability**: Cronbach's alpha and/or composite reliability for each multi-item scale.
16- **Confirmatory factor analysis (CFA)**: report fit (e.g., CFI, TLI, RMSEA, SRMR) and show the hypothesized factor structure fits better than plausible alternatives (one-factor, combined-factor models).
17- **Convergent & discriminant validity**: AVE per construct; AVE > inter-construct squared correlations (or HTMT). Report the correlation matrix with reliabilities on the diagonal.
18- **Aggregation** (multilevel): justify with ICC(1), ICC(2), and r_wg(j) before aggregating to a higher level.
19- **Qualitative analysis**: where the design is qualitative, "validity" becomes trustworthiness — present a Gioia-style data structure (first-order codes → second-order themes → aggregate dimensions), an audit trail, and representative quotations so the path from raw data to constructs is traceable.
20…