Example 02: Hypothesis Testing

Experimental design in Education

Author
Affiliation

Jihong Zhang*, Ph.D

Educational Statistics and Research Methods (ESRM) Program*

University of Arkansas

Published

August 18, 2025

1 Research Question & Data

Context:

  • Data were collected for two species: O. exclamationis and O. niveus. The data are contained in a data frame called crickets with a total of 31 data points.

  • Study on cricket chirp rates and temperature for two species

  • Data:

    • Temperature (°C)

    • Chirps per minute

    • Species: (A) O. exclamationis (B) O. niveus

Motivation:

  • Scientists are interested in whether the temperature affects crickets’ chirp rates.

Research Questions:

  • RQ1: Does temperature affect chirp rate in general?

  • RQ2: Do species differ in their chirp rates?

  • RQ3: Do species respond differently to temperature?

1.1 Load the data into the R

⌘+C 
## If you did not install modeldata and tidyverse packages, execute following codes:
install.packages("modeldata")
install.packages("tidyverse")
⌘+C 
library(tidyverse)
data(crickets, package = "modeldata")

2 Data Visualization

⌘+C 
ggplot(crickets, 
       aes(x = rate, fill = species, color = species)) + 
  geom_density(alpha = .5)

⌘+C 
ggplot(crickets, 
       aes(x = temp, y = rate, color = species)) + 
  geom_point() +
  geom_smooth(method = lm, se = FALSE) +
  labs(x = "Temperature (C)", 
       y = "Chirp Rate (per minute)")

3 Statistical Hypotheses & Model

Null Hypotheses:

  1. Temperature has no effect on the chirp rate.
  2. There are no differences between the species’ chirp rate, e.g., overall chirp rate and sensitivity to temperature

Null Hypotheses for hypothesis testing:

1. H₀: Temperature has no effect (β_{temp} = 0)

2. H₀: No species difference (β_{species} = 0)

3. H₀: No interaction (β_{temp×species} = 0)

⌘+C 
# Full model with interaction
interaction_fit <- lm(rate ~ (temp + species)^2, data = crickets)

4 Model Diagnostics

⌘+C 
par(mfrow = c(1, 2))
plot(interaction_fit, which = 1) # Residuals vs Fitted
plot(interaction_fit, which = 2) # Normal Q-Q

5 Testing Interaction Effect

⌘+C 
# Fit main effects model
main_effect_fit <- lm(rate ~ temp + species, data = crickets)

# Compare models
anova(main_effect_fit, interaction_fit)
Analysis of Variance Table

Model 1: rate ~ temp + species
Model 2: rate ~ (temp + species)^2
  Res.Df    RSS Df Sum of Sq     F Pr(>F)
1     28 89.350                          
2     27 85.074  1    4.2758 1.357 0.2542

Result: Interaction not significant (p > 0.05). Choose simpler model without interaction

6 Final Model Results

⌘+C 
summary(main_effect_fit)

Call:
lm(formula = rate ~ temp + species, data = crickets)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.0128 -1.1296 -0.3912  0.9650  3.7800 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)       -7.21091    2.55094  -2.827  0.00858 ** 
temp               3.60275    0.09729  37.032  < 2e-16 ***
speciesO. niveus -10.06529    0.73526 -13.689 6.27e-14 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.786 on 28 degrees of freedom
Multiple R-squared:  0.9896,    Adjusted R-squared:  0.9888 
F-statistic:  1331 on 2 and 28 DF,  p-value: < 2.2e-16

Interpretation:

  • Temperature effect: +3.75 chirps/°C (p < 0.001)

  • O. niveus: 17 fewer chirps/minute (p < 0.001)

7 Model Predictions

⌘+C 
new_temps <- data.frame(
  species = "O. exclamationis",
  temp = seq(15, 25, by = 5)
)

predict(main_effect_fit, new_temps)
       1        2        3 
46.83039 64.84415 82.85792

8 Conclusions

Statistical Evidence:

1. Strong temperature effect (p < 0.001)

2. Clear species difference (p < 0.001)

3. Similar temperature response (no interaction)

Limitations:

  • Valid within observed temperature range (15-32°C)

  • Only two species studied

9 Takeaway Note: the key steps

Here are the key steps for performing inferential statistics using the cricket example:

  1. Data visualization and propose Null Hypotheses
    • Temperature has no effect on chirp rate
    • No differences between species’ chirp rates
  2. Fit Initial Model
    • Used lm() with interaction terms between temperature and species
    • Formula: rate ~ (temp + species)^2
  3. Check Model Diagnostics
    • Examined residual plots vs predicted values
    • Checked normality of residuals using Q-Q plot
    • Verified assumptions were reasonable
  4. Test Model Components
    • Compared models with/without interaction using ANOVA
    • Found interaction term was not statistically significant (p > 0.05)
    • Selected simpler main effects model
  5. Interpret Final Model
    • Analyzed coefficients and p-values for temperature and species effects
    • Temperature showed significant positive effect
    • Species showed significant difference in baseline chirp rates
    • Limited conclusions to observed temperature range
Back to top