Lesson 4

Education Research

Introduction

Statistical and qualitative analysis are beyond the scope of this course.  Don’t let the below discussion freak you out… there are only a few takeaways you should focus on:

The lesson addresses these take-aways:

  • There are F/OSS alternatives to the expensive software frequently encountered in educational research
  • When “required” to use proprietary research software, investigate further before committing your funds

Advanced Organizer

During the lesson, look for answers to the following questions –

  1. Why do educational researchers continue to use proprietary software?
  2. What alternatives to SPSS are available?
  3. How can students raise awareness of F/OSS alternatives without frustrating their instructors?

Walk-through

1. Educational Research – Overview and Simple Examples

Academicians are engaged in the pursuit, discovery and dissemination of new knowledge. Educational research is the practice of formulating research questions, assessing current knowledge related to those questions, designing research methodologies, applying those designs, and answering the research questions to the extent possible.  This process generally involves the collection and analysis of data – both numerical (quantitative) and non-numerical.

As a simple example, consider how an instructor assesses their own performance in teaching a course. They may wish to answer questions such as:

  1. Was the pace of my class optimal?
  2. How can I improve my instruction next time?

There is seldom a single correct method to approach research questions.

Quantitative Example

For the first question, the instructor might ask the students to respond based on a Likert scale survey, ie:

1. The pace of this class was:

    1. Too slow
    2. Somewhat slow
    3. Neither too fast nor too slow
    4. Somewhat fast
    5. Too fast

The data collected would be numerical – a set of numbers, with values of 1 through 5.  In this simple example, the data could be analyzed with a variety of applications. Any spreadsheet program, such as LibreOffice Calc, would enable the researcher to calculate central tendency with descriptive statistics such as the mean (arithmetic average), median (the midpoint of all responses), and mode (value that appears most often).  However, a spreadsheet would not be so directly useful if we also wanted to examine the relationship between students’ response about the pace of class and their overall GPA.  To do that, a researcher would want to use more complex analyses (such as cross tabulations, scatterplots, or measures of dependence). For such analyses, it is advantageous to use dedicated statistical analysis software.

Qualitative Example

The second question in the example would not be easy to answer using numeric data. It seems simpler to collect open-ended feedback from students, ie:

4. Please describe how the instruction of this course could be improved:  _____________

In this case it is harder to address central tendency or relationships to other variables (such as a student’s overall GPA).  However, using qualitative analysis, a researcher might categorize the responses based on a defined scheme.  For example, a response “Comments on assignments could be faster”, might be coded as “Relating to instructor response time” and assigned a numeric value.  In this manner, the variety of responses that address the instructor’s response time can be aggregated and examined.  For example, a researcher could address the proportion of comments about instructor response time as a percentage of all responses.  This is a very simple qualitative example, and there are many software packages meant for coding and analyzing text, audio, and video data.

2. Common Educational Research Software

Most researchers are quite familiar with Microsoft Excel, and you already know that an excellent F/OSS alternative is LibreOffice Calc.

Common statistical analysis packages are SPSS (Statistical Package for Social Sciences) and SAS. Many graduate statistics courses require one or the other. There is a host of other statistical software available, see this list maintained by Wikipedia.

The most commonly referenced qualitative data analysis software in my experience is NVivo.  However, there is also an extensive Wikipedia list of alternatives for quantitative analysis.

3. F/OSS Statistical Software

The software described in this section is included in the preconfigured Ubuntu Virtual Machine that you installed during lesson 3.4 – feel free to explore it. There are two GUI-based statistics packages you should be aware of for quantitative analysis. The first is PSPP, a GNU project aimed at replacing the proprietary SPSS.

pspp

PSPP is a meaningless acronym, but meant to convey the opposite of SPSS – which requires a fairly expensive license, yet enjoys widespread use in higher education institutions.  You can review the PSPP FAQ for details. Feedback from other professors, and from some online reviews indicate that PSPP is good for introductory statistics, as it corresponds to SPSS base.  PSPP, unlike SPSS, does not limit the number of cases or variables analyzed.  Unlike SPSS, there are no advanced modules to license in addition to the base functionality.

rkward

Another powerful GUI-based statistics package is RKward.  RKward serves as a graphical front-end to R, which is an entire F/OSS statistics programming language.  R is GPL software and extremely powerful – but there is a steep learning curve. The RKward GUI provides one way to use R without the steep learning curve, however, it only scratches the surface of R. You can review the RKWard FAQ for details.

elan

Finally, a powerful Qualitative Analysis package for your future reference is ELAN. Should you find yourself wishing to engage in qualitative research at some point, you can obtain the ELAN software and user guide from the Max Planck Institute for Psycholinguistics – where it is developed.

4. Substituting for Proprietary Software

Students and researchers should evaluate the statistical analyses they need before licensing proprietary software. Because of learning curve investment, instructors tend to teach using the same software they learned on – even when a F/OSS alternative exists.  It is reasonable to inquire if alternative software can be used before you pay for software in class or research projects.  The following list provides some idea as to what can be accomplished with PSPP and RKward – without licensings SPSS.  Consult this list, or install the software and explore the menus and help system, before licensing proprietary software that do the same things.

PSPP

  • Descriptive Statistics: Frequencies, Crosstabs
  • Compare Means: T test (one sample, paired samples, independent samples), One-way ANOVA
  • Bivariate Correlations
  • K-means Cluster
  • Factor Analysis
  • Reliability
  • Linear Regression
  • Non-parametrics (Chi square, Runs test, 1-sample K-S, 2 Related samples, K Related samples)
  • ROC Curve

RKward

  • Correlation (Matrix, Matrix Plot)
  • Crosstabs (N to 1, N to N)
  • Item Response TheoryTwo-variable T testMoments (Anscombe-Glynn test of Kurtosis, Bonnett-Seier Test of Geary’s Kurtosis, D’Agostino Test of Skewness)
    • Dichotomous data (Rasch, two parameter, Birnbaum, linear logistic model fits)
    • Polytomous data (Graded response, rating scale, Partial credit, Generalized partial credit, linear rating scale, linear partial credit model fits)
    • Item Response Theory, Tests (Goodness of fit, Unidimensionality check, Item-fit, Person-fit, Wald test, Anderston L-R plot)
    • Cronbach’s alpha
  • Outlier Tests (Chi-squared Test for Outlier, Dixon Test, Gubbs Test, Find Outlier)
  • Linear Regression
  • Time Series (Box-Pierce or Ljung-Box Tests, Hodrick-Prescott Filter, KPSS Test fo Stationarity, Phillips-Perron Test)
  • Variances, Parametric (Bartlett Test, F-Test, Levene’s Test)
  • Variances, Nonparametric (Ansari-Bradley Two-Sample, Fligner-Killeen Test, Mood Two-Sample)
  • Wilcoxon Tests

RKward can also perform a wide range of plots and distributions analyses. Rather than list them all here, please review RKward’s menus and online documentation. Professional researchers will probably want to learn R itself.

Summary of Readings & Media

  1. PSPP FAQ
  2. RKward FAQ

Activity

TBD

Assignments

  1. Test: Module 4

Additional Material

  1. PSPP Tutorial from NC State
  2. PSPP Overview – with discussion of charting limitations
Back To Top