Introduction to Applied Statistics for Psychology Students

Gordon E. Sarty

Osama Bataineh

Introduction to Applied Statistics for Psychology Students by Gordon E. Sarty is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

See notes on specific copyright for screenshots from IBM® SPSS® Statistics software (“SPSS”) and R Project for Statistical Computing (“RStudio”).

Contents

- About This Book

- Licensing and Copyright

- Acknowledgements

- Statistical Software Used in this Book

- Accessing SPSS and RStudio Through Your School

- Downloading SPSS and RStudio

- Why does this book cover both SPSS and RStudio?

- University of Saskatchewan: Software Access

- On-Campus Lab Access

- Remote / Off-Campus Access

- USask ICT Help

- Data Sets

- 1. Background and Motivation

- 1.1 Overview

- 1.1.1 Textbook Layout, * and ** Symbols Explained

- 1.1.2 Intro to Univariate Statistics

- 1.2 Basic Definitions

- 1.2.1  Types of Data (important!)

- 1.2.2 Measurement Scales (avoid this!)

- 1.2.3 Kinds of Sampling and Studies

- 1.3 Summation Convention

- 2. Descriptive Statistics: Frequency Data (Counting)

- 2.1 Frequency Tables

- 2.2 Plotting Frequency Data

- 2.2.1 Stem and Leaf Plots

- 2.3 SPSS Lesson 1: Getting Started with SPSS

- 2.4 RStudio Lesson 1: Getting Started with RStudio

- 3. Descriptive Statistics: Central Tendency and Dispersion

- 3.1 Central Tendency: Mean, Median, Mode

- 3.1.1 Mean

- 3.1.2 Median

- 3.1.3 Mode

- 3.1.4 Midrange

- 3.1.5 Mean, Median and Mode in Histograms: Skewness

- 3.1.6 Mean, Median and Mode in Distributions: Geometric Aspects

- 3.2 Dispersion: Variance and Standard Deviation

- 3.3 z-score / z-transformation

- 3.4 SPSS Lesson 2: Combining variables and recoding

- 3.5 RStudio Lesson 2: Combining variables and recoding

- 4. Probability and the Binomial Distributions

- 4.1 Probability

- 4.2 Binomial Distribution

- 4.2.1 Practical Binomial Distribution Examples

- 4.3 SPSS Lesson 3: Combining variables - advanced

- 4.4 RStudio Lesson 3: Combining variables - advanced

- 5. The Normal Distributions

- 5.1 Discrete versus Continuous Distributions

- 5.2 **The Normal Distribution as a Limit of Binomial Distributions

- 5.3 Normal Distribution

- 5.3.1 Computing Areas (Probabilities) under the standard normal curve

- 6. Percentiles and Quartiles

- 6.1 Discrete Data Percentiles and Quartiles

- 6.2 Finding Outliers Using Quartiles

- 6.3 Box Plots

- 6.4 Robust Statistics

- 6.5 SPSS Lesson 4: Percentiles

- 6.6 RStudio Lesson 4: Percentiles

- 7. The Central Limit Theorem

- 7.1 Using the Normal Distribution to Approximate the Binomial Distribution

- 7.2 The Central Limit Theorem

- 8. Confidence Intervals

- 8.1 Confidence Intervals Using the z-Distribution

- 8.2 **Bayesian Statistics

- 8.3 The t-Distributions

- 8.4 Proportions and Confidence Intervals for Proportions

- 8.5 Chi Squared Distribution

- 9. Hypothesis Testing

- 9.1 Hypothesis Testing Problem Solving Steps

- 9.2 z-Test for a Mean

- 9.2.1 What p-value is significant?

- 9.3 t-Test for Means

- 9.4 z-Test for Proportions

- 9.5 Chi Squared Test for Variance or Standard Deviation

- 9.6 SPSS Lesson 5: Single Sample t-Test

- 9.7 RStudio Lesson 5: Single Sample t-Test

- 10. Comparing Two Population Means

- 10.1 Unpaired z-Test

- 10.2 Confidence Interval for Difference of Means (Large Samples)

- 10.3 Difference between Two Variances - the F Distributions

- 10.4 Unpaired or Independent Sample t-Test

- 10.4.1 General form of the t test statistic

- 10.4.2 Two step procedure for the independent samples t test

- 10.5 Confidence Intervals for the Difference of Two Means

- 10.6 SPSS Lesson 6: Independent Sample t-Test

- 10.7 RStudio Lesson 6: Independent Sample t-Test

- 10.8 Paired t-Test

- 10.9 Confidence Intervals for Paired t-Tests

- 10.10 SPSS Lesson 7: Paired Sample t-Test

- 10.11 RStudio Lesson 7: Paired Sample t-Test

- 11. Comparing Proportions

- 11.1 z-Test for Comparing Proportions

- 11.2 Confidence Interval for the Difference between Two Proportions

- 12. ANOVA

- 12.1 One-way ANOVA

- 12.2 Post hoc Comparisons

- 12.3 SPSS Lesson 9: One-way ANOVA

- 12.4 R Lesson 9: One-way ANOVA

- 12.5 Two-way ANOVA

- 12.6 SPSS Lesson 9: Two-way ANOVA

- 12.7 R Lesson 9: Two-way ANOVA

- 12.8 Higher Factorial ANOVA

- 12.9 Between and Within Factors

- 12.10 *Contrasts

- 13. Power

- 13.1 Power

- 14. Correlation and Regression

- 14.1 Scatter Plots

- 14.2 Correlation

- 14.3 SPSS Lesson 10: Scatterplots and Correlation

- 14.4 R Lesson 10: Scatterplots and Correlation

- 14.5 Linear Regression

- 14.6 r-squared and the Standard Error of the Estimate of y-prime

- 14.7 Confidence Interval for y-prime at a Given x

- 14.8 SPSS Lesson 11: Linear Regression

- 14.9 R Lesson 11: Linear Regression

- 14.10 Multiple Regression

- 14.11 SPSS Lesson 12: Multiple Regression

- 14.12 R Lesson 12: Multiple Regression

- 15. Chi Squared: Goodness of Fit and Contingency Tables

- 15.1 Goodness of Fit

- 15.2 Contingency Tables

- 15.3 SPSS Lesson 13: Proportions, Goodness of Fit, and Contingency Tables

- 15.4 R Lesson 13: Proportions, Goodness of Fit, and Contingency Tables

- 16. Non-parametric Tests

- 16.1 How to Rank Data

- 16.2 Median Sign Test

- 16.3 Paired Sample Sign Test

- 16.4 Two Sample Wilcoxon Rank Sum Test (Mann-Whitney U Test)

- 16.5 Paired Wilcoxon Signed Rank Test

- 16.6 Kruskal-Wallis Test (H Test)

- 16.7 Spearman Rank Correlation Coefficient

- 16.8 SPSS Lesson 14: Non-parametric Tests

- 16.9 R Lesson 14: Non-parametric Tests

- 16.10 Runs Test

- 17. Overview of the General Linear Model

- 17.1 Linear Algebra Basics

- 17.2 The General Linear Model (GLM) for Univariate Statistics

- Appendix: Tables

1

About This Book

Introduction to Applied Statistics for Psychology Students, by Gordon E. Sarty (Professor, Department of Psychology, University of Saskatchewan) began as a textbook published in PDF format, in various editions between 2014-2017. The book was written to meet the needs of University of Saskatchewan psychology students at the undergraduate (PSY 233, PSY 234) and graduate (PSY 807) levels.

In 2019-2020, funding was provided through the Gwenna Moss Centre for Teaching and Learning, along with technical assistance from the Distance Education Unit, to update and adapt this book, making it more widely available in an easy-to-use and more adaptable digital (Pressbooks) format. This update included an expansion to add chapters on using RStudio, as an alternative to SPSS. The update also made revisions so that the book could be published with a license appropriate for open educational resources (OER).

OERs are defined as “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others” (Hewlett Foundation). This textbook and other OERs like it are openly licensed using a Creative Commons license, and are offered in various digital and e-book formats free of charge.

Printed editions of this book can be obtained for a nominal fee through the University of Saskatchewan bookstore.

Licensing and Copyright

Licensing

Except where otherwise noted (see notes below on the copyright for SPSS and R screenshots), the content of this book is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Under the terms of the CC BY-NC-SA license, you are free to copy, redistribute, modify or adapt this book as long as you provide attribution. You may not use the material for commercial purposes. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. Additionally, if you redistribute this textbook, in whole or in part, in either a print or digital format, then you must retain on every physical and/or electronic page an attribution to the original author(s).

Copyright: SPSS Screenshots

SPSS Inc. was acquired by IBM in October, 2009. Reprints of images (i.e., screenshots) from IBM® SPSS® Statistics software (“SPSS”) appear courtesy of International Business Machines Corporation, © International Business Machines Corporation. IBM, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “IBM Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml. This consolidated credit paragraph and corresponding copyright notices must be listed on a title page or other conveniently viewable location where any reprints of this material appear. Any repurposing of the material in this book should also follow these same requirements.

The University of Saskatchewan Open Press obtained specific permissions from IBM to reprint IBM SPSS Statistics screen images for the purposes of publishing this book, according to the conditions outlined here. Individuals who wish to use, duplicate, or redistribute any of these images are advised to do so in compliance with copyright law or to contact IBM directly for permissions: http://www.ibm.com/contact/submissions/extsub.nsf/copyright. If any derivative version of this book (i.e., remixed, transformed, modified, or built-upon version) is created, additional copyright permission from IBM should be acquired for including any of their images in the derivative version before it is released.

Copyright: RStudio Screenshots

Reprints of images (i.e., screenshots) from RStudio are © the R Foundation, from http://www.r-project.org, and may be reproduced for any purpose provided they are credited to the R statistical software using an attribution such as this.

Cover Image

Cover image by Ron Borowsky and Gordon Sarty, used for public talks and released with a CC BY-NC-SA license. The statistical methods that you will learn in this course were necessary to produce the functional MRI (fMRI) brain maps illustrated on the cover. In particular, a one-way ANOVA technique was used to detect the brain activations shown in the images[1]. The study shown was designed to reveal ventral and dorsal stream processing for ‘what’, ‘where’ and ‘how’ interpretations of words and pictures presented to the experimental subjects while they were in the Magnetic Resonance Imager (MRI)[2].

1. Sarty GE, Borowsky R. “Functional MRI Activation Maps from Empirically Defined Curve Fitting”, Concepts in Magnetic Resonance Part B (Magnetic Resonance Engineering), 24B, 46-55, 2005. ↵

2. Borowsky R, Loehr J, Friesen CK, Kraushaar G, Kingstone A, Sarty GE, “Modularity and Intersection of ‘What’, ‘Where’, and ‘How’ Processing of Visual Stimuli: A New Method of fMRI Localization”, Brain Topography, 18, 67-75, 2005. ↵

2

Acknowledgements

A sincere thank you must go out to the following University of Saskatchewan personnel, in acknowledgement of their support and contributions to this updated open textbook:

- Julie Maier (Instructional Designer, Distance Education Unit), for technical assistance with Pressbooks, OER and licensing guidance, editing and formatting assistance, developing resources for statistical software access, and project coordination.

- Heather Ross (Educational Development Specialist, Gwenna Moss Centre for Teaching and Learning), for support with obtaining the funding that allowed this project to move forward.

- Kate Langrell (Copyright Coordinator, University of Saskatchewan Library), for answering copyright questions, particularly regarding the use of software screenshots and data files.

- Naveed Ahmed (Research Associate, Department of Agriculture and Resource Economics), for content updates and updated data sets for SPSS Lessons 1 to 7.

- Osama Bataineh (Lab Coordinator & Sessional Lecturer, Department of Mathematics & Statistics), for major content editing, editing the SPSS Lessons and writing the new RStudio Lessons throughout this book, devising and compiling the complete collection of finished data sets, porting material into Pressbooks, LaTeX refinement, and – last but not least – piloting this new version of the textbook with his PSY 233 students for the first time in Spring 2020.

3

Statistical Software Used in this Book

Throughout this book you will find Lessons that will take you through procedures to manipulate and analyze given data using two statistical software applications:

1. IBM® SPSS® Statistics software (referred to more simply as “SPSS”)

2. RStudio, from The R Project for Statistical Computing.

Accessing SPSS and RStudio Through Your School

See the page University of Saskatchewan: Software Access for more details on how to do this.

Downloading SPSS and RStudio

SPSS Statistics

SPSS Statistics is not a free program.

A trial version of SPSS can be downloaded at: https://www.ibm.com/analytics/spss-trials

If you really want to download the program (not in a trial version), see some information on student rates at: https://www.ibm.com/analytics/academic-statistical-software; however, consider carefully how necessary this is before you spend any of your own money, and look carefully at any terms of licensing (i.e., some licenses may only give you access for a set number of months). Unless you are in a position where you can get an employer or research supervisor to pay for it, you may want to stick with the cost-free options available to you.

RStudio

A free, open-source, non-commercial desktop version of RStudio can be downloaded at: https://rstudio.com/products/rstudio/download/

Why does this book cover both SPSS and RStudio?

While both SPSS and RStudio are powerful analytical tools, they operate differently and each have their pros and cons.

The history of SPSS Statistics goes back to the 1960s, and for many years it has been a standard for students and researchers working in the social sciences (SPSS, in fact, originally stood for Statistical Package for the Social Sciences, but was later changed to Statistical Product and Service Solutions). It is still an extremely popular and commonly-used package, and one that you are likely to find is used in labs and workplaces when you start to search for research and employment positions. For this reason, it is still essential for psychology graduates to have a solid grasp of how to use this program.

The more-recent R Project for Statistical Computing (which put together RStudio) offers a free, open-source option. This means that anybody can access the software, and its community of users and developers can contribute to improving and updating the software. While it is increasing in popularity, it has yet to reach the ubiquitousness of SPSS.

So which Lessons should you do? SPSS or RStudio?

Well, first ask your instructor — if they want you to submit assignments that utilize a particular program, or if they are likely to ask you to interpret outputs from a particular program on your examinations, then you’d better complete the Lessons for that program!

Second, consider what skills you want to develop. Employers and/or supervisors are still very likely to expect psychology graduates to have SPSS skills. Would it benefit you, or set you apart from other research/employment candidates, to have a good grasp of both SPSS and RStudio? Do both sets of Lessons and practice using each program.

If you are a graduate student or somebody with a bit more freedom, who is just looking for some help analyzing data to support their research, then you could choose either program. Trying both for a little while might give you a good sense of which you prefer and why, or which might work better for your particular research situations.

4

University of Saskatchewan: Software Access

On-Campus Lab Access

If you are a University of Saskatchewan student working on-campus, all computers in the Arts & Science computer labs should have both SPSS and RStudio installed. See https://artsandscience.usask.ca/it/labs/ for a list of lab locations for the Saskatoon campus.

Remote / Off-Campus Access

Virtual Lab

If you are a University of Saskatchewan student working remotely (off-campus), you can access both SPSS and RStudio via the Virtual Lab at http://vlab.usask.ca/.

- Log in with your NSID.

- Click “All” to expand the menu, then click on “Common U of S”.

- Select “SPSS 26” (for SPSS) or “RStudio” (for R) to launch either program within the Virtual Lab.

More information on the Virtual Computer Lab (VLab) can be found here: https://wiki.usask.ca/x/lozDTg

IMPORTANT NOTE: In order to open any of the given Data Sets (.sav files) in the Virtual Lab, they first need to be added to your Cabinet drive. See the next sections for details on how to upload them.

Accessing your Cabinet Drive

The following links will guide you through gaining access your Cabinet drive so that you can then add files to it. Choose from the following options depending on if you are using Windows or Mac.

Ensure you follow the steps for connecting to Cabinet, specifically.

Try the steps without a VPN first. If you have issues, set up the VPN (Virtual Private Network) and try that way. The steps for setting up the VPN can be found here: https://wiki.usask.ca/x/0YnDTg

For Windows

- How do I map a network drive like Cabinet, Jade or Datastore on Windows?: https://wiki.usask.ca/x/_4nDTg

For Mac

- How do I map a network drive like Cabinet, Jade or Datastore on Windows?: https://wiki.usask.ca/x/_4nDTg

Adding Files to your Cabinet Drive

First, download all of the .sav files from the Data Sets page onto your computer.

Once you have access to your Cabinet drive, choose a designated folder within this drive where you will add the .sav files you want to work with; you may wish to create a new folder for this purpose, with a title like, e.g., PSY 233 files.

From there you can copy or move the .sav files from your computer into your designated Cabinet folder.

Then, they will be available for you to access them within the Virtual Lab.

USask ICT Help

Still stuck? Visit https://www.usask.ca/ict/help-support/it-support-services.php for more one-on-one assistance.

5

Data Sets

The dataset files listed here, which are used in the SPSS Lessons and RStudio Lessons of this book, were created by Osama Bataineh. They are released with a CC BY-NC-SA 4.0 license.

HyperactiveChildren.sav

Caregiver.sav

HeightLatency.sav

AgeSmoker.sav

HeadCircum.sav

pHLevel.sav

Methadone.sav

..

ActivityValue.sav

1.1 Overview

1.1.1 Textbook Layout, * and ** Symbols Explained

This textbook has been designed for use in the statistics classes for psychology I teach at the University of Saskatchewan. It is designed to replace the expensive, and inadequate, texts that have traditionally been used for these classes.

The courses covered by this text are:

1. Univariate Statistics I: Chapters 1 to 10 (Psy 233, undergraduate course)

2. Univariate Statistics II: Chapters 11 to 17 (Psy 234, undergraduate course)

3. Multivariate Statistics: Future project (Psy 807, graduate course)

Since these courses are applied statistics courses, students do not need to understand the derivations of the formulae and procedures. So these aspects, the “cookbook” approach, is what you need to learn to pass the applied statistics courses.

Sections Marked with ** : But, in the sections marked with a ** there are detailed derivations for those who don’t want to believe in magic. Most psychology students will want to skip the ** sections.

Sections Marked with * : Other sections are marked with a *; those sections contain applied statistics material that is not part of the course but is material that an experimental psychology student has a good chance of needing in experimental courses and research projects. (The graduate course Psy 805 is a review of Psy 233/234 with the additional * sections covered — so this text might also be used for Psy 805.)

Psychology students at the University of Saskatchewan are required to learn how to use the statistics program SPSS. So “Lessons” for learning SPSS are included included throughout the text, with RStudio Lessons as an alternative using a different program.

For Univariate Statistics I, the class material is organized in 3 blocks:

- Block 1 is an introduction to the basic tools of statistics and probability — Chapters 1 to 6.

- Block 2 gets you into the ideas of hypothesis testing — Chapter 9.

- Block 3 is material on one- and two-sample -tests — Chapters 9 and 10.

1.1.2 Intro to Univariate Statistics

So, to begin the course material proper, we may identify two “kinds” of statistics:

1. Descriptive Statistics: The presentation, organization and description of data. (Graphs, means, standard deviations, etc.) Block 1 material is primarily about descriptive statistics. Descriptive statistics lead to ideas about probability – we will cover probabilities as given by functions known as the binomial distribution and the normal distribution.

2. Inferential Statistics: The use of probability to infer things about a population from a sample through the use of hypothesis testing. Why do we need inferential statistics? Because it is usually impossible to measure (poll) an entire population.

The goal of Univariate Statistics I is to understand inferential statistics as embodied in the -tests. With blocks 2 and 3 we will build up the background for, and then learn 3 kinds of “-tests” to infer means in populations. To foreshadow, let’s take a look at a simple example. Say we are interested in people’s heights. Let’s look at three situations, corresponding to the three types of -tests we will learn.

i. One sample -test. The situation is as illustrated in Figure 1.1.

Figure 1.1: One sample -test

The -test will tell you when you may conclude that:

Here the population could be the height of 10 year old children in Saskatchewan. The quantity is the actual average height of 10 year old kids in Saskatchewan. You could, in principle, measure all the 10 year olds in Saskatchewan but, in practice you can’t. Even if you spent the time finding them all and measuring their heights with a tape measure, they will be growing while you measure them all. It’s generally impossible to measure a population in practice for some reason. Practically, we can only measure a small sample of children from the population. That sample will have a mean that we denote with . The -test is a hypothesis test in which we compare the sample mean to a hypothetical mean and conclude with a probabilistic inference about .

ii. Two sample -test. The situation is as illustrated in Figure 1.2.

The t-test will tell you when you can believe that

on the basis that . (The symbol means “approximately equal to”.)

Figure 1.2: Two sample -test

Here the two populations could be 10 year olds (population 1) and 11 year olds (population 2) in Saskatchewan. You might measure the two populations to get some idea about how much 10 year old kids in Saskatchewan grow in one year. The two sample -test will give you information on the difference of the average heights in the population, on the basis of the difference of the means of small samples that you take from each population, .

iii. Paired -test. The situation is as illustrated in Figure 1.3.

Say we want to know how fast a population grows in 1-year (e.g. pop = 10 year old kids). You can do the two-sample test with two separate populations but if you want to know how the environment affected the growth of the children (maybe you are concerned that they don’t get enough to eat) then the two-sample test is only an approximation. The genetic composition, the natural ability to grow, may be different in the two separate populations. To get at the effect of the environment, without the measurements being confounded by individual differences, we would take a sample of 10 year old kids from the population now and measure their heights. Then we wait a year and measure the height of the same sample of now 11 year old kids. Then we combine the two samples of data into one data sample of differences. The Paired -test will tell you if the average of differences (in heights) is zero or not.

Figure 1.3: Paired -test

1.2 Basic Definitions

Data : The numbers we collect.  (Note the word data is plural.  Datum is singular.)  Data may be grouped into sets, hence data set.

Variable : A mathematical term used to denote something that can take on a range of values. There are important two types of variables :

i. Independent variable (IV) : You set the value, a.k.a. explanatory variable.

ii. Dependent variable (DV) : Value set (generally caused) by the independent variable, a.k.a. outcome variable. See Figure 1.4.

Figure 1.4: In the equation of a line, y is the dependent variable, x is the independent variable.

Random Variable : A dependent variable with random noise added. Value given by a stochastic process. We will only refer to random variables when discussing the theoretical relationship between probability distributions. Random variables, which we will denote with capital letters like , are defined by their probability distribution. A stochastic process produces values that form a probability distribution if you allow the process that generates their values run for long enough.

Note : Data are frequently called “variables” in anticipation of how they will be used. The software program SPSS uses that convention.

1.2.1  Types of Data (important!)

Qualitative variable : described by a word, e.g. gender with “values” male or female.  Qualitative variables are converted to discrete quantitative variables before analysis (e.g. male = 1, female = 2). In SPSS, you need to assign discrete numbers to qualitative variables in the “Values” column in the “Variable View” screen.

Quantitative variable : two types :

i. Discrete variable : integer valued. In mathematical symbols (read “the variable belongs to [the set symbol means “belongs to”] the set of integers ”). e.g. -2, -1, 0, 1, 2, 3, etc.

ii. Continuous variable : real valued (essentially any number). In mathematical symbols (read “the variable belongs to the set of real numbers ”). Geometrically, is the number line.

Note : Continuous variables can be converted to discrete variables by grouping :

heights 5 ft = “short” (group value = 1)

heights " title="Rendered by QuickLaTeX.com" height="11" width="12" style="vertical-align: 0px;"> 5 ft = “tall” (group value = 2)

Groups are also known as classes. We will be spending time defining classes in Chapter 2. Identifying what type of variable you data is will be the best way for you to decide what statistical test you need after you have learned and understood a number of different tests.

1.2.2 Measurement Scales (avoid this!)

Some texts, and the SPSS helper program (although I have never tried it), attempt to classify data into “scales” that try to go somewhat beyond the integers and real numbers. I don’t think such classification is particularly useful and recommend that you avoid such classification. Nevertheless, it exists, so we will take a very quick look at such scales. (There is no agreement about their definitions from source to source.)

One textbook that I used for a Univariate Statistics class for many years[1] lists 4 types of scales :

i. nominal : discrete categories with no order (e.g. profession or gender) – qualitative.

ii. ordinal : discrete categories with order (e.g. grades, A, B, C ) – qualitative.

iii. interval : quantitative measure but no zero: ratios make no sense (e.g. temperature – makes no sense to say that one day was twice as hot as another day).

iv. ratio : has zero, and hence ratios have meaning – quantitative.

SPSS uses :

i. nominal.

ii. ordinal.

iii. scale : this scale is equivalent to the ordinal and ration scales listed above combined — as best as I can make out.

SPSS lets you specify a measurement scale under the “Measure” column in the “Variable View” screen. My recommendation is to leave it at “Unknown” or set it to “Scale”, otherwise it will try to restrict the statistical tests you can do when you don’t want it to. Measurement scales were invented to guide you to an appropriate statistical test but it doesn’t work that well. Instead, consider if your variable is continuous or discrete and then think about your situation.

1.2.3 Kinds of Sampling and Studies

This material properly belongs to a course on research methods and experimental design, but we will take a very quick look here. Ultimately your data need to be selected from the population at random. All mathematical statistical tests assume random sampling. The probability distributions that are used are defined by random sampling (the randomness — probability distribution relationship is pretty much a tautology). The real world is nit ideal, however, and you may be forced to deal with bias introduced by the following sampling schemes :

1. Random Sampling : Samples selected from the population at random.

2. Systematic Sampling : The population is ordered somehow (e.g. by house address or by phone number) and there is a rule for selecting samples (e.g. every 4th house or every 10th phone number).

3. Stratified Sampling : The population is, or can be, ordered into groups and sampling is done at random from the groups.

4. Cluster Sampling : Restrict sampling to a few groups of the population (a few strata).

And, depending on the control you have over your independent variable, studies may be classified as :

1. Observational Study : Just watch. You have no control over the independent variables.

2. Experimental Study : Control some variables to isolate other variables. The object is to manipulate the independent variable.

Astronomy is a passion of mine; observing stars and planets through a telescope is an example of an observational study. Experimental studies can be affected (knowingly or unknowingly) by confound variables.  These are causes (independent variables) that you are not interested in but which affect the outcome (dependent variables) and can lead to data bias that you need to account for. Such issues are beyond the scope of an introductory statistics course.

1. Bluman AG, Elementary Statistics: A Step-by-Step Approach, numerous editions, McGraw-Hill Ryerson, circa 2005. ↵

1.3 Summation Convention

For those of you who were ripped off in your high school education, a brief review of an important symbolic convention is given here. This convention will be used in the formulae that you will need to use.

The capital Greek Sigma, , means sum or add. For example, suppose that you have 5 data sample values, represented abstractly by and , or more abstractly (using set notation) by:

, (or )

If you want to add the 5 values you would write:

or

Sometimes people get lazy and leave off the limits on the summation sign and write

where it is hopefully clear that is the summation index . We can also leave off the summation index and write

just to remind us that we need to add up a bunch of numbers generically represented by . This last convention is useful for us because whenever we need to deal with a sum in a formula, we will get that sum from adding up numbers in a table that we have constructed.

Statistical inference is based on probability and probability is based on counting (at least the “frequentist” definition of probability – more about that in Chapter 4). So let’s start counting!

2.1 Frequency Tables

Most material in this text is introduced first at an abstract level, then generally a step-by-step recipe is given and finally example problems are solved. This general to specific approach to learning statistics is the opposite of how many introductory statistics tests for the social sciences teach. For our first topic of frequency tables, the abstract concept is counting so let’s dive into the recipe with the expectation that you won’t get the complete picture until an example or two is worked.

The construction of a frequency table proceeds in two steps :

Step 1 : Determine the classes. There are two possibilities here, either the classes are given to you (pre-defined) or you have to define the classes based on the number of groups you want. So either

i. Classes are given – nothing to do.

ii. Define classes based on the number of groups you want. There are a number of different ways to group data into classes. We will cover a method here, different from Bluman’s, that works for whole number data only. Here are the steps for that method :

(a) determine high data limit, and the low data limit, .

(b) compute the range

(c) compute the class width :

where is the number of groups (or classes) you want.

(d) Begin the frequency table’s first two columns :

Class

Class Boundaries

to

to

to

to

to

Note : If the classes are given,  you won’t have, or need, the second column.

In the class column above a specific way of labelling classes is given. (We will see how this works exactly in the upcoming example.) This is to make the class names useful for seeing that the classes are uniquely defined — there will be no data points on the boundaries of the classes. The numbers in the labels will be whole numbers, since we are assuming that the data are whole numbers[1]. In general we can label the classes any way we like.

Also we need to note that this procedure of defining classes using the formula given in step (2)(c) will only work for whole number data. In general the process of defining classes is a lot looser; there are few rules beyond thinking about what kind of information you hope to capture by defining the classes. Since I want to keep you focused on learning the basic ideas and not worry about stuff that is not really statistics all assignment and exam questions that ask for the construction of classes from quantitative data will be for whole number data only. The procedure given here does work in general but some data points may end up on class boundaries and will have to make up an arbitrary rule about which class the data point should go in.

Step 2 : Construct the frequency table and fill it in :

Class

Class Boundaries

Tally

Frequency

Cumulative Freq.

Relative Freq.

The last number in the cumulative frequency column, , should equal number of data points as a check since it is the sum of the frequencies. And the sum of the relative frequencies will be 1 — we will see that this is an essential feature of probabilities. The tally column is optional.

Example 2.1 : 25 army inductees were tested for blood type. The data are :

A

B

B

AB

O

O

O

B

AB

B

B

B

O

A

O

A

O

O

O

AB

AB

A

O

B

A

Construct a frequency table.

Solution :

Step 1 : Classes are given : A    B   O   AB

Step 2 : Construct frequency table :

Class

Tally

Frequency

Cumulative Freq.

Relative Freq.

A

|||||

5

5

5/25 = 0.20

B

||||| ||

7

12

7/25 = 0.28

O

||||| ||||

9

21

9/25 = 0.36

AB

||||

4

25

4/25 = 0.16

The tally is actually silly in this case because you count[2] all the instances of A for the class A, etc., and you’re done. The tally column will be more useful for the next example.

Example 2.2 : Given the high temperature data for each of 50 states for the month of July :

112

100

127

120

134

118

105

110

109

112

110

118

117

116

118

122

114

114

105

109

107

112

114

115

118

117

118

122

106

110

116

108

110

121

113

120

119

111

104

111

120

113

120

117

105

110

118

112

114

114

Construct a frequency table using 7 classes.

Solution :

Step 1 :

(a) High limit, H = 134
Low limit, L = 100

(b) Range: R = H – L = 134 – 100 =34

(c) Class width: W =

(d) (and continue to Step 2) :

Step 2 :

Class

Class Boundaries

Tally

Frequency

Cumulative Freq.

Relative Freq.

100 — 104

99.5 to 104.5

||

2

2

0.04

105 — 109

104.5 to 109.5

||||| |||

8

10

0.16

110 — 114

109.5 to 114.5

etc.

18

28

0.36

115 — 119

114.5 to 119.5

13

41

0.26

120 — 124

119.5 to 124.5

7

48

0.14

125 — 129

124.5 to 129.5

1

49

0.02

130 — 134

129.5 to 134.5

1

50

0.02

= 1

Note how we can now use the tally column to keep track of our counting. For example, for the class 100 — 104, we first count all the instances of 100 (there is 1), then 101 (none), 102 (none), 103 (none) and 104 (one). The sum of the frequencies is and the sum of the relative frequencies is 1. Imagine that this data set represented the whole population and not just a sample. Then if you picked a random state there would be a 0.16 probability that the temperature would be between 105 and 109 inclusive. On other words relative frequency = probability for a population. Hence the term frequentist definition of probability.

You can also compute cumulative relative frequency in a frequency table. When you use SPSS to make a frequency table you will run up against the limitations of using black box canned software. SPSS produces only one style of frequency table and it doesn’t match what we’ve been doing. In fact SPSS won’t compute relative frequency; instead it computes “percentage”. You need to convert percentage to relative frequency in your brain by dividing by 100.

1. Whole numbers are 0 and the positive integers. ↵

2. The frequency of A is the number of times A is in the dataset, etc. ← the take-home concept here.↵

2.2 Plotting Frequency Data

In general you may present your data, say in a report or paper, in tabular form or graphical form. Personally, I prefer graphical form — “a picture is worth a thousand words”. For frequency data, the frequency table is the tabular form. There are several ways of presenting the same data graphically, the primary way being the histogram:

1. Histogram – plot of frequency data using steps (mathematically: “step functions”).

2. Frequency polygon – plot of frequency data using straight lines (mathematically: “piece-wise linear functions”).

3. Cumulative frequency graph.

4. Pie charts, Pareto charts, Stem & Leaf plots – alternate ways of plotting frequency data

As a first step to plotting frequency data, you will need to construct a frequency table.

Example 2.3 : Continuing with the frequency table produced from the data given in Example 2.1 :

Class

Frequency

Cumulative Freq.

Relative Freq

A

5

5

0.20

B

7

12

0.28

O

9

21

0.36

AB

4

25

0.16

We will demonstrate most of the graph types using these data.

1. Histograms. First, the straight forward histogram is as shown in Figure 2.1. This is a plot of the data in the frequency column of the frequency table.

Figure 2.1 : Straight Forward histogram. A box or “step function” is used to show the frequency of each class. In this image, generated with SPSS, the classes are labelled with 1, 2, 3, and 4 which correspond to the classes A, B, O and AB. If we take these discrete quantitative class values literally, the class width is one. Keep that in mind when you look at Figure 2.2.

Next, still under the category histograms, is the relative frequency histogram. The relative frequency histogram for the blood type data is shown in Figure 2.2. It is a plot of the data in the relative frequency column of the frequency table.

Figure 2.2 : Relative frequency histogram for the blood type data.

Very Important Concept : Look at Figure 2.2 and define the width of each class to be 1. Then the area under the histogram “curve” is . So, if we image that our data sample of the 25 army inductees is a whole population, then the relative frequency histogram may be interpreted as giving the following probabilities for getting a particular blood type for someone selected randomly from the population:

The probability of having type A blood is 0.20 (or 20).

The probability of having type B blood is 0.28 (or 28).

The probability of having type 0 blood is 0.36 (or 36).

The probability of having type AB blood is 0.16 (or 16).

2. Frequency Polygons. Frequency polygons are just another form of histogram. We have been talking about “area under the curve” to represent probability. The curve of a frequency polygon is a little bit smoother than the curve of a traditional histogram. Frequency polygons can, of course be made for either straight frequency or relative frequency data. A frequency polygon for the relative frequency blood type data is shown in Figure 2.3.

Figure 2.3 : Relative frequency polygon for the blood type data. Plot a dot at the center of each class at the -value of the relative frequency then connect the dots as shown.

3. Cumulative Frequency Graph. Plotting the cumulative frequencies from the frequency table results in a cumulative frequency graph as shown in Figure 2.4. Cumulative relative frequencies can also be computed (add up relative frequencies as you move down the column) and plotted.

The cumulative frequency graph shows the “area under the curve” (of the traditional histogram) from the beginning of the first class up to the given point. Cumulative frequencies or cumulative relative frequencies with therefore show up later as areas under probability distribution curves up to a given point (it represents the probability of having a value equal to or less than the given value if that quantity is pulled at random from the population.)

Figure 2.4 : Cumulative frequency graph for the blood sample data. Plot a dot at the end of the relevant class at a -value equal to the cumulative frequency. Then connect the dots as shown.

4. Pie Chart. A pie chart is a round histogram. Everyone has seen a pie chart, it is intuitive. The angles in the pie chart are computed using:

Angle = Relative Frequency .

For the blood type data, the explicit angle calculations are :

Class

Angle

A

0.20 =

B

0.28 =

O

0.36 =

AB

0.16 =

Check Sum =

The pie chart for the blood type data is shown in Figure 2.5.

Figure 2.5 : Pie chart for the blood type data. It is a very good representation of the probability aspect of relative frequency. If you made the pie chart into a dart board and threw darts at it in a random fashion, then the probability of the dart landing in each class is equal to that class’s relative frequency.

5. Pareto Chart. The Pareto chart is just an ordered histogram with classes ordered from highest to lowest frequency. The classes need to be qualitative for this reordering to make sense of course. To construct a Pareto chart, writing an ordered frequency table down first will help :

Class

Frequency

A

5

B

7

O

9

AB

4

Figure 2.6: Pareto chart for the blood type data.

The Parato chart is plotted in Figure 2.6. The frequencies as ordered in a Parato chart can be given statistical meaning but that is a subject beyond the scope of this course. Here you just have to be aware that such a chart exists and know how it is made.

2.2.1 Stem and Leaf Plots

A stem and leaf plot is a fancy kind of histogram that lets you see all your data instead of just class frequency information.

The steps for making a stem and leaf plot are :

1. Order the data (this is a frequently used, tedious, step for many procedures as we’ll see).

2. Divide into classes of 10’s or 5’s (low decade and high decade).

3. Use “leading” and “trailing” digits of the data values to make the plot.

For step 3 you need to know what “leading” and “trailing” digits are. Let’s illustrate that with an example.

Example 2.4 : Given classes: 50-54, 55-59, 60-64, 65-69, 70-74, 75-79 or equivalently, divide the classes into 5’s and the data in order (i.e. with the tedious ordering step 1 already done) :

|50,51,51,52,53,53,|55,55,56,57,57,58,59,|62,63,|65,65,66,66,67,68,69,69|72,73,|75,75,77,78,79|

where the bars illustrate the division of the data into low and high decades, step 2. The first number of each data point is the leading digit (stem), the last, the trailing digit (leaf). So with this, step 3 leads to :

Stem

Leaf

5

0 1 1 2 3 3

5

5 5 6 7 7 8 9

6

2 3

6

5 5 6 6 7 8 9 9

7

2 3

7

5 5 7 8 9

Notice how, since the numbers are all nicely lined up, that the stem and leaf plot is a histogram on its side. So you can visualize frequency information and see the values of the individual data points as well. One could use that information to compute accurate means from stem and leaf plots whereas, as we’ll see, “class centers” need to be used with histogram (frequency table) data to estimate means with grouped data formulae.

2.3 SPSS Lesson 1: Getting Started with SPSS

The following lesson will take you through an introduction to IBM® SPSS® Statistics software (referred to hereafter as “SPSS”).

First, you need to open SPSS. Ways to do that are detailed in the Front Matter of this book, in the section “Statistical Software Used in this Book“. Also in the Front Matter you will find the collection of provided Data Sets; download the file “HyperactiveChildren.sav” and open it in SPSS.

You should see:

SPSS screenshot © International Business Machines Corporation.

This is the “Data View” window. It is one of the three windows you will see when you use SPSS. The other two windows are the “Variable View”  window and the “Output” window. You can get to the Variable View window either by clicking on the Variable View tab at the bottom of the window, or by double clicking one of the column headings (the “variable name”). But let’s talk about what’s on the Data View window before we look at the other two windows.

The Data View window is arranged in the form of a “data matrix”, which is an essential structure for multivariate statistics. This is the first trap that people who try to use SPSS fall into — they collect data, put the data into SPSS and then go looking for an appropriate statistical test using help or the built-in “statistics coach”. Multivariate statistics is advanced. We need to learn a whole lot of basics before we can competently use multivariate statistics. This textbook covers univariate statistics. We are only going to learn how to deal with one dependent variable at a time. So many of the first SPSS lessons will be about how to combine multiple variables into one variable for analysis.

Back to the Data View window and the data matrix. The rows represent individual subjects in the study. In Psychology, the subjects (“participants”) are generally people but they could also be rats or schools or cities or whatever. To fix ideas, suppose the subjects are people. One line for each person in the study. The columns represent variables. SPSS doesn’t care what kind of variables you define (e.g. independent or dependent) so you need to keep track of their meaning yourself. As we said, we only need one independent variable for univariate tests.

The variables need to be defined. This is done by either double clicking on the variable name at the top of a column or by clicking the “Variable View” button at the bottom. Either way, you’ll end up in the Variable View window that looks like :

SPSS screenshot © International Business Machines Corporation.

Each line in the Variable View window lists the attributes of the variables listed in the Data View window. You can usually leave most of the attributes as they come by default. The big exception is the Values attribute — it’s important and we’ll come back to that after a quick look at the other attributes.

The Name attribute gives the name of the variable as it appears at the top of the columns in the Data View window. Type should be Numeric if you want to use the variable in any kind of statistical calculation. Having this set to String will cause errors if you are trying to use the variable as a qualitative variable (selection is via a pull down menu that appears when you click on a cell). Qualitative variables need to be Numeric and they are handled with the Values attribute — as we’ll see shortly! The Width and Decimals attributes are just to format the appearance of the numbers in the Data View sheet; totally not critical. The Label is left over from early FORTRAN days. SPSS’s heart is written in FORTRAN and variable names in FORTRAN used to be limited to eight characters which frequently makes it awkward to have good name for the variable. With Label you can give the variable a good name. If the is a value for Label then that value will be used on table and graph outputs that SPSS makes. If Label is blank then SPSS will use Name on table and graph outputs. We will largely ignore missing value issues in this course so leave the Missing attribute at None. Columns and Align are again used to make the Data View presentation look a little better; totally not critical. Leave Measure at Unknown or Scale, otherwise SPSS will try to interpret your data for you. SPSS is not very good at that and will tend to give strange errors that will make no sense to you, so leave Measure at Unknown or Scale. Leave Role at Input; this is a relatively new feature of SPSS and I don’t know what it does, so don’t muck with it.

Finally — the Values attribute! Here is where you make the link between a qualitative variable and the discrete values it needs to work in a computer setting. Let’s take a look at the gender variable. Clicking in the cell brings up a thing with three dots :

SPSS screenshot © International Business Machines Corporation.

Clicking on the thing with three dots brings up a menu where you can define the connection between the qualitative description and your discrete number assignments :

SPSS screenshot © International Business Machines Corporation.

Here I have clicked on the 1.00 = “Male” line to show that the Value is 1 (arbitrary discrete quantitative) and the Label is Male (qualitative). To enter new values, type them in the Value and Label box and then click Add to add them to the list.

Let’s go back to the Variable View window to see how quantitative variables with discrete number assignments are handled. Look at the values in the sex variable column in the first image. The numbers 1 and 2 are shown which represent Male and Female. To see that representation explicitly, click on the 1-A icon at the top of the window. You will then see:

SPSS screenshot © International Business Machines Corporation.

There’s more. If you click on a cell in the gender variable, you will get a thing on the side of the cell and if you click on that thing, you will see:

SPSS screenshot © International Business Machines Corporation.

This pop-up allows you to change the value by clicking on the appropriate value. In one of your assignments you will get practice with entering qualitative data this way. In general, to enter data into SPSS from scratch, you can start by typing data into the Data View window and then fix up the attributes later in the Variable View window. For qualitative variables the best approach is to define the variable first in Variable View, getting the proper values into the Values attribute. Then you can go back to the Data View window and enter the qualitative data either by pulling down the menu when the mode of the 1-A icon is to show the labels or by remembering the number assignment and entering the numbers when the 1-A icon is set to show values.

Let’s move on to do some descriptive statistics and see what results will look like in the Output window. For this load in the “Caregiver.sav” file from the Data Sets:

SPSS screenshot © International Business Machines Corporation.

There are 50 subjects in this file and 10 variables. One of the things we’ll be learning, in later SPSS Lessons, is how to combine more than one variable into one variable. This is because we are studying univariate statistics which means we only want to deal with one dependent variable at a time. For now, lets pick on the variable CGDUR and see how we can generate descriptive statistics output. There are three ways to do this and they all begin in the Analyze Descriptive Frequencies menu which looks like this (on a PC; very similar on a Mac):

SPSS screenshot © International Business Machines Corporation.

Pick Frequencies… which brings up:

SPSS screenshot © International Business Machines Corporation.

Move the CGagecat variable over by clicking on the variable then the arrow button or just drag the variable over to get:

SPSS screenshot © International Business Machines Corporation.

Let’s take a look at the submenus and set them up before we hit OK. First the Statistics… submenu. In that menu check off Mean (), Median (MD), Mode, Skewness, Kurtosis, Std. deviation (), Variance (), Range (), Minimum () and, Maximum (). We we look at all of those descriptive statistics in Chapter 3.

SPSS screenshot © International Business Machines Corporation.

Hit Continue, look at the Charts… menu and check off pie charts, just for fun:

SPSS screenshot © International Business Machines Corporation.

Hit Continue. You can look at the Format… and Style… menus if you want, they are not particularly interesting. Make sure “Display frequency tables” is checked (this will be important when you do the assignments), then hit OK. The Output window will pop up and in that window you will see:

SPSS screenshot © International Business Machines Corporation.

The first table, Statistics, shows the descriptive statistics you asked for. Note, especially, for future reference (when we hit skewness in Chapter 3), the value of the skewness. It is . More to the point it is , or positive, meaning that the data set (CGagecatn) is right skewed or positively skewed. The second table, labeled “highestQualification” is the frequency table (note how the variable Name and not the Label was used because the Label attribute for the highestQualification variable was blank). The structure of the frequency table is slightly different from how we will learn to construct one by hand. There is nothing you can do to make SPSS produce a frequency table that matches exactly like what you might want. There are limitations to using canned statistics software.

Scrolling down the Output window you will see the pie chart:

SPSS screenshot © International Business Machines Corporation.

Lets look at the Descriptives… menu next:

SPSS screenshot © International Business Machines Corporation.

SPSS screenshot © International Business Machines Corporation.

Move the CGagecat variable over as before and make sure to check off the “Save standardized values as variables”. We’ll learn about standardized values (-values) in Chapter 3. Take note, this is the only way to get SPSS to compute -values :

Click the options menu and check off descriptive statistics to compute, as before (S.E. mean is Standard Error of the mean which we’ll get to eventually also, we’ll just leave it off for now):

SPSS screenshot © International Business Machines Corporation.

Hit Continue then OK and look at the results in the Output window. The output is straightforward:

SPSS screenshot © International Business Machines Corporation.

In Chapter 3 we will learn that the mean of a -transformed variable is zero and the standard deviation is one. That is confirmed here. If you left the “Save standardized values as variables” box checked when you ran this, you’ll get another variable added in the Data View window — the -transform of the -transform. It’s the same, the -transform of a -transform give back the same numbers. But note that the skewness () of the -transformed variable is the same as the skewness of the original variable. This means that -transforming a variable doesn’t change anything about the variable except its mean and standard deviation. This is important when it comes to using and interpreting any analyses based on the -transformed variable.

Finally, let’s look at the Explore… menu:

SPSS screenshot © International Business Machines Corporation.

Move CGagecat into the “Dependent List”. Don’t worry about “Factor List”, you should leave it blank (for future reference, “factor” is synonymous with “independent variable”):

SPSS screenshot © International Business Machines Corporation.

Take a look at the Statistics… menu. You can leave it as it is (we’ll be learning about Confidence Intervals later):

SPSS screenshot © International Business Machines Corporation.

Hit Continue and open the Plots… menu and check off the items as shown:

SPSS screenshot © International Business Machines Corporation.

We will talk about these different plots soon. For now, hit Continue, the OK and look at the output. First the tables:

SPSS screenshot © International Business Machines Corporation.

The first table is a “missing data report” that many SPSS procedures will output as a matter of course. You can ignore the missing data reports. Pay attention to the “Descriptive” table (it is something you could be asked about on exams!). You can ignore the “Tests of Normality” table. Next the plots. The first one is a histogram:

SPSS screenshot © International Business Machines Corporation.

After we cover skewedness in Chapter 3, come back to this picture and note how the histogram is right skewed.

Next is the stem and leaf plot. Remember that the way to a stem and leaf plot in SPSS is through the Explore menu:

SPSS screenshot © International Business Machines Corporation.

You can ignore the Q-Q plots but note that a boxplot is produced:

SPSS screenshot © International Business Machines Corporation.

This is not a very good boxplot. Again, we’ll be learning about boxplots later.

Looking at stuff here in SPSS before covering the concepts in class is a very real situation that people face in real life. They will go to a program like SPSS in the hopes that it is all they need for data analysis. But it will likely produce output that you don’t understand if you don’t have a basic education in statistics. If provided with output from SPSS (e.g., on an exam) you should able to explain what the output means. For example, if given one of the tables shown above you should be able to determine what the standard deviation of a data set is and be able to use that number in a further calculation. It is also a good idea to do some calculations by hand when you first use SPSS for a procedure. If you can produce the same numbers as SPSS then you are sure you know what it is doing.

2.4 RStudio Lesson 1: Getting Started with RStudio

Osama Bataineh

R is a free open source programing language widely used in statistical data analysis. It is very user friendly and it can be used in a variety of platforms such as MAC, Windows, LINUX, etc. R can be downloaded from https://www.r-project.org/. In most of the cases, R is being used with RStudio. RStudio is basically an additional interface which makes R more user friendly with a lot of additional features. A desktop version of RStudio can be downloaded (for free) at https://rstudio.com/products/rstudio/download/. More details are in the Front Matter of this book, in the section “Statistical Software Used in this Book”.

When you open RStudio, you will see this :

RStudio screenshot © the R Foundation.

Here you can see four windows. In the upper left window, you will write your R commands. The output is shown in the console in the lower left. There are also ways to write commands in the console and subsequently get output in the following lines. The upper right window shows description of the dataset that you are working on such as number of variables, number of observations etc. Finally, the lower right window is showing different packages of R at the moment. We will talk more about the packages little later. The lower right window also has other purposes. For example, if you write commands to produce graphs, it will show here. It also shows the results of help command when you seek help regarding anything in R.

Now let’s discuss a bit about how R functions. As the name suggests, R is a programming language. In other statistical packages such Stata, SPSS, EViews, Tableau etc., you have to choose the right options based on what results you want. However in R, rather than picking the right options and clicking on them, you have to write commands to get the desired output. To aid this process, there are different packages in R. These R packages were built for various purposes based on what analysis you want to do. Some of these packages are already built-in and installed. However, to use an existing installed package, you have to load it in every work secession before starting to use it. There are other packages of R which are not installed but available online. If you think these packages serve your purpose, then you have to install and load it before using it. Next time when you use these newly installed packages, you only have to load it in each work secession before starting to use it.

Now let’s get started working with our dataset. First, download the dataset “HyperactiveChildren.sav” from the textbook Data Sets. Then open RStudio and go to File Import Dataset From SPSS.

RStudio screenshot © the R Foundation.

After that, click the browse button in the pop-up window that will appear and select the dataset from the directory in which you have saved the data in your computer. Then click Import to insert the dataset in R. You can also do the same thing by manually executing the commands written in the Code Preview section by yourself.

RStudio screenshot © the R Foundation.

After inserting the data, you will see this in RStudio.

RStudio screenshot © the R Foundation.

Now we will write our commands in a new window in the upper left interface and save it in our desired folder in the computer to avoid rewriting the commands. This is known as R Script. To open a new R Script, click the arrow located just below and in between File and Edit and then select the first option R Script.

RStudio screenshot © the R Foundation.

After selecting the R script, a new R Script will open in a separate window named Untitled1. Save this script with a suitable name in your desired directory on your computer by clicking the old fashioned floppy disk icon. I have saved the script with the name Lesson 1\_RScript in my computer.

RStudio screenshot © the R Foundation.

Before starting to work, let’s get to know couple of things. Among these things, some are obligatory to know if anybody wants to work in R while others can make our life a lot easier and efficient. You can run the commands a lot quickly through the keyboard by clicking Ctrl and Enter. Other than the commands, you can also write notes in the R Script for your future references. To write anything other than the commands, just give a Hash (\#) sign in the beginning. Another thing you must know is that to work with any dataset in R after inserting it in the beginning, you have to attach it in the current work secession to work with it further. To attach the Smoking dataset for the current work secession, run the following command.

> attach(Hyperactive_Children)

Now let’s start working with our dataset. Similar to other software packages, a variable has to be numeric to use it for statistical analysis. Thus any qualitative string variable needs to be transformed into numeric quantitative variable if we want to conduct analysis with it. In our dataset, the variable sex  is a qualitative string variable with two categories male and female. Let’s create a new variable sex\_new which will take the value 1 for male and 2 for female. We can do it with the help of the function ifelse.

> Hyperactive_Children$Sex_new <- ifelse(Hyperactive_Children$Sex==1,1,ifelse(Hyperactive_Children$Sex
==2,2,NA))

A new variable Hyperactive_Children\_new is being created. If we view the data by running the following command, we will see that at the end an additional column is being created named Hyperactive_Children which has numeric values of 1 and 2 only.

> View(Hyperactive_Children)

RStudio screenshot © the R Foundation.

Here a thing to be noted is the use of dollar sign ($). The dollar sign basically calls the variable mentioned after it from the dataset mentioned prior to it or generates the new one mentioned after the sign to the dataset mentioned prior to it.

To get the frequency distribution of any variable in R similar in the way it’s shown in many statistics textbook, you have to write codes for each of the columns separately. For example- to get the frequencies, cumulative frequencies, relative frequencies etc. – you have to write separate commands unless you are a program wizard and able to create an R package which will produce such table. First to get the frequency of a variable for its different categories, we can use the function table.

> Age.freq <- table(Hyperactive_Children$Age)

> Age.freq

6 7 8 9 10 11 12
2 1 2 3  3  2  2

> cbind(Age.freq)
Age.freq
6      2
7       1
8      2
9      3
10    3
11     2
12    2

Here for a better presentation purpose, we have used another function cbind which basically shows the values of the variable and the number of observation it contains in a column. Then to get the cumulative frequencies, we can utilize the function cumsum.

> Age.cumfreq <- cumsum(table(Hyperactive_Children$Age))
> cbind(Age.freq, Age.cumfreq)

Age.freq     Age.cumfreq
6        2                   2
7        1                    3
8       2                    5
9       3                    8
10     3                    11
11      2                    13
12     2                    15

To produce pie chart, there is a specific function pie in R. However, since we need the pie chart of the frequency, we have to input table(Hyperactive_Children$Age) inside pie. The output will be shown in the lower right window.

> pie(table(Hyperactive_Children$Age))

RStudio screenshot © the R Foundation.

Now let’s have a look at the descriptive statistics of this variable Educ. To calculate descriptive statistics, we need to install a specific package named psych. After installing and loading this package, we have to use the function describe which is a part of this new package. If any warning signs come, please ignore it.

> library(psych)

> describe(Hyperactive_Children$Age)

vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1    15   9.2 1.93      9         9.23       1.48    6     12      6        -0.21 -1.18      0.5

Similarly to get the confidence interval, you need another new package named Rmisc. While installing this package you have to keep the dependencies to true. After installing and loading this package, we have to use one of it’s function CI to get the confidence interval. Also remember that in addition to inserting Hyperactive_Children$Age in the domain of the function, you have to also specify the level of confidence interval. Ignore the warning signs here too.

> install.packages(‘Rmisc’, dependencies = TRUE)
> library(Rmisc)

> CI(Hyperactive_Children$Age, ci = 0.95)
upper        mean        lower
10.271372 9.200000 8.128628

Finally, to produce the histogram, stem and leaf display and boxplot, there are specific functions in R with their names. These functions are hist, stem and boxplot respectively.

> hist(Hyperactive_Children$Age)

RStudio screenshot © the R Foundation.

> stem(Hyperactive_Children$Age)

The decimal point is at the |

6 | 000
8 | 00000
10 | 00000
12 | 00

> boxplot(Hyperactive_Children$Age)

RStudio screenshot © the R Foundation.

3.1 Central Tendency: Mean, Median, Mode

Mean, median and mode are measures of the central tendency of the data. That is, as data are collected while sampling from a population, there values will tend to cluster around these measures. Let’s define them one by one.

3.1.1 Mean

The mean is the average of the data. We distinguish between a sample mean and a population mean with the following symbols :

The formula for a sample mean is :

where is the number of data points in the sample, the sample size. For a population, the formula is

where is the size of the population.

Example 3.1 : Find the mean of the following data set :

84

12

27

15

40

18

33

33

14

4

To illustrate how the indexed symbols that represent the data in the formula work, they have been written below the data values. To get in the habit, let’s organize our data as a table. We will need to do that for more complicated formulae and also that’s how you need to enter data into SPSS, as a column of numbers :

label

84

12

27

15

40

18

33

33

14

4

Total = 280

Since we have .

☐

Mean for grouped data : If you have a frequency table for a dataset but not the actual data, you can still compute the (approximate) mean of the dataset. This somewhat artificial situation for datasets will be a fundamental situation when we consider probability distributions. The formula for the mean of grouped data is

(3.1)

where is the frequency of group , is the class center of group and is the number of data points in the original dataset. Recall that so we can write this formula as

which is a form that more closely matches with a generic weighted mean formula; the formula for the mean of grouped data is a special case of a more general weighted mean that we will look at next. The class center is literally the center of the class — the next example shows how to find it.

Example 3.2 : Find the mean of the dataset summarized in the following frequency table.

Class

Class Boundaries

Frequency,

Midpoint, x_{m_{i}}

1

5.5 – 10.5

1

8

8

2

10.5 – 15.5

2

13

26

3

15.5 – 20.5

3

18

54

4

20.5 – 25.5

5

23

115

5

25.5 – 30.5

4

28

112

6

30.5 – 35.5

3

33

99

7

35.5 – 40.5

2

38

76

sums

n= = 20

= 490

Solution : The first step is to write down the formula to cue you to what quantities you need to compute :

We need the sum in the numerator and the value for in the denominator. Get the numbers from the sums of the columns as shown in the frequency table :

☐

Note that the grouped data formula gives an approximation of the mean of the original dataset in the following way. The exact mean is given by

So the approximation is that

which would be exact only if all in group were equal to the class center .

Generic Weighted Mean : The general formula for weighted mean is

(3.2)

where is the weight for data point . Weights can be assigned to data points for a variety of reasons. In the formula for grouped data, as a weighted mean, treats the class centers as data points and the group frequencies as weights. The next example weights grades.

Example 3.3 : In this example grades are weighted by credit units. The weights are as given in the table :

Course

Credit Units,

Grade,

English

3

80

240

Psych

3

75

225

Biology

4

60

240

PhysEd

2

82

164

= 12

= 297

= 869

The formula for weighted mean is

so we need two sums. The double bars in the table above separate given data from columns added for calculation purposes. We will be using this convention with the double bars in other procedures to come. Using the sums for the table we get

Note, that the unweighted mean for these data is

which is, of course, different from the weighted sum.

☐

3.1.2 Median

The symbol we use for median is MD and it is the midpoint of the data set with the data put in order. We illustrate this with a couple of examples :

- If there are an odd number of data points, MD is the middle number.

Given data in order:  180  186  191  201  209  219  220

- If there are an even number of data points, MD is the average of the two middle points :

Given data in order:  656  684  702  764  856  1132  1133  1303

In these examples, the tedious work of putting the data in order from smallest to largest was done for us. With a random bunch of numbers, the work of finding the median is mostly putting the data in order.

3.1.3 Mode

In a given dataset the mode is the data value that occurs the most. Note that :

- it may be there is no mode.

- there may be more than one mode.

Example 3.4 :  In the dataset

, 9, 9, 14, , , 10, 7, 6, 9, 7, , 10, 14, 11, , 14, 11

8 occurs 5 times, more than any other number. So the mode is 8.

☐

Example 3.5 :  The dataset

110, 731, 1031, 84, 20, 118, 1162, 1977, 103, 72

has no mode. Do not say that the mode is zero. Zero is not in the dataset.

☐

Example 3.6 : The dataset

15, 20, 22, 26, 26

has two modes: 18 and 24. This data set is bimodal.

The concept of mode really makes more sense for frequency table/histogram data.

☐

Example 3.7: The mode of the following frequency table data is the class with the highest frequency.

Class

Class Boundaries

Freq

1

5.5 – 10.5

1

2

10.5 – 15.5

2

3

15.5 – 20.5

3

4

20.5 – 25.5

5 (Modal Class)

5

25.5 – 30.5

4

6

30.5 – 35.5

3

7

35.5 – 40.5

2

☐

3.1.4 Midrange

The midrange, which we’ll denote symbolically by MR, is defined simply by

where and are the high and low data values.

Example 3.8 : Given the following data : 2,  3,  6,  8,  4,  1. We have

☐

3.1.5 Mean, Median and Mode in Histograms: Skewness

If the shape of the histogram of a dataset is not too bizarre[1] (e.g. unimodal) then we may determine the skewness of the dataset’s histogram (which would be a probability distribution of the data represented a population and not a sample) by comparing the mean or median to the mode. (Always compare something to the mode, no reliable information comes from comparing the median and mean.) If you have SPSS output with the skewness number calculated (we will see the formula for skewness later) then a left skewed distribution will have a negative skewness value, a symmetric distribution will have a skewness of 0 and, a right skewed distribution will have a positive skewness value.

Figure 3.1: A right skewed histogram (or distribution) generally has the mean and median to the right, or positive side of the mode. The tail of the histogram stretches to the right or positive side.

Symmetric distribution

Figure 3.2: A symmetric distribution (histogram) has the mean, median and mode all in the same place. Its shape is symmetric.

Negatively skewed or left skewed histograms

Figure 3.3: A left skewed histogram (or distribution) generally has the mean and median to the left, or negative side of the mode. The tail of the histogram stretches to the left or negative side.

3.1.6 Mean, Median and Mode in Distributions: Geometric Aspects

To understand the geometrical aspects of histograms we make the abstraction of letting the class widths shrink to zero so that the histogram curve becomes smooth. So let’s consider the mode, median and mean in turn.

Mode

The mode is the value where the frequency is maximum, see Figure 3.4. More accurately the mode is a “local maximum” of the histogram[2] (so if there are multiple modes, they don’t all have to have the same maximum value).

Figure 3.4: The mode is the maximum of the histogram (distribution).

Median

The area under the curve is equal on either side of the median. In Figure 3.5 each area is the same. For relative frequencies (and so for probabilities) the total area under the curve is one. So the area on each side of the median is half. The median represents the 50/50 probability point; it is equally probable that is below the median as above it.

Figure 3.5: The median divides the area under the histogram into two equal areas .

Mean

The mean is the balance point of the histogram/distribution as shown in Figure 3.6.

Figure 3.6: The mean is the balance point of the histogram. It is where the “first moments” of the area of the histogram balance. Here the moments are and balance. .

**A proof that the mean is the center of gravity of a histogram:

In physics, a moment is weight moment arm :

where is moment, is weight and is the moment arm (a distance).

Say we have two kids, kid1 and kid2 on a teeter-totter (Figure 3.7).

Figure 3.7

Kid1 with weight is heavy, kid2 with weight is light.

To balance the teeter-totter we must have

The moment arm, , of the heavier kid must be smaller than the moment arm, , of the lighter kid if they are to balance.

So now let’s define the center of gravity. If you have a bunch of weights with corresponding moment arms then the center of gravity (c of g) is the moment arm (distance) that satisfies :

where is the total weight.

With histograms, instead of weight we have area . You can think of area as having a weight. (Think of cutting out a piece of the blackboard with a jigsaw after you draw a histogram on it.) So for a histogram (see Figure 3.8):

Figure 3.8

(We assume, for simplicity but “without loss of generality”, that are integers and also the classes. This is the case for discrete probability distributions as we’ll see.) So, for the c of g,

translates to

where we have used because the class widths are one, so

Because our “weight” is area, is technically called the “1st moment of area”. (Variance, covered next, is the “2nd moment of area about the mean”.)

◻

1. For the purposes of deciding the skewness of a dataset in assignments and exams, you can assume that the histogram shape is not too bizarre. ↵

2. **In calculus terms, local maximums and minimums (and inflexion points) are where the derivative equals zero, . ↵

3.2 Dispersion: Variance and Standard Deviation

Variance, and its square root standard deviation, measure how “wide” or “spread out” a data distribution is. We begin by using the formula definitions; they are slightly different for populations and samples.

1. Population Formulae :

Variance :

(3.3)

where is the size of the population, is the mean of the population and is an individual value from the population.

Standard Deviation :

The standard deviation, , is a population parameter, we will learn about how to make inferences about population parameters using statistics from samples.

2. Sample Formulae :

Variance :

(3.4)

where = sample size (number of data points), = degrees of freedom for the given sample, and is a data value.

Standard Deviation :

Equations (3.3) and (3.4) are the definitions of variance as the second moment about the mean; you need to determine the means ( or ) before you can compute variance with those formulae. They are algebraically equivalent to a “short cut” formula that allow you to compute the variance directly from sums and sums of squares of the data without computing the mean first. For the sample standard deviation (the useful one) the short cut formula is

(3.5)

At this point you should figure out how to compute , and on your calculator for a given set of data.

Fact (not proved here) : The sample standard deviation is the “optimal unbiased estimate” of the population standard deviation . is a statistic”, the best statistic it turns out, that is used to estimate the population parameter . It is the in the denominator that makes the optimal unbiased estimator of . We won’t prove that here but we will try and build up a little intuition about what that should be so — why dividing by should be better than dividing by . ( is known as the degrees of freedom of the estimator ). First notice that you can’t guess or estimate a value for (i.e. compute ) with only one data point. There is no spread of values in a data set of one point! This is part of the reason why the degrees of freedom is and not . A more direct reason is that you need to remove one piece of information (the mean) from your sample before you can guess (compute ).

Coefficient of Variation

The coefficient of variation, CVar, is a “normalized” measure of data spread. It will not be useful for any inferential statistics that we will be doing. It is a pure descriptive statistic. As such it can be useful as a dependent variable but we treat it here as a descriptive statistic that combines the mean and standard deviation. The definition is :

Example 3.9 : In this example we take the data given in the following table as representing the whole population of size . So we use the formula of Equation (3.3) which requires us to sum .

10

60

50

30

40

20

Using the sum in the first column we compute the mean :

Then with that mean we compute the quantities in the second (calculation) column above and sum them. And then we may compute the variance :

and standard deviation

Finally, because we can, we compute the coefficient of variation:

◻

Example 3.10 : In this example, we have a sample. This is the usual circumstance under which we would compute variance and sample standard deviation. We can use either Equation (3.4) or (3.5). Using Equation (3.4) follows the sample procedure that is given in Example 3.9 and we’ll leave that as an exercise. Below we’ll apply the short-cut formula and see how may be computed without knowing . The dataset is given in the table below in the column to the left of the double line. The columns to the right of the double line are, as usual, our calculation columns. The size of the sample is .

11.2

= 125.44

11.9

= 141.161

12.0

exercise

= 144

12.8

= 163.84

13.4

= 179.56

14.3

= 204.49

To find compute

So

Note that is never negative! If it were then you couldn’t take the square root to find . Also not that we have not yet determined the mean. We can do that now:

And with the mean we can then compute

◻

Grouped Sample Formula for Variance

As with the mean, we can compute an approximation of the data variance from frequency table, histogram, data. And again this computation is precise for probability distributions with class widths of one. The grouped sample formula for variance is

(3.6)

where is the number of groups or classes, is the class center of group , is the frequency of group and

is the sample size. Equation (3.6) the short-cut version of the formula. We can also write

or if we are dealing with a population, and the class width is one so that the class center ,

which will be useful when we talk about probability distributions. In fact, let’s look ahead a bit and make the frequentist definition for the probability for as (which is the relative frequency of class ) so that

(3.7)

If we make the same substitution in the grouped mean formula, Equation (3.1) with population items and in place of the sample items and , then it becomes

(3.8)

More on probability distributions later, for now let’s see how we use Equation (3.6) for frequency table data.

Example 3.11 : Given the frequency table data to the left of the double dividing line in the table below, compute the variance and standard deviation of the data using the grouped data formula.

Class

Class Boundaries

Freq,

Class Centre

1

5.5 – 10.5

1

8

2

10.5 – 15.5

2

13

3

15.5 – 20.5

3

18

4

20.5 – 25.5

5

23

5

25.5 – 30.5

4

28

6

30.5 – 35.5

3

33

7

35.5 – 40.5

2

38

The formula

tells us that we need the sums of and after we compute the class centres and their squares — these calculations we do in the columns added to the right of the double bar in the table above. With the sums we compute

So

The mean, from one of the sums already finished is

and the coefficient of variation is

◻

Now is a good time to figure out how to compute and (and ) on your calculators.

3.3 z-score / z-transformation

The -score is the result of transformation of data that converts a dataset of values, , that has a mean of and standard deviation to a set of values that has a mean of and a standard deviation of . It will be very useful when we need to compute probabilities associated with normal distributions. The -transformation is defined by

Example 3.12 : Find the -scores of the data given in the left column of the table below.

Data

-score,

18

324

(18-9.9)/6.2 = 1.3

15

225

(15-9.9)/6.2 = 0.8

12

144

(12-9.9)/6.2 = 0.3

6

36

(6-9.9)/6.2 = -0.6

8

64

(8-9.9)/6.2 = -0.3

2

4

(2-9.9)/6.2 = -1.3

3

9

(3-9.9)/6.2 = -1.1

5

25

(5-9.5)/6.2 = -0.8

20

400

(20-9.5)/6.2 = -1.7

10

100

(10-9.5)/6.2 = 0.1

The dataset size is . You need to compute the -score for each data value separately. To do the calculation, both and are needed. So in addition to the sum of the data, , we also need the sum of the values. The work of getting those sums is shown in the table above. With the and sums we get

and

and

Using these values for and in the third column of the table above, compute the -scores as shown. If we had computed the -scores more accurately, they would add up to zero, (the mean of the -scores is zero.)

◻

3.4 SPSS Lesson 2: Combining variables and recoding

Frequently data collection results in a collection of many variables. This happens, for example, with tests or surveys where people answer questions on a 5 or 7 point Lickert scale where questions range from, say, “strongly agree” to “somewhat agree” to to “strongly disagree”. A bunch of those questions may refer to, say, happiness and adding up the scores, perhaps averaging them, will lead to a single variable, one dependent variable, that becomes our measurement of happiness. This gives us not only a univariate variable that we can subject to a statistical test but likely gives us a stronger and more reliable measurement of happiness. A problem with combining variables in this way arises if the response “1” for “strongly agree” means happiness for one question (e.g. “I wake up happy”) and sadness in another question (e.g. “I go to bed sad”). In such a situation some of the variables will need to be reverse-scaled or recoded before they can be added. Let’s see how to combine and recode variables in SPSS.

Open the file “Caregiver.sav” from the textbook Data Sets. This dataset is about the different attributes of diamonds such as its color, price, carat, cutting quality etc. Here one of the variables is cut\_new which basically represents the cutting quality of diamond and takes values from 1 to 5 depending on the cutting quality with 5 being the best quality. Now let’s assume that we need to reverse scale this variable to use it in other calculations in a meaningful manner. To recode cut\_new first open the Transform Recode in Same Variables… menu :

SPSS screenshot © International Business Machines Corporation.

You can choose the Recode into Different Variables… if you want to, instead. That choice will lead to the creation of a new variable that you would use in place of cut\_new for your analysis. With our choice of Recode in Same Variables… we will overwrite the old values of cut\_new with new ones. (This is a danger if you make a mistake.) Our job is now to map 1 to 5, 2 to 4, 3 to 3, 4 to 2 and 5 to 1, recoding the variable. First move the cut\_new variable over in the pop up menu :

SPSS screenshot © International Business Machines Corporation.

then hit the Old and New Values.. button that will bring up a new pop up menu. Next enter 1 under Old Value and 5 in New Value :

SPSS screenshot © International Business Machines Corporation.

then hit Add :

SPSS screenshot © International Business Machines Corporation.

Continue this way to complete the recoding list :

SPSS screenshot © International Business Machines Corporation.

Hit Continue, then OK. The variable cut\_new will now have the new values in the Data View window.

Now suppose we want to add multiple variables to create a new variable. Let’s open the dataset Caregiver from the course website. This dataset is regarding the test scores of students from diverse background in UK. Here we will add the test scores of read, write, math and science to create a new variable totalscore. Pick the Transform Compute Variable… menu :

SPSS screenshot © International Business Machines Corporation.

This will bring up a menu which is essentially the calculator feature of SPSS :

SPSS screenshot © International Business Machines Corporation.

Fill in the menu as shown above. You can move variable names into the Numeric Expression box by double clicking on the variable name, by clicking on the variable name and the arrow or by simply typing it. There are fancier ways to get a sum of variables experesion in Numeric Expression, but we will keep it simple for now. The target variable name is totalscore which, after you hit OK, shows up as a new variable, ready for statistical analysis, in the last column in the Data View window :

SPSS screenshot © International Business Machines Corporation.

Let’s do a couple of (descriptive) analysis with this new variable. Let’s take Caregiver as our dataset. Suppose we want to find the median of the totalscore values. To do this task by hand, we need to put the data in order from smallest to largest. This is tedious but SPSS can do it with a couple of mouse clicks (yes, yes SPSS can compute the median directly but whatever). There are a couple of approaches in SPSS to ordering, or ranking, data. One is to compute the rank, that is, give rank 1 to the lowest value, 2 to the next lowest up to for the highest value. Pick Transform Rank Cases and move totalscore into the Variable(s) box :

SPSS screenshot © International Business Machines Corporation.

This is a new menu for us, so let’s take a look at the submenus. First, the Rank Types menu :

SPSS screenshot © International Business Machines Corporation.

Pretty fancy. Much to advanced for our use, so let’s leave that one be, hit Continue. Next look at Ties…

SPSS screenshot © International Business Machines Corporation.

We will assign the average (mean) rank to ties in out classes. To understand the ties options, think of two people in a race who cross the finish line at exactly the same time, a tie. With the mean rank, they both come in 1.5 place. With lowest, they both come in 1st place, with highest, they both come in 2nd place. Hit Continue, the OK and a new variable Rtotalscore with be formed in the Data View menu :

SPSS screenshot © International Business Machines Corporation.

Here the variable RCG ranks the total score of the students. But it’s very difficult from this data view to identify which students’ rank the highest or lowest, let alone who falls in the middle to find the median. This is not quite what we are after to easily get the median. Ranking will become useful on Psy 234 (in Chapter 16), but it’s not that useful for us now. What we need, is to shuffle the numbers around from lowest to highest (of course we can do that directly). To shuffle pick Data Sort Cases :

SPSS screenshot © International Business Machines Corporation.

which brings up, after moving over the RCG total score variable :

SPSS screenshot © International Business Machines Corporation.

Keep the ascending button selected (sort from lowest to highest), then hit OK to sort the file :

SPSS screenshot © International Business Machines Corporation.

Everything is sorted now. (Note how useful the id variable is now. If that wasn’t there, we’d lose track of who’s data was what.) Now if we scroll down, we will find that the middle two total test scores are both 210. Thus the median of total score is 210.

As a final analysis of the Caregiver data, suppose we wanted some descriptive statistics for the male students separate from the female students. To do this we use the “split file” feature of SPSS. Select Data Split File to get

SPSS screenshot © International Business Machines Corporation.

where the gender variable has been moved into the “Groups Based on” box — you will need to click on the “Organize output by groups” button also. We’ll also leave the “Sort the file by grouping variables” (gender in this case), this will shuffle the file yet again, putting all the males and females together. So, when you hit OK the result is

SPSS screenshot © International Business Machines Corporation.

Now the file is sorted into Male and Female (the 1-A button at the top has been pressed). Also note that “Split by gender” appears on the lower right corner of the Data View window. Now let’s do a simple descriptive statistics analysis of the total score variable. The output looks like :

SPSS screenshot © International Business Machines Corporation.

To unsplit the file, go back to Data Split File and hit the “Analyze all cases, do not create groups” button. This will remove the “Split” message from the lower right corner and when the descriptive statistics is run again, you will get :

SPSS screenshot © International Business Machines Corporation.

From here, with the file unsplit, we can use gender as a factor to get separate descriptive statistics for males and female. Select Analyze Explore and use gender as the factor, which results in :

SPSS screenshot © International Business Machines Corporation.

From here, with the file unsplit, we can use gender as a factor to get separate descriptive statistics for males and female. Select Analyze Explore and use gender as the factor :

SPSS screenshot © International Business Machines Corporation.

The result is :

SPSS screenshot © International Business Machines Corporation.

3.5 RStudio Lesson 2: Combining variables and recoding

[Coming soon]

4.1 Probability

The basic definition of probability is a ratio of things you can count (a ratio of their frequencies) :

(4.1)

where

is the probability that event happens,
is the number of ways can happen and
is the total number of outcomes (all possibilities).

Example 4.1 : What is the probability of drawing a queen from a deck of cards :

▢

To use mathematically we set

Where, probability-wise:

0 means definitely will not occur, and
1 means definitely will occur.

This is a method we can use instead of using percent. To compute probabilities, we first need to know how to count.

Fundamental Counting Rule

Say you have n events in order, and for event there are ways for it to happen. Then the number of ways for the events to play out is :

(The giant pi symbolizes a multiplication convention in the same way that a giant sigma symbolizes a summation convention as described in Section 1.3.)

Example 4.2 How many combinations are there on a lock with 3 numbers?

Lay out the events as : , , and . Note that each number can be anything from 0 to 9 giving 10 possibilities () for each event. So the number of possible lock combinations is

Note that you could have guessed this because the combination range from 000 to 999 — counting in base 10.

▢

Example 4.3 Suppose that a hardware store can produce paints with the following qualities :

Colour : red, blue, white, black, green, brown, yellow (7 colours)

Type : latex, oil (2 types)

Texture : flat, semigloss, high-gloss (3 textures)

Use : indoor, outdoor (2 uses)

How many ways are there to combine these qualities to produce a can of paint?

Answer : From the above list and the number of possible paint kinds is:

▢

Applications of the Fundamental Counting Rule

We are interested in applying the fundamental counting rule to two special, important cases :

1. Permutations.

2. Combinations.

Let’s define each one.

1. Permutations.

The number of ways, or permutations, of selecting objects from a collection or objects, while keeping track of the order of selection is [1]

This formula follows from the fundamental counting rule. With objects there are ways to select the first object. After selecting the first object there are ways to choose the second object so , etc. up to :

Example 4.4 : How many ways are there to choose 5 numbered balls from a bucket of 25 to make a lottery number?

Answer : possibilities.

▢

2. Combinations.

The number of ways of selecting objects from a collection of objects without caring about the order is :

That last symbol is colloquially called “ choose ”. The second last expression demonstrates the application of the fundamental counting principal, it says

where is just the number of ways of arranging objects while caring about the order, .

As a practical matter, never try to compute It will usually be unimaginably big. Use the formula that directly shows the fundamental counting rule as shown in the following example.

Example 4.5 : How many ways are there to select 10 balls from a bucket of 100?

Answer :

▢

The symbol is also known as the binomial coefficient because it shows up in algebra when you expand expressions of the form . For example[2]

The binomial coefficients can be quickly computed using Pascal’s triangle :

Referring to Pascal’s triangle we can quickly write

for example.

1. Recall that the definition of factorial follows etc. ↵

2. You don't need this algebra for this statistics course. It's just interesting. ↵

4.2 Binomial Distribution

Given a success/failure situation (or yes/no, black/white, any 2 outcome, dichotomous situation) and a probability of success (and so a probability of failure ), what is the probability of achieving successes in trials? In symbols[1] what is ( successes | trials)? Or with simpler notation, what is ? The answer is :

(4.2)

**Proof of the formula

Use the boxes we used in defining the fundamental counting rule to represent each trial.

Consider .

The probability that a success occurs is the definition of . So

Consider . What is ? This is all failures :

The probability of each failure is so the probability of getting FF is . So

(Note that by definition. There is exactly one way to draw no things from a collection of 2.)

What is ? Each probability of ( for the first one, for the second one). So

For we have

We can continue this way for but this is clearly tedious. The way of “mathematical induction” is the formal way to proceed but let’s try a more intuitive approach.

For successes in trials, consider our boxes, then any given sequence with successes will have failures and so that given sequence will have a probability of . But how many specific sequences with successes are there? Think of it this way. Of the boxes, how many ways are there to write S’s in the boxes? There are possibilities ( boxes are available) to write the first S, ways after that to write the second S, etc. But we don’t care which order we wrote the S’s into the boxes so divide by . In other words there are specific sequences with successes. Putting it all together :

▢

Example 4.6 : In bucket of 100 toys with 20 dinosaurs and 80 bugs, consider drawing a dinosaur a success. So and . Let us make an approximation and assume that does not change with each draw[2]

Say we want to know (3 successes 10 trials). In other words, what is the probability that if I take 10 toys out of the bucket that exactly 3 of them are dinosaurs? Using Equation (4.2) we find

The actual process of doing this calculation is somewhat tedious and therefore error prone. So in a test, for example, you will want to use the Binomial Distribution Table included in this text in the Appendix. In the Binomial Distribution Table, you simply find the appropriate and then in the column on the left and then look under the appropriate column to find for the given .

▢

The complete binomial distribution specifies the probabilities of all successes from 0 to , and can be plotted as a histogram. Note that there is a binomial distribution for each and . Let’s plot the binomial distribution for getting successes (dinosaurs) in forming a sample of toys with . The Binomial Distribution Table contains the relative frequency table for the histogram that represents the binomial distribution shown in Figure 4.1.

Figure 4.1 : The binomial distribution for the example of forming samples of toys with representing the number of dinosaurs in the sample and being the probability of selecting a dinosaur in forming the sample. Note that the probability of = 8, 9 or 10 is not zero, just less than 0.001.

The binomial distribution is an example of a discrete probability distribution. It is a histogram of relative frequencies obtained by counting possibilities in sample space.[3]

The mean and variance of any discrete distribution are given by

These two formulae come from the grouped data expressions and , by substituting . If we substitute Equation 4.2 for in these general equations we get

which are the mean and variance for a binomial distribution with parameters and . The mean is the expected value.

Example 4.7 : For the bucket of toys example:

So given any random sample of 10 toys we expect that 2 of them will be red.

▢

4.2.1 Practical Binomial Distribution Examples

The examples given here illustrate the sampling theory for forming samples from a dichotomous (with success/fail items; items of interest and no interest) population. In this situation we know exactly what is in the population and ask questions about what kind of samples can be formed and what is their probability. The sampling theory is completely described by the binomial distribution. Later, we will have a sampling theory based on the Central Limit Theorem which will lead us to the normal distribution.

In practically solving these kinds of problems keep in mind that you need to identify: , and .

Example 4.8 : It was reported that 5 of Americans are afraid of being alone in a house at night. In a random sample of 20 Americans, what are the probabilities that the sample contains

1. exactly 5 afraid people?

2. at most 3 afraid people?

3. at least 3 afraid people?

Solution : First identify: , and the as specific to each question :

1. For this case, , so from the Binomial Distribution Table get .

2. For this case , 1, 2 and 3 and we have to add up the probabilities
From the Binomial Distribution Table:

So

3. , 4, 5, 6, 7, , 20
From the Binomial Distribution Table:

Since the probabilities of high are too small to appear in the Binomial Distribution Table (and there would be many terms to consider if they weren’t) we should use the following trick :

▢

1. Here the | is read as "given". ↵

2. **By assuming that does not change, we will be lead to the binomial distribution. If we more accurately assume that changes with each draw we will be lead to the hypergeometic distribution. For fun, let's consider the case where changes with each draw. It's just another application of the fundamental counting rule. To begin, there are ways of drawing 10 toys from the bucket without caring if it is a dinosaur or a bug. This is the size of the sample space; it is how many ways there are to make a sample of size 10 from the bucket of 100 choices; it is in Equation (4.1). There are samples of 10 in the bucket. If we want 3 dinosaurs in our sample, as in the example in text then of the 20 dinosaurs in the bucket, there are ways to get 3 dinosaurs and ways to get 7 bugs from the 80 in the bucket. So there are ways to draw 3 dinosaurs and 7 bugs from the bucket. This number is in Equation (4.1). And so

Note how close this is to the answer from the binomial distribution of 0.201. ↵

3. Sample space is the set of all possible samples. ↵

4.3 SPSS Lesson 3: Combining variables - advanced

In SPSS Lesson 2 we saw how we can take variables defined on a Lickert scale and add them together, reverse scaling if necessary, to produce a single, better, variable for analysis. This works because the Lickert scale variables all have the same “units” (number of answer choices). You can combine any variables that have the same units, like feet or years or whatever. But if the units are different, but the variables still measure the same thing, like, for example, number of diet days per week and calories eaten per meal both measure levels of healthy eating habits but it makes no sense to simply add two such variables. It is literally like adding apples and oranges. The solution is to -transform the variables you want to add first. The -transform converts whatever units the original variable has to the -transformed variable’s units of standard deviation distance from the mean. So when you add two -transformed variables you end up with another variable whose units are standard deviation distance from the mean.

Let’s start by opening the file “HeightLatency.sav” from the Data Sets. There are two variables in this file that we will combine into fewer variables. We begin by combining the variables Height and Latency into a new variable.

SPSS screenshot © International Business Machines Corporation.

Since Height and Latency have different units, we need to -transform them first by running a descriptive analysis, making sure you have the “Save standardized values as variables” box checked :

SPSS screenshot © International Business Machines Corporation.

Hit Ok. This will produce two new variables, visible in the Data View window, called ZHeight and ZLatency. We don’t care about the actual descriptive statistics output here. Now you can simply add the -transforms to produce the required new variable :

SPSS screenshot © International Business Machines Corporation.

Now let’s combine a couple of sets of variables that have compatible units. First add ZHeight to ZLatency (note the fancy new way to add) to produce a new variable Sub :

SPSS screenshot © International Business Machines Corporation.

The new variable shows clearly on SPSS sheet :

SPSS screenshot © International Business Machines Corporation.

Next we will make a conversion from a quantitative variable to a qualitative variable essentially by dividing the data into classes. First a simple case. Create the new variable Life from the variable Latency as the following :

SPSS screenshot © International Business Machines Corporation.

We’ll need to do this in two steps. First pull up Transform Compute Variable and set it up so that 1 is in the Numeric Expression box. Then hit the If… button at the bottom left hand of the menu window to bring up :

SPSS screenshot © International Business Machines Corporation.

Then click Continue, then hit OK. That will create the new LatencyCat variable, with missing values. Those values will be filled in the next step.

SPSS screenshot © International Business Machines Corporation.

Pull up Transform → Compute Variable again and, leaving LatencyCat where it is, put 2 in the Numeric Expression box, then hit the “If” button again and change the expression in the condition box, then hit Continue, then OK. Now LatencyCat is either 1 or 2 with no missing values :

SPSS screenshot © International Business Machines Corporation.

4.4 RStudio Lesson 3: Combining variables - advanced

[Coming soon]

5.1 Discrete versus Continuous Distributions

We can describe populations in terms of discrete variables () or continuous variables (). In the last chapter we saw how to describe discrete probability distributions with the example of the binomial distributions. Discrete probabilities need to be added in inferential statistics and this can lead to complicated formulae. Calculus turns sums into integrals[1] which generally lead to simpler formulae. In the following table we compare, and show the relationship between, discrete and continuous variables and their associated probability distributions.

Discrete

Continuous

- We have a finite number of values between the high and low values

- A histogram plot of the random variables may be interpreted as a probability distribution.

- We have an infinite number of values between the high and low values.

- With continuous random variables we have a probability density.

By increasing the number of values in an appropriate limiting way you make
the discrete probability distribution
approach a probability density.

- The units of are probability.

- The units of are probability density. Probabilities are given by areas under the curve only.

We will be slurring our language and call a probability density, a probability distribution. So we’ll say normal distribution instead of normal density. Continuing the comparison, probability distributions and densities have means, moments, skewness, etc. :

- Means and variances of a discrete probability distribution, , are given by the application of the grouped data formulae we saw in Chapter 4 :

- Means and variances of a continuous probability density, are given by the integrals :

Recall that the variance is the second moment of about the mean .

We don’t have to stop at the second moment about the mean. The third and fourth moments about the mean are called skewness and kurtosis respectively :

Discrete

Continuous

Skewness

Kurtosis

SPSS will easily compute skewness and kurtosis. is positive for a positively skewed distribution, negative for a negative skewed distribution. The and are “normalization” factors; they make the moments of the normal distribution simple.

The moments of a probability distribution are important. In fact, if you specify all the moments of a distribution then you have completely specified the distribution. Let’s say that in another way. The specify a probability distribution you can either give its formula (as generally derived from counting) or you can give all its moments. The normal distribution with a mean of and a variance of is specified by the formula

(5.1)

or by its moments. The normal distribution with a mean of and a variance of is the only continuous probability distribution with moments (from first to second an on up) of: , , 0, 1, 0, 1, 0, . The normal distribution is special that way among probability distributions.

1. If you have no calculus background, an integral is a way of calculating areas under curves. ↵

5.2 **The Normal Distribution as a Limit of Binomial Distributions

The results of the derivation given here may be used to understand the origin of the Normal Distribution as a limit of Binomial Distributions[1]. A mathematical “trick” using logarithmic differentiation will be used.

First, recall the definition of the Binomial Distribution[2] as

(5.2)

where is the probability of success, is probability of failure and

(5.3)

is the binomial coefficient that counts the number of ways to select items from items without caring about the order of selection. Here is a discrete variable, , with .

The trick is to find a way to deal with the fact that ( is a discrete variable) for the Binomial Distribution and ( is a continuous variable) for the Normal Distribution[3] In other words as we let we need to come up with a way to let shrink[4] so that a probability density limit (the Normal Distribution) is reached from a sequence of probability distributions (modified Binomial Distributions). So let represent the Normal Distribution with mean and variance . We will show how where each Binomial Distribution also has mean and variance .

The heart of the trick is to notice[5] that

(5.4)

This is perfectly true for the density . The trick is to substitute the distribution for the density in the RHS of Equation (5.4) to get :

(5.5)

because . The trick is to now pretend that is a continuous function defined at all ; we just don’t know what its values should be for non-integer . With such a “continuation” of we can write[6]

(..)

Equation (5.8) has no limit; it blows up as . We need to transform in such a way to gain control on (getting it to shrink as ) and to get something that converges. To do that we introduce and a new variable . With this transformation of variables, the chain rule gives

(5.9)

and the RHS of Equation (5.8) becomes, using

(..)

Using Equation (5.9), for the LHS, and Equation (5.14), for the RHS, Equation (5.8) becomes

(..)

where means terms that will go to zero as , and we have used the relation to get Equation (5.16}) and to go from Equation (5.17) to Equation (5.18). Dividing both sides of Equation (5.19) by leaves

(5.20)

Our transformation, with its , has given us the exact control we need to keep the limit from disappearing or blowing up. Integrating Equation (5.20) gives

(5.21)

where is the a constant of integration. Switching back to the variable

(..)

To evaluate the constant of integration, , we impose because we want to be a probability distribution. So

(5.25)

so

(5.26)

and

(5.27)

which is the Normal Distribution that approximates Binomial Distributions with the same mean and variance as gets large.

Figure 5.1 : The transformation effectively shrinks the of the Binomial Distribution with mean and variance by pulling a continuous version back to the constant Normal Distribution . Another way of thinking about it is that the transformation takes the fixed Normal Distribution to the Normal Distribution that provides a better and better approximation of as .

You may be wondering why that transformation worked because it seems to have been pulled from the air. According to Lindsay & Margenau, it was Laplace who first used this transformation and derivation in 1812. What this transformation does is pull the Binomial Distribution back to have a mean of zero (by subtracting ) which keeps from running off to infinity and, more importantly, allows us to define a function with that has a constant variance of that we can match to when we transform back to at each , see Figure 5.1. Looking at it the other way around, the Normal Distribution[7] with is an approximation for Binomial Distribution that “asymptotically” approaches as .

This is not the only way to form a probability density limit from a sequence of Binomial distributions. It is one that gives a good approximation of the Binomial Distribution when is fairly small if the term in Equation (5.18) becomes small quickly. If is very small, this does not happen and another limit of Binomial Distributions that leads to the Poisson Distribution is more appropriate. When and are close to 0.5 or more generally when and then the Normal approximation is a good one. Either way, the density limit is a mathematical idealization, a convenience really, that is based on a discrete probability distribution that just summarizes the result of counting outcomes. Counting gives the foundation for probability theory.

1. The formula for the Binomial Distribution was apparently derived by Newton according to: Lindsay RB, Margenau. Foundations of Physics. Dover, New York, 1957 (originally published 1936). For that claim, Lindsay & Margenau quote: von Mises R. Probability, Statistics, and Truth. Macmillan, New York, 1939 (originally published 1928). The derivation of the Normal Distribution presented here largely follows that given in Lindsay & Margenau's book. ↵

2. In class we denoted the Binomial distribution as . Here we use to avoid using too many P's and p's. ↵

3. Remember that the Normal Distribution is technically a probability density but we slur the use of the word distribution between probability distribution (discrete ) and probability density (continuous ) like everyone else. ↵

4. for the Binomial Distribution. ↵

5. Remember that and use the chain rule to notice this. ↵

6. You can probably imagine many ways to continue the Binomial Distribution from to . It doesn't matter which one you pick as long as the behaviour of your new function is not too crazy between the integers; that is, should exist at all . ↵

7. Our symbols here are not mathematically clean; we should write something like instead of or composed with at , , instead of . But to emphasize the intuition we use . In clean symbols, the function asymptotically approaches where . ↵

5.3 Normal Distribution

Let us now take a detailed look at the normal distribution and learn how to apply it to probability problems (in sampling theory) and statistical problems. Its formula (which you will never have to use because we have tables and SPSS) is again:

(5.28)

The factor is a normalization factor that ensures that the area under the whole curve is one:

Without that factor we just have a bell-shaped curve [1] with the area under the curve equal to one we have a probability function since the total probability is one. For those with a bad math background, the letters in Equation (5.28) are: [2], [3], = mean and = standard deviation of the normal distribution. The normal distribution’s shape is as shown in Figure 5.2.

Figure 5.2: The normal distribution. It is a bell-shaped curve with its mode (= mean and median because it’s symmetric, ) centred on its mean . On the left is a distribution with a large and on the right one with a smaller .

To work with normal distribution, in particular so we can use the Standard Normal Distribution Table and the t Distribution Table in the Appendix, we need to transform it to the standard normal distribution using the -transform. We need to transform , which has a mean and standard deviation to which has a mean of 0 and a standard deviation of 1. Recall the definition of the -transform:

applying this to gives

(5.29)

If we substitute Equation (5.28) into Equation (5.29) and do the algebra we get :

(5.30)

Equation (5.30) defines the standard normal distribution, or as we’ll call it, the -distribution.
Areas under are given in the Standard Normal Distribution Table in the Appendix.

5.3.1 Computing Areas (Probabilities) under the standard normal curve

Here we learn how to use the Standard Normal Distribution Table to get probabilities associated with any old area under the normal curve that we can dream up. The general layout of areas under the -distribution is shown in Figures 5.3 and 5.4.

Figure 5.3 : The -distribution is a probability distribution (total area = 1) and symmetric, so the area on either side of the mean (which is 0) is a half. You will need to remember this information as you calculate areas using the Standard Normal Distribution Table.

Figure 5.4 : The units of in are standard deviations. No matter what the measurement units of were before the -transformation, the units of are “standardized” to be standard deviation units. With SPSS you will learn how to standardize (-transform) variables so that you can sensibly combine multiple dependent variables into one dependent variable for univariate statistical analysis. The areas, probabilities, associated with each increment in are shown here.

Let’s divide the types of areas we want to compute into cases, following Bluman[4]. For all these cases we’ll use the notation to represent the area we look up in the Standard Normal Distribution Table associated with .

Case 1 : Areas on one side of the mean. This is the case of finding an area between 0 (which corresponds to the mean before any -transformations) and a given . For this case we simply use the tabulated values, , see Figure 5.5. This case also covers when is a negative number: .

Figure 5.5 : Case 1: Areas on one side of the mean.

Example 5.1 : Find the probability that is between 0 and 2.34.

Solution : Look up in the Standard Normal Distribution Table, see Figure 5.6. . (Note that it makes no difference whether we use or because the probability of a single value is 0. That’s why we need to use areas.)

Figure 5.6 : The situation for Example 5.1.

▢

Example 5.2 : Find the probability that is between -1.75 and 0.

Solution : , see Figure 5.7.

Figure 5.7 : The situation for Example 5.2.

▢

Case 2 : Tail areas. A tail area is the opposite of the area given in the Standard Normal Distribution Table on one half of the normal distribution, see Figure 5.8. The tail area after a given positive is z) = 0.5 - A(z)" title="Rendered by QuickLaTeX.com" height="18" width="211" style="vertical-align: -4px;"> or before a given negative value is .

Figure 5.8 : Case 2 : Tail areas.

Example 5.3 : What is the probability that 1.11" title="Rendered by QuickLaTeX.com" height="13" width="64" style="vertical-align: -1px;">?

Solution : 1.11) = 0.5 - A(1.11) = 0.5 - 0.3665 = 0.1335" title="Rendered by QuickLaTeX.com" height="18" width="408" style="vertical-align: -4px;">, see Figure 5.9.

Figure 5.9 : The situation for Example 5.3.

▢

Example 5.4 : What is the probability that ?

Solution : , see Figure 5.10.

Figure 5.10 : The situation for Example 5.2.

▢

Case 3 : An interval on one side of the mean. Recall that for the -distribution. So we are looking for the probabilities for an interval to the right of the mean or for an interval to the left of the mean. In either case , see Figure 5.11.

Figure 5.11: Case 3: An interval on one side of the mean.

Example 5.5 : What is the probability that is between 2.00 and 2.97?

Solution : , see Figure 5.12.

Figure 5.12: The situation for Example 5.5.

▢

Example 5.6 : What is the probability that is between -2.48 and -0.83?

Solution : , see Figure 5.13.

Figure 5.13: The situation of Example 5.6.

▢

Case 4 : An interval containing the mean. The situation is as shown in Figure 5.14 with the interval being between a negative and a positive number. In that case .

Figure 5.14: Case 4: An interval containing the mean.

Example 5.7 : What is the probability that is between -1.37 and 1.68?

Solution : , see Figure 5.15.

Figure 5.15: The situation for Example 5.7.

▢

Cases 5 & 6 : Excluding tails. Case 5 is excluding the right tail, . Case 6 is excluding the left tail, -z)" title="Rendered by QuickLaTeX.com" height="18" width="83" style="vertical-align: -4px;">. See Figure 5.16. Case 5 is the situation which gives the percentile position of if you multiply the are by 100. More about percentiles in Chapter 6. In either case, .

Figure 5.16: Left: Case 5. Right: Case 6.

Case 7 : Two unequal tails. In this case we add the areas of the left and right tails, see FIgure 5.17. The special case where the tails have equal areas (i.e. when in the notation we have been using) is the case we will encounter for two-tail hypothesis testing. .

Figure 5.17: Case 7: Two unequal tails.

Example 5.8 : Find the areas of the tails shown in Figure 5.18.

Solution :

2.43)" title="Rendered by QuickLaTeX.com" height="18" width="197" style="vertical-align: -4px;">

.

Figure 5.18: The situation for Example 5.8.

▢

Using the Standard Normal Distribution Table backwards

Up until now we’ve used the Standard Normal Distribution Table directly. For a given , we look up the area . Now we look at how to use it backwards: We have a number that represents the area between 0 and , what is ? Let’s illustrate this process with an example.

Example 5.9 : We are given an area as shown in Figure 5.19. What is {z}?

Solution : Look in the Standard Normal Distribution Table for the closest value to the given . In this case 0.2123 corresponds exactly to .

Figure 5.19: The situation for Example 5.9.

▢

Example 5.9 was artificial in that the given area appeared exactly in the Standard Normal Distribution Table. Usually it doesn’t. In that case pick the nearest area in the table to the given number and use the associated with the nearest area. This, of course, is an approximation. For those who know how, linear interpolation can be used to get a better approximation for .

The -transformation preserves areas

In a given situation of sampling a normal population, the mean and standard deviation of the population are not necessarily 0 and 1. We have just learned how to compute areas under a standard normal curve. How do we compute areas under an arbitrary normal curve? We use the -transformation. If we denote the original normal distribution by and the -transformed distribution by then areas under will be transformed to areas under that are the same. The -transformation preserves areas. So we can compute areas, or probabilities under using the Standard Normal Distribution Table and instantly have the probabilities we need for the original . Let’s follow an example.

Example 5.10 : Suppose we know that the amount of garbage produced by households follows a normal distribution with a mean of pounds/month and a standard deviation of pounds/month. What is the probability of selecting a household that produces between 27 and 31 pounds of trash/month?

Solution : First convert and to their -scores:

Then, referring to Figure 5.20, we see that the probability is .

→ z-transform →

Figure 5.20 : The situation of Example 5.10. Left is the given population, . On the right is the -transformed version of the population . The value 27 is -transformed to -0.5 and 31 is -transformed to 1.5.

▢

In Example 5.10 we used the Standard Normal Distribution Table directly. You will also need to know how to solve problems in which you use this table backwards. The next example shows how that is done. For this kind of problem you will find the first and then you will need to find using the inverse -transformation :

which is derived by solving the -transformation, for .

Example 5.11 : In this example we work from given . To be a police person you need to be in the top 10\% on a test that has results that follow a normal distribution with an average of and .

What score do you need to pass?

Solution : First, find the such that z) = 0.10" title="Rendered by QuickLaTeX.com" height="18" width="163" style="vertical-align: -4px;">. That is a right tail area (Case 2), so we need , look at Figure 5.21 to see that. Then, going to the Standard Normal Distribution Table, look for 0.4 in the middle of the table then read off backwards. The closest area is 0.3997 which corresponds to . Using the inverse -transformation, convert that to an :
to get

or, rounding, use . There are frequently consequences to our calculations and in this case we want to make sure that we have a score that guarantees a pass. So we round the raw calculation up to ensure that.

← inverse z-transform ←

Figure 5.21 : The situation of Example 5.11

▢

1. **Whose shape is determined essentially by the shape of . Plot and think about the square preventing any negative values for the argument. ↵

2. ** The number is the natural base implied by functions whose values match how fast it changes, i.e. the derivative of the function is the same as the function. ↵

3. ** Of course, comes from circles: = circumference/diameter. ↵

4. Bluman AG, Elementary Statistics: A Step-by-Step Approach, numerous editions, McGraw-Hill Ryerson, circa 2005. ↵

The concept of percentile[1] applies to either a data set (sample, as represented by a histogram — a discrete distribution) or to a continuous distribution (which represents a population) as shown in Figure 6.1.

Figure 6.1: The concept of percentile applies to either a data set or to a continuous distribution.

The percentile position of the data point , denoted here by , is the percentage of the area under the curve up to the point . Notation warning : Do not confuse percentile and probability, we use to denote both!! (They are related though.)

To determine the percentile position for from a normal distribution of values, convert to via the -transformation, determine the area under the standard normal curve up to and multiply by 100. We have, therefore, already seen how to compute given or how to compute for a given percentile . See Case 5 in Section 5.3 and remember how to use the Standard Normal Distribution Table forward and backwards.

1. This percentile stuff is all about cumulative frequency or (thinking about probabilities) cumulative relative frequencies. The corresponding probability functions are called Cumulative Distribution Functions or CDFs. You will encounter CDFs in SPSS; they are mentioned later in this chapter. ↵

6.1 Discrete Data Percentiles and Quartiles

Before we get into how to calculate percentile in a data set, note that we can see percentiles directly on a cumulative frequency plot, see Figure 6.2.

Figure 6.2 : With a cumulative frequency plot, we can read percentiles off the axis. If you have a newborn baby and take it to the doctor for their first check up, they will measure the baby’s head circumference and tell you the baby’s head size percentile by looking at such a chart. The doctor’s chart will be based on an accumulation of a very large number of essentially population data. Cumulative frequency graphs, or more exactly cumulative probability graphs, can be made for continuous distributions like the normal distribution. The resulting function is the Cumulative Distribution Function, or CDF, and is, for example, P(z) represents the z-distribution then CDF . We will see this CDF in SPSS.

Computing percentile positions of discrete data. Let be the ordered position of a data set of data points, then we define the percentile position of to be

(6.1)

This formula has the property that and . It is what we will use as a percentile formula but it is not the only one. Look at Figure 6.1. The way the histogram there is shaded the formula would be which would have the property that and . There are other, not necessarily wrong, ways to define the percentile position of discrete data but we will use Equation 6.1.

If you want to find the position, , of the data point corresponding to a given percentile then compute

(6.2)

Equation (6.2) is derived by solving Equation (6.1) for . Note that Equation (6.2) gives the position of the data point , not its value. To clarify that, let’s look at an example.

Example 6.1 : Consider the dataset given below. Data would originally be given as the numbers in the first line. So the first step in answering any question about percentiles is to order the data, the same as what you need to to to determine the median of a dataset. Once the data are ordered, then you may assign a position number to each data point as shown in the third line.

original data

18

15

12

6

8

2

3

5

20

10

ordered data

2

3

5

6

8

10

12

15

18

20

1

2

3

4

5

6

7

8

9

10

Q : What is the percentile rank of ?
A : so percentile.

Q : What is the value corresponding to the percentile, ?
A :

The closest is 3 and . We can write .

▢

Decile :

The decile of data value in the ordered position is defined as

We will not make much use of decile except to see that quartile is defined in the same way.

Quartile :

The quartile of data value in the ordered position 1.

(6.3)

Notation : (This notation also applies to and .) We write :

&=& quartile
& = & quartile
& = & quartile
& = & quartile
& = & quartile

Quartiles are useful because we do not have to compute percentile first and then divide by 25 as given by Equation (6.3). Instead, we can use the following handy tricks after ordering our data:

Example 6.2 : Example with an even number of data points. With the data in order, first find the median, then the medians of the two halves of the dataset :

▢

Example 6.3 : Example with an even number of data points. With the data in order, first find the median, then the medians of the two halves of the dataset :

▢

6.2 Finding Outliers Using Quartiles

We can use quartiles to identify outliers or data points that are wildly discrepant with the rest of the data. For this application, we need another definition of data dispersion :

With the IQR any data value that satisfies:

(a) less than
or
(b) greater than

…is considered an outlier. This is one of many ways one can define an outlier. As we will discuss below, it is a robust way of identifying outliers.

Example 6.4 : Consider the data of Example 6.2. We found

so,

Following our rules for finding outliers, we compute:

(a) lower acceptable value limit

(b) upper acceptable value limit

and 50 > 36.5 so 50 is considered an outlier.

▢

6.3 Box Plots

A box plot is a plot that shows , and MD (= ) along with H and L (= and ) as shown in Figure 6.3. It especially emphasizes the IQR.

Figure 6.3: The features of a box plot, also known as a box-and-whiskers plot. When one of the whiskers is more than 1.5 times the length of the box (the IQR) then there are outliers by our definition in Section 6.2. The data line shown below the box plot is a construction line and not part of the box plot.

Example 6.5 : Construct a box plot for the data shown in Figure 6.4. Again, someone has done the first, tedious, step of ordering the data for us.

Figure 6.4: Construction of a box plot from the given data.

▢

Box plots can also be drawn vertically. SPSS draws box plots vertically; this is especially useful for comparing datasets.

6.4 Robust Statistics

A robust statistic or resistant statistic is one that is less affected by outliers than a non-robust or non-resistant statistic. If you look at the numbers in Example 6.2 you can see that the value of the MD (and IQR) is completely unaffected by the value of the outlier data point 50. The mean and the standard deviation will, however, be greatly affected by the value of the outlier. So while some people may identify outliers as those being (say) 3 from the mean, we see that that is a non-robust way of identifying outliers. In summary:

Measures of central tendency and dispersion

Robust

Non-robust

MD
IQR

It would seem that inferential statistics based on robust statistics would be better than statistics based on non-robust values. Maybe. But, traditionally, statistical analysis like the -tests, ANOVA and regression, are based on the non-robust statistics of means and standard deviations (or variance). People tend to use robust statistics in “Exploratory Data Analysis” (EDA). With EDA one is not concerned so much with testing hypothesis as in trying to get an understanding of general trends in the data. The techniques, and statistics, the fall under the two categories are:

Traditional

Exploratory Data Analysis (EDA)

Frequency Tables
Histogram
Mean,
Standard Deviation,

Stem and Leaf Plot
Box Plot
Median, MD
Interquartile Range, IQR

You will find an EDA menu under Analyze → Descriptives in SPSS.

6.5 SPSS Lesson 4: Percentiles

To follow along, load in the file “AgeSmoker.sav” from the Data Sets. We will pick on the variable Age. We will compute the percentile rank of each value in the Age dataset two ways. One way, we will treat the data as a discrete data set and will compute the percentile position following Equation 6.1. The other way, we will treat the data as if they came from a normal population.

SPSS screenshot © International Business Machines Corporation.

First, treat the data as a stand alone discrete data set. First we need to rank the data; the ranks are the values in Equation 6.1. Use Transform Rank Cases :

SPSS screenshot © International Business Machines Corporation.

This produces the ranking variable RAge, visible in the Data View window.

SPSS screenshot © International Business Machines Corporation.

Now use that ranking variable in Equation 6.1 by pulling up Transform Compute Variable :

SPSS screenshot © International Business Machines Corporation.

The result, in the Data View window, looks like :

SPSS screenshot © International Business Machines Corporation.

We can sort the data on RAge using Data Sort Cases :

SPSS screenshot © International Business Machines Corporation.

Note how the smallest value has percentile rank 0. If you scroll to the end of the list you will see that the largest value has percentile rank 100.

SPSS screenshot © International Business Machines Corporation.

SPSS screenshot © International Business Machines Corporation.

CDF stands for Cumulative Distribution Function. It is literally the cumulative area under a probability distribution function, in this case the normal distribution. So multiplying it by 100 give the percentile rank. The output, in the Data View window looks like :

SPSS screenshot © International Business Machines Corporation.

Note how the percentile ranks of gparank are different from, but close to, the percentile ranks of perrank computed using the data’s own distribution. This indicates that the data themselves follow an approximately normal distribution.

6.6 RStudio Lesson 4: Percentiles

[Coming soon]

Before we can learn about confidence intervals in Chapter 8 and hypothesis testing in the Chapter 9, we need a couple of results that form the foundation of the usefulness of the normal distribution. We have mentioned that the normal distribution can be derived as a limit of binomial distributions. This fact can be used in reverse and we can use the normal distribution to approximate the binomial distribution. This approximation will be useful for inferences (confidence intervals and hypothesis testing) on proportions. The second result is the very important central limit theorem where the normal distribution pops out as the answer to the characterization of random sample means. The central limit theorem gives us the sampling theory for all statistical inference procedures involving means.

7.1 Using the Normal Distribution to Approximate the Binomial Distribution

Recall the definitions: = probability of success, = probability of failure and = sample size. When and then the normal distribution is very close, numerically, to the binomial distribution.

Using the histogram way of drawing the binomial distribution, a good fit looks like that shown in Figure 7.1.

Figure 7.1: A normal distribution with mean and standard deviation is a good fit to the binomial distribution with the same mean and standard deviation as long as and .

A couple of things to note about this approximation:

1. Although the values of the normal and the binomial distributions match well at equal to integer values when  and , the areas match not as well. A “correction for continuity” can be used to better make the areas match but we won’t be worrying about such fine details in our studies.

2. We will use the normal approximation to the binomial make inferences on proportions.  In that case , the probability of success will represent a proportion in a population.

7.2 The Central Limit Theorem

Now we come to the very important central limit theorem. First, let’s introduce it intuitively as a process :

1. Suppose you have a large population (in theory infinite) with mean and standard deviation (and any old shape).

2. Suppose you have a large sample, size , of values from that population. (In practise we will see that is large.) Take the mean, , of that sample. Put the sample back into the population[1]

3. Randomly pick another sample of size . Compute the mean of the new sample, . Return the sample to the population..

4. Repeat step 3 an infinite number of times and build up your collection of sample means .

5. Then[2] the distribution of the sample means will be normal will have a mean equal to the population mean, , and will have a standard deviation of

where is the population’s standard deviation. is known as the standard error of the mean.

Now let’s visualize this same process using pictures :

- Take a sample of size from the population and compute the mean (see Figure 7.2a).

Figure 7.2a

- Put them back and take more data points.

- Do this over and over to get a bunch of values for . Those values for will be distributed as shown in Figure 7.2b.

Figure 7.2b

The central limit theorem is our fundamental sampling theory. It tells us the if we know what the mean and standard deviation of a population[3] are then we can assign the probabilities of getting a certain mean in a randomly selected sample from that population via a normal distribution of sample means that has the same mean as the population and a standard deviation equal to the standard error of the mean.

To apply this central limit theorem sampling theory we will need to compute areas under the normal distribution of means. In order to do that, so we can use the Standard Normal Distribution Table, we need to convert the values to a standard normal using the -tranformation as usual: . So, for the distribution of sample means the appropriate -transformation is :

Example 7.1 : Assume that we know, say from SGI’s database, that the mean age of registered cars is months and that the population standard deviation of the cars is months. We make no assumption about the shape of the population distribution. Then, what is the answer to the following sampling theory question: What is the probability that the mean age is between 90 and 100 months in a sample of 36 cars?

Solution : The central limit theorem tells us that sample means will be distributed as shown in Figure 7.3.

Figure 7.3 : Distribution of mean age from samples of 36 cars.

Convert 90 and 100 to -scores as usual:

Then, the required probability using the Standard Normal Distribution Table is

▢

1. This is redundant since the population is infinite, but for conceptual purposes imagine that you return the items to the population. ↵

2. More precisely, the distribution of sample means asymptotically approaches a normal distribution as . But 30 is close enough to infinity for most practical purposes and the statistical inferential tests that we will study will assume that the distribution of sample means will be normal. ↵

3. In hypothesis testing we know what the mean of the population in the null hypothesis is. ↵

8.1 Confidence Intervals Using the z-Distribution

With confidence intervals we will make our first statistical inference. Confidence intervals give us a direct inference about the population from a sample. The probability statement is one about hypotheses about the mean of the population based on the mean and standard deviation of the sample. This is a fine point. The frequentist definition of probability gives no way to assign a probability to a hypothesis. How do you count hypotheses? The central limit theorem makes a statement about the sample means on the basis of a hypothesis about a population, about its mean and standard deviation . If the population is fixed then the central limit theorem gives the results of counting sample means, frequentist probabilities. If we let represent a hypothesis about a population (i.e. that it is described by and ) and let represent data (with mean ) then the central limit theorem gives the probability . The confidence intervals that we’ll look at first give . We’ll look at the recipe for computing confidence intervals for means first, then return to this discussion about probabilities for hypotheses.

Our goal is to define a symmetric interval about the population mean that will contain all potentially measured values of with a probability[1] of .

Typically will be

The assumptions that we need in order to use the -distribution to compute confidence intervals for means are :

1. The population standard deviation, , is known (a somewhat artificial assumption since it is usually not known in an experimental situation) or

2. The sample size is greater than (or equal to) 30, and we use , the sample standard deviation in our confidence interval formula.

Definition : Let where be the -value, from the Standard Normal Distribution Table that corresponds to an area, between 0 and of as shown in Figure 8.1.

Figure 8.1 : The -distribution areas of interest associated with .

To get our confidence interval we simply inverse -transform the picture of Figure 8.1, taking the mean of 0 to the sample mean and the standard deviation of 1 to the standard error as shown in Figure 8.2.

Figure 8.2 : The inverse -transformation of Figure 8.1 gives the confidence interval for .

So here is our recipe from Figure 8.2. The -confidence interval for the mean, under one of the two assumptions given above, is :

or using notation that we will use as a standard way of denoting symmetric confidence intervals

(8.1)

where

The notation is more convenient for us than because we will use the t Distribution Table in the Appendix to find very quickly. We could equally well write

but we will use Equation (8.1) because it explicitly gives the bounds for the confidence interval.

Notice how the confidence interval is backwards from the picture that the central limit theorem gives, the picture shown in Figure 8.3. We actually had no business using the inverse -transformation to arrive at Figure 8.2. It reverses the roles of and . We’ll return to this point after we work through the mechanics of an example.

Figure 8.3 : The central limit theorem is about distributions of sample means.

Example 8.2 : What is the 95 confidence interval for student age if the population is 2 years, sample , ?

Solution : So . First write down the formula prescription so you can see with numbers you need:

First determine . With the tables in the Appendices, there are two ways to do this. The first way is to use the Standard Normal Distribution Table noting that we need the associated with a table area of . Using the table backwards we find . The second way, the recommended way especially during exams, is to use the t Distribution Table. Simply find the column for the 95 confidence level and read the from the last line of the table. We quickly find .

Either way we now find

so

with 95 confidence.

▢

1. Because of this issue about probabilities of hypotheses, many prefer to say "confidence" and not probability. But we will learn enough about Bayesian probability to say "probability". ↵

8.2 **Bayesian Statistics

Now that we’ve seen how easy it is to compute confidence intervals, let’s give it a proper probabilistic meaning. To extend probability from the frequentist definition to the Bayesian definition, we need Bayes’ rule. Bayes’ rule is, for events and :

Study Figure 8.4 to convince yourself that Bayes’ rule is true. Notice that

and

So, equating from each of those two perspectives, we get Bayes’ rule.

If we let (hypothesis) and (data), Bayes’ rule gives us a way to define the probability of hypothesis through

(8.2)

The quantity is known as the prior probability of the data relative to the hypothesis and is something that can be computed in theory if probabilities are assigned in a reasonable manner. The specification of prior probabilities is a contentious issue with the Bayesian approach. Really, it represents a prior belief. The quantity is what sampling theory, like the central limit theorem, gives and is known as the likelihood. Finally the quantity is known as the posterior probability. Equation (8.2) is an expression about probability distributions as well as individual probabilities (just allow and to vary).

Figure 8.4 : Venn diagram illustration of Bayes rule.

If we assign for the prior probability then . We can switch the roles of and ! Of course is not a probability distribution because the area under a function whose value is always 1 is infinite. The area under a probability distribution must be 1. So is an improper distribution (as a function of either or ). But note that an improper distribution times a proper distribution here gives rise to a proper distribution. With this slight of hand, we can give confidence intervals a probabilistic interpretation.

8.3 The t-Distributions

As a broad introduction, the -distributions are family of distributions that give different approximations to the -distribution as shown in Figure 8.5.

Figure 8.5 : The -distributions are a family of distributions, labeled here by their degrees of freedom as in .

As the degrees of freedom, , increases, become closer to , . In practice, as reflected in the t Distribution Table, is very very close to .

The -distributions arise as a corollary to the central limit theorem; they give the distribution of sample means when knowledge of the population is replaced by using the sample mean . When we encounter the distribution later, we will give a more exact mathematical specification of the -distributions.

Similar, to the -distribution case, the confidence interval for the mean for small samples is given by

where, now

With this new formula for we have replaced with in comparison with the formula we used in Section 8.1: Confidence Intervals using the z-distribution and, of course, replaced with . Some books use like the of Section 8.1. We use because we’ll look up its value in the t Distribution Table in the column for confidence intervals (just like we did with ) and with the degrees of freedom specifying the row. The formula for the degrees of freedom in this case is :

The specify a probability as shown in Figure 8.6. As before, the inverse -transform, in the form from the -distribution on the left of Figure 8.6 to the distribution on the right of Figure 8.6 leads to our confidence interval formula for small means. And as before we should justify using that transform from a Bayesian perspective.

→ inverse -transform →

Figure 8.6 : Derivation of confidence intervals for means of small samples.

Example 8.2 : Given the following data:

find the 99% confidence interval for the mean.

Solution : First count and then, with your stats calculator compute

Using the t Distribution Table with in the 99% confidence interval column, find

With these numbers, compute

so

is the 99 confidence interval for .

▢

8.4 Proportions and Confidence Intervals for Proportions

We will now make use of the approximation of the binomial distribution by the -distribution given in Section 7.1: Using the Normal Distribution to Approximate the Binomial Distribution. As usual, the confidence interval will switch the roles of population and sample quantities. The recipe will be laid out first, then we will connect it to what you know about the binomial distribution.

First some definitions. Let be the number of items in a population of size that have a given quality. (e.g. the number of females in a population; or the number of people at the U of S wearing yellow sweaters). Then the proportion of the population having the given quality is

Given a sample from the population of size , the best estimate for is:

where is the number of items in the sample having the given quality. To go along with we also have

which is is the proportion of items in the sample without the given quality.

To compute an confidence interval for a proportion we need to compute

and it must be true that both and (otherwise we need to use the binomial distribution directly).

With , the confidence interval for a proportion is given by

To derive the proportions confidence interval formula we’ll begin with the sampling theory given by the binomial distribution and the corresponding -approximation. Then we’ll switch the roles of and . Let

be the mean, the expected value, of that you expect to find in a sample of size randomly selected from the population with a proportion of items of interest. This is true because is also the probability of randomly selecting an item of interest (the probability of success) from the population as per what we did in Chapter 4. The binomial distribution tells you the probability of getting different numbers of items of interest in your sample given . The binomial distribution that describes our situation is shown in Figure 8.7; it has a standard deviation of .

Figure 8.7 : The binomial distribution relevant to forming a sample of size with items of interest from a population with a proportion of items of interest. The normal distribution with the same and is shown.

Moving to the normal approximation, we have the picture of Figure 8.8.

Figure 8.8 : The normal distribution relevant to forming a sample of size with items of interest from a population with a proportion of items of interest. The boundaries of the area follow from an inverse -transform of the -distribution to a normal distribution of mean and standard deviation , .

Figure 8.8 says :

with a (frequentist) probability of . This is our sampling theory. Divide by :

Swapping the roles of the population and sample, we arrive at the confidence interval formula :

Time for a worked example.

Example 8.3 : A sample of 500 nursing applications included 60 men. Find the 90% confidence interval of the true proportion of men who applied to the nursing program.

Solution : From the t Distribution Table, look up

and compute

Then

is the confidence interval with 90% confidence.

▢

Sample size need for a poll

Measuring proportions is what pollsters do. For example in an election you might want to know how many people will vote for liberals (items of interest) and how many will vote for conservatives (items not of interest)[1] In a news paper you might see: “The poll says that 72 of the voters will vote liberal. The poll is considered accurate to 2 percentage points 19 time out of 20.” This means that the 95 confidence interval (19/20 = 0.95) of the proportion of liberal voters is (note how proportions are presented as percentages in the newspaper). The error here is . Before the pollster starts telephoning people, she must know how many people to phone to arrive at that goal error of 2. She needs to know what the sample size needed is. In general, the minimum sample size needed to attain a goal error on a confidence interval of is

Here and could come from a previous survey if available. If there is no such survey or if you want to be sure of ending up with an error equal to or less than a goal E, then use , see Figure 8.9.

Figure 8.9 : The formula is a quadratic formula. Substitute to get or . The maximum of is at .

Example 8.4 : We want to estimate, with 95 confidence, the proportion of people who own a home computer. A previous study gave an answer of 40. For a new study we want an error of 2. How many people should we poll?

Solution : From the question we have :

From the t Distribution Table (or the Standard Normal Distribution Table if you think about the areas correctly) we find

Therefore

Which we round up to a sample size of 2305 to ensure that .

▢

1. We assume here that there are only two parties. For the real life situation of more than two parties we need the multinomial distribution and to approximate it with a multivariate normal distribution. That is a topic for multivariate statistics but the principles are the same as what we cover here. ↵

8.5 Chi Squared Distribution

The (chi squared) distribution is a consequence of a random process based on the normal distribution. It is derived from the normal distribution as the result of the following stochastic process :

1. Suppose you have a population that has variance and is normally distributed.

2. Take a sample of size from the population and compute using the sample standard deviation from that sample.

3. Put the sample back into the population.

4. Take another sample of size from the population and compute using the sample standard deviation from that sample.

5. etc.

6. The distribution of the values of values will be a distribution with degrees of freedom.

Like the -distributions, the distributions are a family, see Figure 8.10.

Figure 8.10 : The distributions are enumerated by degrees of freedom.

The distribution underlies why is the best estimate for . It mean, or expected value is so the expected value of is . The expected value of in a random sample of size is not .

Confidence Intervals on and

The distribution is already normalized in its definition through including in its definition. Therefore no -transforms are needed and we can work directly with a table that gives right tail areas under the distribution. That table is the Chi-squared Distribution Table, in the Appendix, and it gives values of for given values of area to the right of , see Figure 8.11.

Figure 8.11 : The Chi-squared Distribution Table gives associated with given right tail areas.

We’ll need and such that the tail areas are equal and such that the area between them is , see Figure 8.12.

Figure 8.12 : The values and define the confidence region .

Notation : Let’s call the in the Chi-squared Distribution Table and let be the table value that corresponds to . In other words is the value that corresponds to a right tail area of .

So given , the appropriate and are the following values from the Chi-squared Distribution Table:

Note the symmetry of the Chi-squared Distribution Table. If comes from the column 3 columns from the right edge of the table then comes from a column 3 columns from the left edge of the table. Only small and large areas appear in the table, there are no intermediate values.

Finally, the confidence interval for is given by

and for by:

Where the distribution with degrees of freedom (giving the line to use in the Chi-squared Distribution Table) is used.

Example 8.5 : Find the 90 confidence interval on and for the following data

Solution : Compute, using your calculator :

From the Chi-squared Distribution Table, in the line, find :

and

So

Taking square roots:

▢

The process of hypothesis testing can be simplified into :

1. Transform (“reduce”) your given data into a test statistic that you can locate on probability distribution given by the sampling theory under a null hypothesis () about the population. (e.g. or test statistic).

2. See if your test statistic falls into a critical region of the distribution or not. The critical, or rejection region as we’ll call it, represents an area of low probability that the null hypothesis, is true. If the test statistic falls in the rejection region, the we make the decision to reject as the conclusion of the hypothesis test.

Before we define the critical region under the null hypothesis, we need to define what a null hypothesis is.  We’ll define two hypotheses, actually, because the null hypothesis needs to contrasted to its logical opposite :

: Null Hypothesis, the hypothesis that nothing is going on; no effect; no signal.

: Alternative Hypothesis, the hypothesis that is not true; there is an effect; there is a signal.

A good experimental design will be set up so that the effects of interest define . (Your “claim” will be .) Why? It’s about signal to noise ratios. A test statistic is literally signal/noise, a signal to noise ratio. When you do not reject you are saying that there is more noise than signal. When you reject (essentially accepting ) you are saying that there is more signal than noise. Usually you are interested in the signal (also known as an “effect”) so your claim would be . You perform your experiment to find evidence for . If you are interested in noise (can happen, for example to test assumptions on which tests are based) then your claim would be . The examples that follow here don’t follow these experimentally correct rules for which of or should be the claim to emphasize the logical nature of the decision making process. But test statistics are signal to noise ratios and in real life you will be interested in signals.

To fix ideas about hypothesis testing, we’ll first look at hypotheses on the means of populations ().  Later we’ll consider hypotheses on and on (proportions).

With means there are three combinations of and to consider :

Two-Tailed Test

Right-Tailed Test

Left-Tailed Test

:

:

:

:

:

:

Here is a given number. Not that the rightness or the leftness of the one-tailed test is reflected in . is generally what people are interested in. Then the critical regions, which are on distributions as we’ll see, for each case look like :

1. Two-tailed test:

2. Right-tailed test:

3. Left-tailed test:

The critical regions, or rejection regions, appear in the probability distributions , which is the probability distribution that the sample test statistic, , that would occur if were true. These -distributions are -transforms of the distribution of sample means under given by the central limit theorem. More about this when we introduce the formula for the distribution. For now, let’s focus on the decision making process.

When your statistic ends up in the critical region, you conclude that is false. You reject . The critical region is the rejection region.

In the two tailed test, the critical region, with total area is the opposite to the region that we have been using for confidence intervals. Compare the two-tail critical region sketch above to Figure 8.1.

There are four possible outcomes to a statistical hypothesis test given by the so-called[1] “confusion matrix” :

true

true

Reject (believe )

Type I error

Correct decision 1-

Do not reject (believe )

Correct decision 1-

Type II error

The probabilities are relative to the realities. The probabilities in the columns add to 1. The probability of making a Type I error, , is the area in the critical region. The diagram with the critical region on it assumes that is the reality. We will see how to compute in Chapter 13. The quantity is defined as the power of the statistical test.

We can view the confusion matrix from a medical test point of view. A medical test is a hypothesis test has the following hypotheses pairs :

: negative test result, healthy patient

: positive test result, sick patient

Then :

Healthy

Sick

Positive Result
(believe sick)

Type I error

Correct decision 1-

Negative Result
(believe healthy)

Correct decision 1-

Type II error

In medical tests, the quantity is known as the test’s specificity, the probability of finding true negatives. The quantity is the test’s sensitivity, the probability of finding true positives. Generally and are functions of some other decision parameter. In the hypothesis tests that we consider here, is the decision parameter.

Back to understanding the meaning of hypothesis testing. As we said, a good experimental design will be set up so that is your favourite theory that there is an effect. In that case represents the case that there is no effect : the position of away from , or away from 0 (in the case of hypothesis testing of ) is just due to noise. If your experiment is then successful in proving your theory, i.e. you reject , then represents the probability that you are wrong. The number actually defines a decision point for rejecting . Later we will see how to compute a value, , that is associated with the test statistic. This -value is then a more refined value for the probability that you are wrong if you reject . From another point of view, would be the probability that your measurement is entirely due to noise.

Let’s do some examples to build our mechanical skills at defining critical regions for distributions.

Example 9.1 : Critical Areas on -distributions with hypothesis testing on the mean, .

(a) Left-tailed test with . Find the critical value .

First step, draw a picture :

With the tables we have in the Appendix, there are two ways to find :

- Method (a) : Look up area in the Standard Normal Distribution Table equal to 0.40 : Closest is 1.28 so .

- Method (b) : Use the last line in the t Distribution Table for the one tailed test column. Find a of 1.282 and add a minus sign because we have a left tail test. So .

Use Method (b) on tests and exams. It is faster, requires less thinking about areas (and so less chance for making a mistake) and gives a slightly more accurate result. The critical area or critical region or the rejection region is where . The critical value that defines the region in this case is .

(b) A two tailed test with . Find the critical value .

Draw a picture :

- Method (a) : Look up area in the Standard Normal Distribution Table equal to 0.49. The closest is 2.33. So, because we have a two-tailed test, .

- Method (b): Use the last line in the t Distribution Table, for two tailed test, . Find , .

Again, Method (b) is the recommended approach.

So the critical areas are those where

and the critical values are and .

(c) A right tailed test with . Find the critical value .

Draw a picture :

- Method (a) Look up area in the Standard Normal Distribution Table equal to 0.495, the Closest is 2.58. So

- Method (b) Use the last line in the t Distribution Table for one tailed test, and find .

So the critical area is that where and the critical value is .

▢

One final note on setting up the hypotheses. When setting up the hypotheses and , one of the two alternatives will be the claim (what the problem says you really want to test). As mentioned before, a good experimental design will have as the claim. But this may not always be possible to arrange (especially in tests of assumptions). So many of the exercises in the text and assignments will have as the claim.

1. So called not because it is confusing but because you are never 100 sure which decision is correct. ↵

9.1 Hypothesis Testing Problem Solving Steps

Now that we have some background on setting up hypotheses and finding critical regions, we introduce the steps needed for every hypothesis testing procedure. Hypothesis testing is based directly on sampling theory and the probabilities that the sampling theory gives. Here are the steps we will follow :

1. Hypotheses : Formulate and . State which is the claim

2. Critical statistic : Find the critical values and regions. (Use tables of , , , etc. values).

3. Test statistic : Compute the {\em test statistic} from your data. It summarizes your data in one number. The -value follows from the test statistic.

4. Decision : If the test statistic falls in the critical region (rejection region), reject . (This decision can also be made using the -value.)

5. Interpretation : Summarize results in a sentence and/or present a graphic or table.

The definition of a -value will be covered below. For now you should know that a computer program (SPSS) will give you a -value but not a critical statistic. So there is no Step 2 if you use SPSS.

A generic test statistic may be defined by :

The numerator represents a signal or an effect. The denominator represents noise. Not all test statistics will have this form (e.g. some test statistics), but all test statistics represent a signal-to-noise ratio. Much of the tabular output of SPSS gives the numerator and denominator of this generic form with or without the corresponding test statistic.

9.2 z-Test for a Mean

This is our first hypothesis test. Use it to test a sample’s mean when :

1. The population is known.

2. Or When , in which case use in the test statistic formula.

The possible hypotheses are as given in the table you saw in the previous section (one- and two-tailed versions):

Two-Tailed Test

Right-Tailed Test

Left-Tailed Test

:

:

:

:

: k" title="Rendered by QuickLaTeX.com" height="17" width="43" style="vertical-align: -4px;">

:

In all cases the test statistic is

(9.1)

In real life, we will never know what the population is, so we will be in the second situation of having to set in the test statistic formula. When you do that, the test statistic is actually a test statistic as we’ll see. So taking it to be a is an approximation. It’s a good approximation but SPSS never makes that approximation. SPSS will always do a -test, no matter how large is. So keep that in mind when solving a problem by hand versus using a computer.

Let’s work through a hypothesis testing example to get the procedure down and then we’ll look at the derivation of the test statistic of Equation (9.1).

Example 9.2 : A researcher claims that the average salary of assistant professors is more than $42,000. A sample of 30 assistant professors has a mean salary of $43,260. At , test the claim that assistant professors earn more than $42,000/year (on average). The standard deviation of the population is $5230.

Solution :

1. Hypothesis :

42,000" title="Rendered by QuickLaTeX.com" height="16" width="124" style="vertical-align: -4px;">  (claim)

(This is a right-tailed test.)

2. Critical Statistic.

- Method (a) : Find such that from the Standard Normal Distribution Table: ; or

- Method (b) : Look up in the t Distribution Table corresponding to one tail (column), and read the last () line:

Method (b) is the recommended method not only because it is faster but also because the procedure for the upcoming -test will be the same for the -test.

3. Test Statistic.

4. Decision.

Draw a picture so you can see the critical region :

So is in the non-critical region: Do not reject .

5. Interpretation.

There is not enough evidence, from a -test at , to support the claim that professors earn more than $42,000/year on average.

▢

So where does Equation (9.1) come from? It’s an application of the central limit theorem! In Example 9.2, , , and on the null hypothesis of a right-tailed test. The central limit theorem says that if is true then we can expect the sample means, to be distributed as shown in the top part of Figure 9.1. Setting means that if the actual sample mean, ends up in the tail of the expected (under ) distribution of sample means then we consider that either we picked an unlucky 5 sample or the null hypothesis, , is not true. In taking that second option, rejecting , we are willing to live with the 0.05 probability that we made a wrong choice — that we made a type I error.

Figure 9.1: Derivation of the test statistic.

Referring to Figure 9.1 again, on the lower picture defines the critical region of area (in this case). It corresponds to a value on the upper picture which also defines a critical region of area . So comparing to on the original distribution of sample means, as given by the sampling theory of the central limit theorem, is equivalent, after -transformation, to comparing with . That is, is the -transform of the data value , exactly as given by Equation (9.1).

One-tailed tests

From a frequentist point of view, a one-tailed test is a a bit of a cheat. You use a one-tailed test when you know for sure that your test value or statistic is greater than (or less than) the null hypothesis value. That is, for the case of means here, you know for sure that the mean of the population, if it is different from the null hypothesis mean, if greater than (or less than) the null hypothesis mean. In other words, you need some a priori information (a Bayesian concept) before you do the formal hypothesis test.

In the examples that we will work through in this course, we will consider one-tailed tests when they make logical sense and will not require formal a priori information to justify the selection of a one-tailed test. For a one-tail test to make logical sense, the alternate hypothesis, , must be true on the face value of the data. That is, if we substitute the value of for into the statement of (for the test of means) then it should be a true statement. Otherwise, is blatantly false and there is no need to do any statistical testing. In any statistical test, must be true at face value and we do the test to see if is statistically true. Another way tho think about this is to think of as a fuzzy number. As a sharp number a statement like “ k" title="Rendered by QuickLaTeX.com" height="13" width="43" style="vertical-align: 0px;">” may be true, but is fuzzy because of (think to get the fuzzy number idea). So “ k" title="Rendered by QuickLaTeX.com" height="13" width="43" style="vertical-align: 0px;">” may not be true when is considered to be a fuzzy number[1]

When we make our decision (step 4) we consider the equality part of the statement in one-tailed tests. This equality is the strict under all circumstances but we use or is statements simply because they are the logical opposite of or " title="Rendered by QuickLaTeX.com" height="11" width="12" style="vertical-align: 0px;"> in the statements. So people may have an issue with this statement of but we will keep it because of the logical completeness of the , pair and the fact that hypothesis testing is about choosing between two well-defined alternatives.

p-Value

The critical statistic defines an area, a probability, that is the maximum probability that we are willing to live with for making a type I error of incorrectly rejecting . The test statistic also defines an analogous area, called or the -value or (by SPSS especially) the significance. The -value represents the best guess from the data that you will make a type I error if you reject . Computer programs compute -values using CDFs. So when you use a computer (like SPSS) you don’t need (or usually have) the critical statistic and you will make your decision (step 4) using the -value associated with the test statistic according to the rule:

\alpha \mbox{ do not reject } H_{0}.\]" title="Rendered by QuickLaTeX.com">

The method of comparing test and critical statistics is the traditional approach, popular before computers because is is less work to compute the two statistics than it is to compute . When we work problem by hand we will use the traditional approach. When we use SPSS we will look at the -value to make our decision. To connect the two approaches pedagogically we will estimate the -value by hand for a while.

Example 9.3 : Compute the -value for of Example 9.2.

Solution : This calculation can happen as soon as you have the test statistic in step 3. The first thing to do is to sketch a picture of the -value so that you know what you are doing, see Figure 9.2.

Figure 9.2 : The -value associated with in a one-tail test.

Using the Standard Normal Distribution Table to find the tail area associated with , we compute :

That is . Since (\alpha = 0.05)" title="Rendered by QuickLaTeX.com" height="18" width="198" style="vertical-align: -4px;">, we do not reject in our decision step (step 4).

▢

When using the Standard Normal Distribution Table to find -values for a given you compute).

- For two-tailed tests: . See Figure 9.3.

- For one-tailed tests: (as in Example 9.3)[2].

Don’t try to remember these formula, draw a picture to see what the situation is.

Figure 9.3 : The -value associated with a two-tailed . Since is defined by, , is defined by .

9.2.1 What p-value is significant?

By culture, psychologists use to define the decision point for when to reject . In that case, if then it means that the data (the test statistic) indicates there is less than a 5% chance that the result is a statistical fluke; that there is less than a 5% chance that the decision is a Type I error. So, in this course, we assume that unless is otherwise given explicitly for pedagogical purposes. The choice of is actually fairly lax and has led to the inability to reproduce psychological experiments in many cases (about 5% of course). The standards in other scientific disciplines can be different. In particle physics experiments, for example, is referred to as “evidence” for a discovery and they must have before an actual discovery, like the discovery of the Higgs boson, is announced. With z test statistics, represents the area in the tails of the distribution 3 standard deviations, or , from the mean. The value represents tail area , from the mean. So you may hear physicists saying that they have “5 sigma” evidence when they announce a discovery.

1. Fuzzy numbers can be treated rigorously in a mathematical sense. See, e.g. Kaufmann A, Gupta MM, Introduction to fuzzy arithmetic: theory and applications, Van Nostrand Reinhold Co., 1991. ↵

2. Of course substitute in the formula for a left tail test. ↵

9.3 t-Test for Means

Hypothesis testing for means for sample set sizes in where is used as an estimate for is the same as for except that and not is the test statistic[1]. Specifically, the test statistic is

for from any of the hypotheses listed in the table you saw in the previous section (one- and two-tailed versions):

Two-Tailed Test

Right-Tailed Test

Left-Tailed Test

:

:

:

:

:

:

The critical statistic is found in thet Distribution Table with the degrees of freedom .

Example 9.4 : A physician claims that joggers, maximal volume oxygen uptake is greater than the average of all adults. A sample of 15 joggers has a mean of 40.6 ml/kg and a standard deviation of 6 ml/kg. If the average of all adults is 36.7 ml/kg, is there enough evidence to support the claim at ?

1. Hypothesis.

(claim)

2. Critical statistic.

In thet Distribution Table, find the column for one-tailed test at and the line for degrees of freedom . With that find

3. Test statistic.

To compute this we need : , and from the problem statement. From the hypothesis we have . So

At this point we can estimate the -value using thet Distribution Table, which doesn’t have as much information about the -distribution as the Standard Normal Distribution Table has about the -distribution, so we can only estimate. The procedure is: In the row, look for values that bracket . They are 2.145 (with in the column heading for one-tailed tests) and 2.624 (associated with a one-tail ).

So,

is our estimate[2] for .

4. Decision.

Reject . We can also base this decision on our -value estimate since :

5. Interpretation.

There is enough evidence to support the claim that the joggers’ maximal volume oxygen uptake is greater than 36.7 ml/kg using a -test at .

▢

Fine point. When we use in a (or test) as an estimate for , we are actually assuming that distribution of sample means is normal. The central limit theorem tells us that the distribution of sample means is approximately normal so generally we don’t worry about this restriction. If the population is normal then the distribution of sample means will be exactly normal. Some stats texts state that we need to assume that the population is normal for a -test to be valid. However, the central limit theorem’s conclusion guarantees that the -test is robust to violations of that assumption. If the population has a very wild distribution then may be bad estimate for because the distribution of sample values will not follow the distribution. The chance if this happening becomes smaller the larger the , again by the central limit theorem.

Origin of the -distribution

We can easily define the -distribution via random variables associated with the following stochastic processes. Let :

Then the random variable

is a random variable that follows a -distribution with degrees of freedom.

1. Again, SPSS applies the -test, uses directly, for any sample size. ↵

2. If you know how to interpolate then you can find a single value for . ↵

9.4 z-Test for Proportions

The possible hypothesis pairs are :

Two-tailed Test

Right-tailed Test

Left-tailed Test

The steps in hypothesis testing for proportions are the same as hypothesis testing for means. Even the generic test statistic formula is the similar :

but now the observed and expected values are proportions, and respectively. The standard error in this case is

Using this information with the generic form, which mimics a test statistic, the proportions test statistic is

where is the number which appears in the hypothesis statement (see table above). This test statistic is valid only if and (so that the normal distribution provides a good approximation for the relevant binomial distribution). But, even though the test statistic can be moulded into the generic form, the proportions test statistic comes from the sampling theory given by the binomial distributions and not from any distribution that has a standard error {\em per se}. The normal distribution with and (remember those binomial distribution formulae?) -transformed to a -distribution with mean 0 and standard deviation 1 gives the test statistic formula. See the discussion in Section 8.4.

Example 9.5 : An attorney claims that more than 25 of all lawyers advertise. A sample of 200 lawyers in a certain city showed that 63 had used some form of advertising. At , is there enough evidence to support the attorney’s claim?

Solution :

1. Hypotheses.

,    (claim)

2. Critical statistic.

Using thet Distribution Table (last line) for a one tailed test at we find

3. Test statistic.

So using

find

We can also find the value along with the critical statistic. (See the picture for the next step.) Use the Standard Normal Distribution Table to find

4. Decision.

Refer to the diagram in Figure 9.4. It shows in the rejection region. So we reject .

Figure 9.4 : The null hypothesis situation for Example 9.5

We come, of course, to the same decision by considering the -value :

5. Interpretation.

There is enough evidence, using a -test at , to support the claim that more than of the lawyers use some form of advertising.

▢

9.5 Chi Squared Test for Variance or Standard Deviation

The possible hypothesis pairs are, for variance :

Two-tailed Test

Right-tailed Test

Left-tailed Test

For standard deviation we use the square roots of everything :

Two-tailed Test

Right-tailed Test

Left-tailed Test

Note that we did not square root . This is because we are using to stand in for whatever number. That number from will appear in our formulae as either or depending on the set up. Generally we will work with variance as we work through the problem and convert to standard deviation only in the last interpretation step if required by the wording of the question.

The new test statistic is :

where comes from the sample and comes from the number in . The degrees of freedom associated with the test statistic (for finding the critical statistic) is . There is no mystery where this test statistic came from — this is just how as a probability distribution is defined. So, for this test to be valid, the population must be normally distributed. The test here is not very robust to violations of that assumption because there is no normalizing intermediate central limit theorem here.

The critical regions on the distribution will appear as shown in Figure 9.5.

Figure 9.5 : Schematics of the critical regions for tests of variance. In the two-tailed situation the tail areas are equal.

Let’s work through an example of each hypotheses pair case. In all of the examples we assume that the population is normally distributed.

Example 9.6 : An instructor wishes to see whether the variance in scores of the 23 students in her class is less than the variance of the population. The variance of the class is 198. is there enough evidence to support the claim that the variation of the students is less than the population variance at ?

Solution :

1. Hypotheses.

2. Critical statistic.

Refer to Figure 9.6 as we get the critical statistic from the Chi-squared Distribution Table. As we see in that figure, we must look in the column that corresponds to a right tail area of 0.95. The row we need is for . With that information we find .

Figure 9.6 : Schematics of the critical regions for tests of variance. In the two-tailed situation the tail areas are equal.

3. Test statistic.

The values we need for the test statistic are (from ), and from the information in the problem. So :

At this point we can also estimate the value from the Chi-squared Distribution Table. The value is the area under the distribution with to the left of . In the row of the Chi-squared Distribution Table (in general use the closest if your particular value is not in the Chi-squared Distribution Table) hunt down the test statistic value of 19.38. You won’t find it but you can bracket it with values higher and lower than 19.38. Those numbers are 14.042 which has a right tail area of 0.90 (and so a left tail area of 0.10) and 30.813 which has a right tail area of 0.10 (and so a left tail area of 0.90). Recall that the in the column headings of the Chi-squared Distribution Table refers to right tail areas. So, considering the left tail areas we know that since for the relevant values.

4. Decision.

Since doesn’t fall in the rejection region, do not reject . We come to the same conclusion with our -value estimate:

5. Interpretation.

There is not enough evidence, at with a test, to support the claim that the variation in test scores of the class is less than 225.

▢

Example 9.7 : A hospital administrator believes that the standard deviation of the number of people using out-patient surgery per day is greater than eight. A random sample of 15 days is selected. The data are shown below. At is there enough evidence to support the administrator’s claim?

Solution :

0. Data reduction.

We’ll introduce a step 0 when it looks like we should do some preliminary calculations with or data. In this case we should enter the dataset into our calculations and determine . We find .

1. Hypotheses.

Note conversion to right away.

2. Critical statistic.

In the line and column of the Chi-squared Distribution Table, look up

3. Test statistic.

To estimate the value, find the bracketing values of in the line of the Chi-squared Distribution Table. They are : 26.119 () and 29.141 (), so .

4. Decision.

Reject since is in the rejection region. Our estimate of leads to the same conclusion :

5. Interpretation.

There is enough evidence, at with a test, to support the claim that the standard deviation is greater than 8. (Note how we convert to a statement about standard deviation after working through the problem using variances.)

▢

Example 9.8 : A cigarette manufacturer wishes to test the claim that the variance of the nicotine content of its cigarettes is 0.644. Nicotine content is measured in milligrams, assume that it is normally distributed. A sample of 20 cigarettes has a standard deviation of 1.00 kg. At , is there enough evidence to reject the manufacturer’s claim?

Solution :

1. Hypotheses.

2. Critical statistic.

Figure 9.7 : Critical regions for a two tailed test.

Referring to Figure 9.7, we see that we need two values, one with a tail area of 0.025 and the other with a tail area of 1 – 0.025 = 0.975. From the Chi-squared Distribution Table in the line find from the column and from the column.

3. Test statistic.

To estimate the value find the bracketing value of in the row, They are 27.204 () and 30.144 (). The are right tail areas, which is ok, but we need to multiply them by 2 because those right tail areas represent as shown in Figure 9.8. So .

Figure 9.8 : Areas for associated with the test statistic (29.50 here) in a two tail test.

4. Decision.

Do not reject . The estimate value leads to the same conclusion :

5. Interpretation.

There is not enough evidence, at with a test, to reject the manufacturer’s claim that the variance of the nicotine content of the cigarettes is equal to 0.644.

Notice, with the claim on , that failing to reject does not provide any evidence that is true. We just have the weaker conclusion that we couldn’t disprove it. Such is the double negative nature of the logic behind hypothesis testing that arises where we don’t assign probabilities to hypothesis.

▢

9.6 SPSS Lesson 5: Single Sample t-Test

Open “HeadCircum.sav” from the textbook Data Sets:

SPSS screenshot © International Business Machines Corporation.

Look at how simple it is! One variable. This is our single sample. Let’s do a -test for the hypotheses:

(9.2)

where we have used as the potentially inferred population value. Selecting the value for is something that you will need to think about when doing single sample -tests. Some possibilities are: past values, data range midpoints or chance level values. To run the -test in SPSS, pick Analyze Compare Means One-Sample T Test:

SPSS screenshot © International Business Machines Corporation.

The pop up menu is:

SPSS screenshot © International Business Machines Corporation.

where we have moved our variable into the Test Variable(s) box. If more than one variable is in this box then a separate -test will be run for each variable. The value has been entered into the Test Value box. That’s how SPSS knows that the hypotheses to test is that of the statement (9.2) above. If you open the Options menus, you will have a chance to specify the associated confidence interval. Running the analysis gives the very simple output:

SPSS screenshot © International Business Machines Corporation.

The output is simple but it requires your knowledge of the -test to interpret. As you get more experience with using SPSS, or any canned statistical software, you will get into the habit of looking for the -value. In SPSS it is in the Sig. (for Significance) column. Here , which is less than , so we reject the null hypothesis and conclude that there is evidence that the population mean is not 34.5. Note that this -value is for a two-tailed test. What if you wanted to do a one-tailed test? Well, then you have to think because SPSS won’t do that for you explicitly. For a one-tailed test, , half that of the two-tailed test. Remember that the two-tailed has two tails, each with an area of 0.016 as defined by , so getting rid of one of those areas gives the for the one- tailed test. Another way to remember to divide the two-tailed by 2 to get the one-tailed value is to remember that people try to go for a one-tailed test when they can because it has more power — it is easier to reject the null hypothesis with a one-tailed test meaning the -value will be smaller for a one-tailed test.

Let’s look at the rest of the output. There is a lot of redundant information there. You can use that redundant information to check to make sure you know what SPSS is doing and I can use that redundant information to see if you understand what SPSS is doing by reducing the redundancy and asking you to calculate the missing pieces. In the first output table, “One-Sample Statistics” is the information that you would get out of your calculator. The first three columns are , and . The last column is .

In the second output table “One-Sample Test”, notice that the test value of 33.8 is printed to remind you what the hypotheses being tested is. Te columns give: , , and . Notice that the first column, is the fourth column divided by the last column of the first table, . The last two columns give the 95% confidence interval

(9.3)

Note that zero is not in this confidence interval which is consistent with rejecting the null hypothesis. Simply add to Equation (9.3) to get the form we go for when we do confidence intervals by hand:

(9.4)

You can use the output here to compute a further quantity, known as standardized effect size. You’ll get a little practice with doing that in the assignments. The standardized effect size, , is a purely descriptive statistic (although it can be used in power calculations) and is defined by

(9.5)

where, by we mean . Being a descriptive statistic, people use the following rule of thumb to describe . If is approximately 0.2 then is considered “small”; if is approximately 0.5 then is considered “medium”; is approximately 0.8 then is considered “large”.

For the presentation of data graphically in reports and papers, an error bar plot is frequently used. To get such a plot for the data here, select Graphs Legacy Dialogs Error Bar:

SPSS screenshot © International Business Machines Corporation.

Choose Simple and “Summaries of separate variables”:

SPSS screenshot © International Business Machines Corporation.

and hit Define. Then set up the menu as follows:

SPSS screenshot © International Business Machines Corporation.

noting that we have chosen “Bars Represent” as “Standard error of the mean” so that the error bars will be :

SPSS screenshot © International Business Machines Corporation.

With an error bar plot like this, you can intuitively check the meaning of rejecting from the formal -test. Here the error bars do not include the value of 33.80 which is consistent with the conclusion that we reject 33.80 as a possible value for the population mean. We can see this more directly, and exactly, if we choose the value 95 confidence interval in the Bars Represent pull down of the plot menu.

This is a plot of Equation (9.4). The value is not in the 95% confidence interval.

Finally, selecting Graphs Legacy Dialogs Boxplot gives a EDA type of data presentation:

SPSS screenshot © International Business Machines Corporation.

9.7 RStudio Lesson 5: Single Sample t-Test

Osama Bataineh

Let’s insert and attach the dataset “HeadCircum.sav” (from Data Sets). To get the descriptive statistics, first we need to install and load the package pastecs. I have already installed it before. So I just loaded this package using the library function. Then we utilize a function of this package round to get the descriptive statistics. Here we use our variable age as the argument and set digits equal to 2 to make it look nice.

> library(pastecs)
Warning message:
package ‘pastecs’ was built under R version 3.4.4
> descriptive_stat <- round(stat.desc(age), digits = 2)
> descriptive_stat
x
nbr.val 5534.00
nbr.null 0.00
nbr.na 0.00
min 10.00
max 43.00
range 33.00
sum 129718.00
median 23.00
mean 23.44
SE.mean 0.06
CI.mean.0.95 0.12
var 22.29
std.dev 4.72
coef.var 0.20

Now we will be doing a T-test for mean equal to 33.80. To do it, we will take the help of the function t.test. Here we can see that in the argument, we have to mention the value of our null hypotheses (33.80 in the case), the type of T-test we are doing (one sided or two sided) and the confidence interval.

Now we will be doing a T-test for mean equal to 23.30. To do it, we will take the help of the function t.test. Here we can see that in the argument, we have to mention the value of our null hypotheses (23.30 in the case), the type of T-test we are doing (one sided or two sided) and the confidence interval.

> t.test(age, mu=23.30, alternative=”two.sided”, conf.level=0.95)

One Sample t-test

data:  age

t = 2.2088, df = 5533, p-value = 0.02723

alternative hypothesis: true mean is not equal to 23.3

95 percent confidence interval:

23.31577 23.56461

sample estimates:

mean of x

23.44019

> t.test(age, mu=23.30, alternative=”greater”, conf.level=0.95)

One Sample t-test

data:  age

t = 2.2088, df = 5533, p-value = 0.01361

alternative hypothesis: true mean is greater than 23.3

95 percent confidence interval:

23.33578      Inf

sample estimates:

mean of x

23.44019

There are two types of two-sample t-tests. (The test we covered in Chapter 9 that compared the mean of one sample to a fixed number is known as a one-sample -test.) These tests are:

Unpaired or independent sample -test:

The two populations are “independent”. There is no relation between the and variables (as we’ll call them). This is a “between subjects” test, the experimental subjects in each of the two populations are different.

Paired or dependent sample -test:

There is a natural pairing between the two variables and , usually they are measured from the same subject. A paired -test is an example of a “repeated measures” or “within subject” test.

We will introduce the independent sample -test with a -test approximation first to build ideas. As before, note that SPSS doesn’t do these approximate -tests. It does -tests even for large samples.

10.1 Unpaired z-Test

We have two populations and two sample sets, one from each population :

Sample Mean

Sample std. dev.

From population 1

From population 2

The population means are and and just as with the single population test, there are 3 possible hypothesis tests :

Two Tailed

Right Tailed

Left Tailed

or

or

or

= 0

In the second row the hypotheses are written in terms of a difference. Irrespective of which way you write the hypotheses, give population 1 priority. Write population 1 first. That way you won’t mess up your signs or your interpretation.

The test statistic to use, in all cases[1] is

(10.1)

where = sample set size from population 1 and = sample set size from population 2. This test statistic is based on a distribution of sample means as shown in Figure 10.1.

Figure 10.1 : The distribution of the difference of sample means under the null hypothesis . A one-tail example is shown here. The test statistic of Equation 10.1 follows from a -transformation of this picture.

Example 10.1 : A researcher hypothesizes that the average number of sports colleges offer for males is greater than the average number of sports offered for females. Samples of the number of sports offered to each sex by randomly selected colleges is given here :

Males (pop. 1)

Females (pop. 2)

At is there enough evidence to support the claim?

Solution :

1. Hypotheses.

Note that () so is true on the face of it. If is not true on the face of it then is just plain false without the need for any statistical test. With the hypotheses direction set correctly, the question becomes: Is significantly greater than ? The term “statistically significant” corresponds to “reject “.

2. Critical statistic.

From the t Distribution Table, one-tailed test at we find

Note that is positive because this is a right-tailed test. For left tailed tests make negative. For two-tailed tests you have .

3. Test statistic.

Using the Standard Normal Distribution Table, we can find the -value. Since , .

4. Decision.

Do not reject since is not in the rejection region. The -value reflects this :

5. Interpretation.

There is not enough evidence, at under a -test, to support the claim that colleges offer more sports for males than females.

▢

1. You could specify a non-zero null hypothesis, e.g. , in which case you would have . We won't consider that case in this course. ↵

10.2 Confidence Interval for Difference of Means (Large Samples)

Swapping the roles of sample and population in the sampling theory, we have the confidence interval corresponding to the hypothesis test of Section 10.1

where

Example 10.2 : Find the 95 confidence interval for the difference between the means for the data of Example 10.1.

Solution : First, recall our data :

,  ,

.  ,

From the t Distribution Table, look up the for the 95 confidence interval: . Then compute:

and

so

or

with 95 confidence. Notice that it is also correct to write with 95 confidence.

▢

This is a good point to make an important observation. A two-tailed hypothesis test at a given is complementary to a confidence interval of in the sense that if 0 is in the confidence interval then the complementary hypothesis test will not reject .

Let’s illustrate this principle with a one-sample -test under . (We need for this principle to work.) Look at the two possible outcomes :

Case 1 : 0 in the confidence interval, fail to reject . In the hypothesis test you would find :

In the confidence interval calculation you would find:

Putting the two pictures together gives:

See, is in the confidence interval if ¯ is not in the rejection region. The red distribution that defines the confidence interval is just the blue (identical) distribution slid over from to . The distance is the same because .

Case 2 : 0 not in the confidence interval, reject . In this case the combined picture looks like:

Before we can consider the independent sample -test, we need a tool for checking what the variances of the populations are. The formula for the test statistic will depend on whether the two variances are the same or not. So let’s take a look at comparing population variances.

10.3 Difference between Two Variances - the F Distributions

Here we have to assume that the two populations (as opposed to sample mean distributions) have a distribution that is almost normal as shown in Figure 10.2.

Figure 10.2: Two normal populations lead to two distributions that represent distributions of sample variances. The distribution results when you build up a distribution of the ratio of the two sample values.

The ratio follows an -distribution if . That distribution has two degrees of freedom: one for the numerator (d.f.N. or ) and one for the denominator (d.f.D. or ). So we denote the distribution more specifically as . For the case of Figure 10.2, and . The ratio, in general is the result of the following stochastic process. Let be random variable produced by a stochastic process with a distribution and let be random variable produced by a stochastic process with a distribution. Then the random variable will, by definition, have a distribution.

The exact shape of the distribution depends on the choice of and , But it roughly looks like a distribution as shown in Figure 10.3.

Figure 10.3: A generic distribution.

and are related :

so the statistic can be viewed as a special case of the statistic.

For comparing variances, we are interested in the follow hypotheses pairs :

Right-tailed

Left-tailed

Two-tailed

We’ll always compare variances () and not standard deviations () to keep life simple.

The test statistic is

where (for finding the critical statistic), and .

Note that when , a fact you can use to get a feel for the meaning of this test statistic.

Values for the various critical values are given in the F Distribution Table in the Appendix. We will denote a critical value of with the notation :

Where:

= Type I error rate
= d.f.N.
= d.f.D.

The F Distribution Table gives critical values for small right tail areas only. This means that they are useless for a left-tailed test. But that does not mean we cannot do a left-tail test. A left-tail test is easily converted into a right tail test by switching the assignments of populations 1 and 2. To get the assignments correct in the first place then, always define populations 1 and 2 so that . Assign population 1 so that it has the largest sample variance. Do this even for a two-tail test because we will have no idea what on the left side of the distribution is.

Example 10.3 : Given the following data for smokers and non-smokers (maybe its about some sort of disease occurrence, who cares, let’s focus on dealing with the numbers), test if the population variances are equal or not at .

Smokers

Nonsmokers

Note that so we’re good to go.

Solution :

1. Hypothesis.

2. Critical statistic.

Use the F Distribution Table; it is a bunch of tables labeled by “” that we will designate at , the table values that signify right tail areas. Since this is a two-tail test, we need . Next we need the degrees of freedom:

So the critical statistic is

3. Test statistic.

With this test statistic, we can estimate the -value using the F Distribution Table. To find , look up all the numbers with d.f.N = 25 and d.f.N = 17 (24 17 are the closest in the tables so use those) in all the the F Distribution Table and form your own table. For each column in your table record and the value corresponding to the degrees of freedom of interest. Again, corresponds to for a two-tailed test. So make a row above the row with . (For a one-tailed test, we would put .)

0.20      0.10      0.05      0.02      0.01
0.10       0.05     0.025    0.01      0.005

1.84       2.19      2.56       3.08      3.51         3.6 is over here somewhere so

Notice how we put an upper limit on because was larger than all the values in our little table.

Let’s take a graphical look at why we use in the little table and for finding for two tailed tests :

But in a two-tailed test we want split on both sides:

4. Decision.

Reject . The -value estimate supports this :

5. Interpretation.

There is enough evidence to conclude, at with an -test, that the variance of the smoker population is different from the non-smoker population.

▢

10.4 Unpaired or Independent Sample t-Test

In comparing the variances of two populations we have one of two situations :

1. Homoscedasticity :

2. Heteroscedasticity :

These terms also apply when there are more than 2 populations. They either all have the same variance, or not. This affects how we do an independent sample -test because we have two cases :

1. Variances of the two populations assumed unequal. .

Then the test statistic is :

This is the same formula as we used for the -test. To find the critical statistic we will use, when solving problems by hand, degrees of freedom

(10.2)

This choice is a conservative approach (harder to reject ). SPSS uses a more accurate

(10.3)

You will not need to use Equation (10.3), only Equation (10.2). Equation (10.3) gives fractional degrees of freedom. The test statistic for this case and the degrees of freedom in Equation (10.3) is know as the Satterwaite approximation. The -distributions are strictly only applicable if . The Satterwaite approximation is an adjustment to make the -distributions fit this case.

2. Variances of the two populations assumed equal. .

In this case the test statistic is:

This test statistic formula can be made more intuitive by defining

(10.4)

as the pooled estimate of the variance. is the data estimate for the common population . is the weighted mean of the sample variances and . Recall the generic weighted mean formula, Equation (3.2). The weights are and ; their sum is . In other words

and we can write the test statistic as

(10.5)

See that is clearly a standard error of the mean.

10.4.1 General form of the t test statistic

All statistics have the form :

Remember that! Memorizing complicated formulae is useless, but you should remember the basic form of a test statistic.

10.4.2 Two step procedure for the independent samples t test

We will use the test to decide whether to use case 1 or 2. SPSS uses a test called “Levine’s test” instead of the test we developed to test .  Levine’s test also produces an test statistic. It is a different than our but you interpret it in the same way. If the -value of the is high (larger than ) then assume , if the -value is low (smaller than ) then assume .

In real life, homoscedasticity is almost always assumed because the -test is robust to violations of homoscedasticity until one sample set contains twice as many, or more, data points as the other.

Example 10.4: Case 1 example.

Given the following data summary :

(Note that (s_{2}=12)" title="Rendered by QuickLaTeX.com" height="18" width="164" style="vertical-align: -4px;">. If that wasn’t true, we could reverse the definitions of populations 1 and 2 so that 1" title="Rendered by QuickLaTeX.com" height="15" width="66" style="vertical-align: -3px;">.) Is significantly different from ? That is, is different from ? Test at .

Solution :

So the question is to decide between

a two-tailed test. But before we can test the question, we have to decide which test statistic to use: case 1 or 2. So we need to do two hypotheses tests in a row. The first one to decide which statistic to use, the second one to test the hypotheses of interest given above.

Test 1 : See if variances can be assumed equal or not.

1. Hypothesis.

(Always use a two-tailed hypothesis when using the test to decide between case 1 and 2 for the test statistic.)

2. Critical statistic.

(from the F Distribution Table)

(Here we used given for the -test question. But that is not necessary. You can use in general; the consequence of a type I error here is small because the -test is robust to violations of the assumption of homoscedasticity.)

3. Test statistic.

4. Decision.

4.20" title="Rendered by QuickLaTeX.com" height="13" width="95" style="vertical-align: -1px;"> ( F_{\rm crit}" title="Rendered by QuickLaTeX.com" height="16" width="89" style="vertical-align: -4px;"> — drawing a picture would be a safe thing to do here as usual) so reject .

5. Interpretation.

Assume the variances are unequal, , and use the test statistic of case 1.

Test 2 : The question of interest.

1. Hypothesis.

2. Critical statistic.

From the t Distribution Table, with , and a two-tailed test with we find

3. Test Statistic.

The -value may be estimated from the t Distribution Table using the procedure given in Chapter 9: from the t Distribution Table, line, find the values that bracket 0.57. There are none, the smallest value is 0.711 corresponding to . So all we can say is 0.50" title="Rendered by QuickLaTeX.com" height="17" width="66" style="vertical-align: -4px;">.

4. Decision.

is not in the rejection region so do not reject . The estimate for the -value confirms this decision.

5. Interpretation.

There is not enough evidence, at with the independent sample -test, to conclude that the means of the populations are different.

▢

Example 10.5 (Case 2 example) :

The following data seem to show that private nurses earn more than government nurses :

Private Nurses Salary

Government Nurses Salary

Testing at , do private nurses earn more than government nurses?

Solution :

First confirm, or change, the population definitions so that s_{2}^{2}" title="Rendered by QuickLaTeX.com" height="21" width="55" style="vertical-align: -6px;">. This is already true so we are good to go.

Test 1 : See if variances can be assumed equal or not. This is a test of vs. . After the test we find that we believe that at . So we will use the case 2, equal variances, -test formula for test 2, the test of interest.

Test 2 : The question of interest.

1. Hypothesis.

\mu_{2}\]" title="Rendered by QuickLaTeX.com">

(Note how reflects the face value of the data, that private nurses appear to earn more than government nurses in the population — it is true in the samples.)

2. Critical statistic.

Use the t Distribution Table, one-tailed test, (column) and to find

3. Test statistic.

To estimate the -value, look at the line in the t Distribution Table to see if there are a pair of numbers that bracket . They are all smaller than 5.47 so is less than the associated with the largest number 2.921 whose is 0.005 (one-tailed, remember). So .

4. Decision.

Reject since is in the rejection region and .

t_{crit} \hspace{.25in} (5.47 > 2.583)\]" title="Rendered by QuickLaTeX.com">

5. Interpretation.

From a -test at , there is enough evidence to conclude that private nurses earn more than government nurses.

▢

10.5 Confidence Intervals for the Difference of Two Means

The form of the confidence interval is

but, as with hypothesis testing, we have two cases to choose from to get the formula for :

Case 1 : Variances of the 2 populations unequal}

where the degrees of freedom to use when looking up in the t Distribution Table is

Case 2 : Variances of the 2 populations equal

where we use

when looking up .

To select the appropriate formula for we need to do a preliminary hypothesis test on . An odd combination of hypothesis test followed by confidence interval calculation.

Insight! By now you should have noticed that the formulae for are just times standard error of the mean. This whole -transformation thing should be becoming somewhat transparent.

Example 10.6 : Find the 95 confidence interval for for the data of Example 10.4 :

Solution :

First use -test to see which formula to use.  We did this already in Example 10.4 (the data come from that question) and found that we believed with .

Next, look up in the t Distribution Table for 95 confidence interval for :

Compute

So

be careful of the order!

▢

10.6 SPSS Lesson 6: Independent Sample t-Test

To follow along, load in the Data Set titled “pHLevel.sav”:

SPSS screenshot © International Business Machines Corporation.

This is the first time we have an independent variable, Species in this case, and it has two values, setosa and versicolor, that label the two populations. Notice, especially, that we do not have separate columns for each sample. There is only one dependent variable, Sepal.Length in this case. As we cover the more advanced statistical tests in later chapters (part of PSY 234) the nature and complexity of the independent variable will evolve but we will always have just one independent variable.

Running the -test is easy, pick Analyze Compare Means Independent Samples T Test :

SPSS screenshot © International Business Machines Corporation.

Select Sepal.Length as the Test Variable (dependent variable) and Species as the group variable (independent variable) :

SPSS screenshot © International Business Machines Corporation.

You need to do some work to let SPSS know that the two levels of the “grouping variable” are 1 and 2 (as can be seen in the Variable View window). So hit Define Groups… and enter:

SPSS screenshot © International Business Machines Corporation.

Hit Continue, then OK (the Options menu will allow you to set the confidence level percent) to get:

SPSS screenshot © International Business Machines Corporation.

The first table shows descriptive statistics for the two groups independently. These numbers, excluding standard error numbers can be plugged into the formulae for pencil and paper calculations.

The important table is the second table. First, what hypothesis are we testing? It is important to write it out explicitly:

(10.6)

This, as you recall, is our test of interest. When we did this test by hand, we had to do a preliminary test the see if we could assume homoscedasticity or not :

(10.7)

That preliminary test is given to us as Levine’s test in the first two columns of the second table. Levine’s test is similar to but not exactly the same as the test we used but it also uses as a test statistic. Here we see with , so we reject and assume that population variances are unequal. That means we look at only the second line of the second table corresponding to “Equal variances not assumed”. SPSS computes and using both formulae but it does not decide for you which one is correct. You need to decide that yourself on the basis of the Levine’s test.

Again the information is fairly redundant. Looking across the second row we have (note that it is the same as the in the first row – that’s because the sample is large, making a good approximation for both), (notice the fractional here for the heteroscedastic case — recall Equation (10.3)), (note that it is for a two-tailed hypothesis, if your hypothesis is one-tailed then divide by 2), , and the standard error, the denominator of the test statistic formula ( is mean over standard error). The value is small, so we reject , the difference of the sample means is significant. The last two columns give the 95% confidence interval as

(10.8)

Notice that zero is not in the confidence interval, consistent with rejecting .

We can also make an error bar plot. Go through Graphs Legacy Dialogs Errorbar and pick Simple and “Summaries for groups of cases” in the next menu and:

SPSS screenshot © International Business Machines Corporation.

which results in:

SPSS screenshot © International Business Machines Corporation.

or you could generate a boxplot comparison:

SPSS screenshot © International Business Machines Corporation.

Finally, we throw in a couple of effect size (descriptive) measures. One is the standardized effect size defined as:

(10.9)

where is the pooled variance as given by Equation (10.4). Another measure is the strength of association

(10.10)

which measures a kind of “correlation'” between and . The larger , the closer is to 1.

10.7 RStudio Lesson 6: Independent Sample t-Test

[Coming soon]

10.8 Paired t-Test

Here two measurements and are taken from every subject. We could say that we measure a vector as the independent variable for every subject instead of just a number as the independent variable[1]. This is a within subject design. Within subject designs tend to be more statistically powerful than independent or between subjects designs that have two completely different bunches of people for each variable. The extra power comes because we take the difference for every subject. So any overall differences, or variances, in or due to individuals has been removed from the data.

The paired -test is a univariate test. The difference between univariate and multivariate statistics is the the independent variables are numbers for univariate statistics and vectors for multivariate statistics. For the paired -test, the vector is converted to a number by taking a difference. To convert vector data to difference data, make a table :

1

2

-1

2

3

-1

3

5

-2

1

-2

3

Note here that the differences in individuals are gone after we take differences .

The data from the column are what you will work with. Compute and the mean and sample standard deviation of these data. With the procedure becomes a single sample -test of against zero. Specifically we can test :

Two-tailed

Left-tailed

Right-tailed

The test statistic is

with (for finding ).

Example 10.7 : A Physical Education director claims that a vitamin will increase a weight lifter’s strength. Eight athletes are selected and tested on how much they can bench press. They are each tested once before taking the vitamin and again after taking the vitamin for two weeks. We want to test the director’s claim at

The data are :

Athlete

Before()

After()

1

210

219

-9

2

230

236

-6

3

182

179

3

4

205

204

1

5

262

270

-8

6

253

250

3

7

219

222

-3

8

216

216

0

Here we have listed the differences which is actually part of step 0 of the solution. The and columns are what you enter into SPSS as your independent variables. With SPSS you never see the differences.

Solution :

0. Data reduction.

Compute , by entering the difference data into your calculator.

1. Hypothesis.

Note that a negative difference, based on (always consistently give population 1 priority if you want to stay out of trouble without thinking), indicates an increase in strength. It is important to interpret positive or negative differences correctly by thinking about what they mean.

2. Critical statistic.

Using the t Distribution Table with the column for one-tailed tests, , and row , find

(We added the negative sign because this is a left-tailed test.)

3. Test statistic.

To estimate the -value, from the t Distribution Table, line, find .

4. Decision.

Do not reject . .

5. Interpretation.

Under a paired -test, at , the is not enough evidence to conclude that the vitamin increases strength.

▢

1. An introduction to vectors will be given in Chapter 17. ↵

10.9 Confidence Intervals for Paired t-Tests

The usual form applies :

where now

and can found from the t Distribution Table in the line using the “confidence intervals” heading.

10.10 SPSS Lesson 7: Paired Sample t-Test

To follow along, load in the Data Set “Methadone.sav”:

SPSS screenshot © International Business Machines Corporation.

As set up, the file has two dependent variables. This “within subjects” dataset is fundamentally multivariate. When we did the paired -test by hand we converted the multivariate data to univariate data by taking differences. SPSS will do the differences behind the scene and you won’t actually see them. Run the -test by picking Analyze Compare Means Paired -Samples T-Test:

SPSS screenshot © International Business Machines Corporation.

Move the two variables into Pair 1 and hit OK (Options again allows you to specify a confidence intervals percentage):

SPSS screenshot © International Business Machines Corporation.

The output is:

SPSS screenshot © International Business Machines Corporation.

The first two tables are descriptive statistics. The last table gives the stuff we want: , , the confidence interval

(10.11)

, and for the two-tailed hypotheses pair

(10.12)

The very low -value (0 in this case) and the absence of 0 in the confidence interval guide us to reject , the differences are significantly different from zero.

The standardized effect size and strength of association for the paired -test are

(10.13)

and

(10.14)

respectively.

10.11 RStudio Lesson 7: Paired Sample t-Test

[Coming soon]

In this Chapter we will use a test to compare proportions and extend what we do here with the -distribution.

11.1 z-Test for Comparing Proportions

11.2 Confidence Interval for the Difference between Two Proportions

12.1 One-way ANOVA

12.2 Post hoc Comparisons

12.3 SPSS Lesson 9: One-way ANOVA

12.4 R Lesson 9: One-way ANOVA

12.5 Two-way ANOVA

12.6 SPSS Lesson 9: Two-way ANOVA

12.7 R Lesson 9: Two-way ANOVA

12.8 Higher Factorial ANOVA

12.9 Between and Within Factors

12.10 *Contrasts

13.1 Power

14.1 Scatter Plots

14.2 Correlation

14.3 SPSS Lesson 10: Scatterplots and Correlation

14.4 R Lesson 10: Scatterplots and Correlation

14.5 Linear Regression

14.6 r-squared and the Standard Error of the Estimate of y-prime

14.7 Confidence Interval for y-prime at a Given x

14.8 SPSS Lesson 11: Linear Regression

14.9 R Lesson 11: Linear Regression

14.10 Multiple Regression

14.11 SPSS Lesson 12: Multiple Regression

14.12 R Lesson 12: Multiple Regression

15.1 Goodness of Fit

15.2 Contingency Tables

15.3 SPSS Lesson 13: Proportions, Goodness of Fit, and Contingency Tables

15.4 R Lesson 13: Proportions, Goodness of Fit, and Contingency Tables

16.1 How to Rank Data

16.2 Median Sign Test

16.3 Paired Sample Sign Test

16.4 Two Sample Wilcoxon Rank Sum Test (Mann-Whitney U Test)

16.5 Paired Wilcoxon Signed Rank Test

16.6 Kruskal-Wallis Test (H Test)

16.7 Spearman Rank Correlation Coefficient

16.8 SPSS Lesson 14: Non-parametric Tests

16.9 R Lesson 14: Non-parametric Tests

16.10 Runs Test

17.1 Linear Algebra Basics

17.2 The General Linear Model (GLM) for Univariate Statistics

1

Appendix: Tables

- Binomial Distribution Table (PDF) (Word)

- Standard Normal Distribution Table (PDF) (Word)

- t Distribution Table (PDF) (Word)

- Chi Squared Distribution Table (PDF) (Word)

- F Distribution Table (PDF) (Word)