Exercise 2: Reliability and Validity 

For this exercise, your task is to estimate the reliability and validity of a measure of Need for Cognition (nCog; Cacioppo & Petty, 1982; Cacioppo, Petty, & Kao, 1984).  

An SPSS data file is included in the assignment folder, with responses from 294 college students. Within the data file you will find raw scores for nCog items, as well as composite scores for several other variables (see page 2 for descriptions).  

Your tasks for this assignment are described below. To answer each of these questions, run the analyses using SPSS and report relevant information in your write-up (i.e., the reliability or validity coefficient). Write a report explaining your findings for each of the questions above. Provide an interpretation of what the results mean. In other words, explain your findings in sufficient detail so that another person could understand what you examined and why.  

Reliability  Estimate Coefficient Alpha for nCog using the items in the data set.   Estimate test-retest reliability for nCog – data were collected 2 weeks after the original nCog data (NCOG_T2). 

Criterion-Related Validity   Report the criterion-related validity estimate for nCog using College GPA (CollegeGPA).  Report the criterion-related validity estimate for nCog using Student Satisfaction (StudentSat).  Report the incremental validity estimate of nCog above and beyond High School GPA (HS_GPA) and ACT Scores (ACT) in predicting College GPA (CollegeGPA).   Report the incremental validity estimate of nCog above and beyond High School GPA (HS_GPA) and ACT Scores (ACT) in predicting Student Satisfaction (StudentSat).  

Construct-Related Validity   Report convergent validity evidence using Intrinsic Motivation (IntMot) and Mastery Goal Orientation (MGO).   Report discriminant validity evidence using Extrinsic Motivation (ExtMot), Performance Approach Goal Orientation (AppGO), Performance Avoid Goal Orientation (AvdGO), and Social Desirability (SocDes).    

Variables in Data Set/Assignment 

Need for Cognition –described as a drive to understand situations and the world around us (Cacioppo & Petty, 1982; Cacioppo et al., 1984). People with a high need for cognition tend to enjoy problem solving, abstract reasoning, and enjoy complex situations/problems. Items were rated on a 5-point scale (1 = Strongly Disagree to 5 = Strongly Agree), and higher scores indicate higher levels of the construct. Items were taken from Cacioppo et al. (1984), in the attached article.  Note that several of these items are reverse-scored. You will need to re-code them prior to running the analyses.  

High School GPA – is cumulative, high school grade point average, on a 4.0 scale. Scores were taken from admissions records.  

ACT Scores – ACT scores were gathered from university records.  

College GPA – is cumulative, college grade point average on a 4.0 scale. This was taken from university records. In this study College GPA was collected one year after Need for Cognition was measured. 

Student Satisfaction – is the extent that students are satisfied with their experiences at the university. Six items were written for the purpose of conducting this study. Cronbach’s alpha for this scale was .90. In this study student satisfaction was collected one year after Need for Cognition was measured.  

Academic Motivation focuses on the reasons that students go to college. This is based on Deci and Ryan’s (1985) self-determination theory (Ryan & Deci, 2000). For the purposes of this data collection ten items were written to measure intrinsic motivation and ten items were written to measure extrinsic motivation.   Intrinsic Motivation – is based on a drive to learn and accumulate knowledge. The process of learning itself is internally motivating for individuals with a high level of intrinsic motivation. Coefficient alpha for this scale was .92.  Extrinsic Motivation – is based on the anticipation of some other, external reward, Individuals with a high level of intrinsic motivation are less interested in learning itself, but motivated because with a college degree they are more likely to obtain a high-paying job or promotion at work. Coefficient alpha for this scale was .87. 

Goal Orientation – reflects the purpose, or type of goal that a person aims to achieve when pursuing a particular task (Elliot et al., 1999). This can be composed of mastery goal orientation, performance approach goal orientation, and performance avoid goal orientation. Seven items were written to measure each of the three dimensions below.   Mastery Goal Orientation – has a focus on improving oneself as a function of task pursuit, and is often associated with deep-level processing and drive to learn and understand material (e.g., expanding one’s knowledge). Coefficient alpha for this scale was .90.  Performance Approach Goal Orientation – is associated with achieving positive outcomes and successful task accomplishment. However, this is more often associated with surface-level processing, with a focus on simply obtaining task outcomes (e.g., getting a good grade). Coefficient alpha for this scale was .83.  Performance Avoid Goal Orientation – is also focused on achieving outcomes, but specifically on avoiding failure or negative outcomes. Instead of focusing on learning material or getting a favorable grade or expanding one’s knowledge, the focus remains on not failing a class, for example.   Coefficient alpha for this scale was .89. 

Social Desirability – involves providing what would be perceived as favorable, or socially desirable, responses (Crowne & Marlowe, 1960). The KR20 internal consistency estimate for this scale was .87.  
 

References 

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131.  

Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 3, 306-307.  

Crowne, D.P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349–354. 

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum. 

Elliott, A. J., McGregor, H. A., & Gable, S. (1999). Achievement goals, study strategies, and exam performance: A mediational analysis. Journal of Educational Psychology, 91, 549-563.  

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78.