APPENDIX A
Recommended Readings
Basant, K. P. (1996). Statistics in practice: An illustrated guide to SPSS. London: Arnold; Pallant, J. (2001). SPSS survival manual: A step by step guide to data analysis using SPSS for Windows (version 10). Buckingham: Open University Press. These are two books particularly useful as resources for students who are not familiar with SPSS. Both books are well organized and present material using accessible yet detailed descriptions and explanations.
Cronbach, L. J. (1990). Essentials of psychological testing (5th ed.). New York: HarperCollins. Though this classic text is unfortunately out of print, the reader may wish to invest the effort in obtaining a copy. We find Lee Cronbachs discussion of evaluating a tests criterion validity especially useful and eloquent. Needless to say, chapter 6 is obligatory reading for readers interested in learning more about the Cronbach alpha as a measure of test reliability.
Dawes, R. M., & Smith, T. L. (1985). Attitude and opinion measurement, in G. Lindzey and E. Aronson (Eds.), The handbook of social psychology (3rd ed., vol. 1, pp. 509-566). Hillsdale, NJ: Erlbaum. This is a new version of an influential article on attitude research that discusses attitude theory and a number of methodological considerations in attitude research.
Lewis-Beck, M. S. (1980). Applied regression: An introduction. Thousand Oaks, CA: Sage. This handy (and short) book provides one of the more accessible explanations on regression that we have come across. The intended audience is similar to that of our book: beginning social researchers, with limited quantitative expertise, who are interested in using regression and related forms of quantitative analyses.
Patten, M. L. (2000). Understanding research methods (2nd ed.). Los Angeles: Pyrczak. In Part D: Measurement, Mildred Patten does a fine job of explaining validity in its various forms. She also discusses the relationship between validity and reliability and ways to evaluate both. Readers may find the rest of her book a useful overview to research methods, particularly for those readers interested in a basic, yet accessible introduction.
Peterson, R. A. (1999). Constructing effective questionnaires. Thousand Oaks, CA: Sage. This book presents a more technical approach to instrument construction than the one we describe in this book, focusing specifically on questionnaires. However, for the reader who is ready to move on to greater levels of expertise, Petersons volume may provide a useful next step. The text includes chapters on open- and closed-ended questions and on rating scales, as well as on questionnaire structure and evaluation.
Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (Eds.). (1991). Measures of personality and social psychological attitudes. New York: Academic Press. This book contains an extensive and technical literature on scaling (beyond Likert, Thurstone, and Guttman), which might interest the mathematically advanced reader. The text focuses on attitude scales, some of which have benefited from considerable test construction research.
Schuman, H., & Presser, S. (1996). Questions and answers in attitude surveys. Thousand Oaks, CA: Sage; Krebs, D., & Schmidt, P. (Eds.). (1993). New directions in attitude measurement. Berlin: de Gruyter. These two books comprise yet another presentation on scales that is accessible to readers with some basic understanding of tests and test construction.
Stapleton, C. D. (1997). Basic concepts in exploratory factor analysis (EFA) as a tool to evaluate score validity: A right-brained approach. Available: ericae.net/ft/tamu/Efa.HTM (current as of 8/1/02). This paper presents one of the most accessible presentations of factor analysis, both exploratory and confirmatory, as pertaining to tests for research purposes and validity. Though we disagree with Stapletons position on the use of exploratory factor analysis for the evaluation of construct and criterial validity, we still think that readers may find her essay useful.
APPENDIX B
Instructions for Performing Necessary Statistical Procedures in SPSS
Recoding Data Entered in Noncriterial Direction
Before attempting any item analysis computations, it is often simpler to enter all responses as marked on test and then make the necessary changes for criterial directionality rather than the other way around. What follows are the steps required to ensure that the responses to all the items point in the appropriate direction.
- From the SPSS Data Editor window, use the Transform menu and choose Compute.
- Assign a name to the new column that will contain the corrected item scores (e.g., it1.9crit) in the Target variable box.
- In the Numeric Expression: box, enter a minus sign and a left parenthesis; do not leave a space in between.
- Select the item you wish to recode from the list of variables available in the scrollable window on the left and use the button to place it next to the open parenthesis.
- Immediately next to the item label (i.e., leave no spaces), enter a minus sign, followed by a whole number. Use the following decision rules to decide what this number is: If the values you have entered for other items range from 1 to whatever the highest response option is (e.g., 1-4, 16, or 18), enter a number that is higher by one (i.e., 5, 7, or 9, respectively). If, however, you chose to enter values ranging from 0 to the highest value, enter the highest value. Close the parentheses.
- The finished formula should resemble the following (where item1.9 is the variable name):
(item1.9-5)
If so, click OK.
- The new variable with the label you chose containing the items criterial values should have appeared at the right end of the spreadsheet. Remember to use this variable for future analyses. In fact, it may be a good idea to delete the original column and save the spreadsheet with all recoded item responses as a new file. (On SPSS Data Editor window, from File menu choose Save as.)
Calculating Mean Difficulty Scores for Each Item
Before obtaining mean item difficulty scores, all item scores must be coded in the criterial direction (see the prior section). Calculating mean scores is a rather simple procedure that may be performed either in SPSS or in other spreadsheet software (e.g., MS Excel). However, obtaining a histogram for mean scores in SPSS involves a rather complicated set of procedures. We recommend that readers construct the histogram on chart paper after obtaining the mean scores by following these steps:
- From the SPSS Data Editor window, use the Analyze menu to select Reports. From the options available, choose Case Summaries.
- Select all the items and use the button to move them to the Variables box.
- Click the Statistics button and, from the options available on the next window, choose Mean. Click Continue and OK.
- The results will be available on the SPSS Viewer window. If this window does not appear on its own, it is available from the Window menu. Once on this window, click on the Case Summaries icon on the navigation bar on the left. The mean scores will be at the bottom of the columns.
- To print the results, from the File menu, choose Print. It is also possible to save the results as a text file. From File menu, choose Save As....