Florian Kaǧan Meyer - Language Proficiency Testing for Chinese as a Foreign Language: An Argument-Based Approach for Validating the Hanyu Shuiping Kaoshi (HSK)

Language Testing
and Evaluation

Series editors: Rdiger Grotjahn
and Gnther Sigott

Volume 32

Florian Kaan Meyer

Language Proficiency
Testing for Chinese
as a Foreign Language

An Argument-Based Approach for Validating
the Hanyu Shuiping Kaoshi (HSK)

Bibliographic Information published by the Deutsche Nationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the internet at http://dnb.d-nb.de.

Zugl.: Bochum, Univ., Diss., 2012

Library of Congress Cataloging-in-Publication Data
Meyer, Florian Kagan, 1976
Language proficiency testing for Chinese as a foreign language : an argument-based approach for validating the Hanyu Shuiping Kaoshi (HSK) / Florian Kagan Meyer. Peter Lang Edition.
pages cm. (Language testing and evaluation ; Volume 32)
ISBN 978-3-631-64891-9
1. Chinese languageAbility testing. 2. Chinese languageAcquisition.
3. Chinese languageStudy and teachingForeign speakers. 4. Chinese languageSpoken ChineseExaminations. 5. Chinese languageExaminations, questions, etc. I. Title.
PL1119.M49 2014
495.180076dc23

2014005520

D 294
ISSN 1612-815X
ISBN 978-3-631-64891-9 (Print)
E-ISBN 978-3-653-03934-4 (E-Book)
DOI 10.3726/978-3-653-03934-4

Peter Lang GmbH
Internationaler Verlag der Wissenschaften
Frankfurt am Main 2014
All rights reserved.
Peter Lang Edition is an Imprint of Peter Lang GmbH.

Peter Lang Frankfurt am Main Bern Bruxelles New York Oxford Warszawa Wien

All parts of this publication are protected by copyright. Any utilisation outside the strict limits of the copyright law, without the permission of the publisher, is forbidden and liable to prosecution. This applies in particular to reproductions, translations, microfilming, and storage and processing in electronic retrieval systems.

This book is part of the Peter Lang Edition list and was peer reviewed prior to publication.

www.peterlang.com

About the author(s)/editor(s)

Florian Meyer studied Sinology, Communication Science and Korean at Free University Berlin, and Chinese at Peking University. He worked as a lecturer for Modern Chinese at Ruhr University Bochum (Germany), where he studied Language Teaching Research and completed his PhD.

About the book

How did the (old) Hanyu Shuiping Kaoshi (HSK) assess the Chinese proficiency of non-native speakers of Chinese? What inferences can be derived from HSK test taker scores, especially from Western test takers? How difficult is it to learn Chinese according to the HSK? Thirty years of research have been synthesized into an argument-based approach for validating the most widespread test for Chinese as a foreign language. In addition, the author has analyzed the scores of a sample of over 250 German test takers in order to investigate how many hours German natives needed on average to study for reaching a specific HSK level. This work also extensively discusses validation theory for psychological testing, and it demonstrates how to put an argument-based approach to validation into practice.

This eBook can be cited

This edition of the eBook can be cited. To enable this we have marked the start and end of a page. In cases where a word straddles a page break, the marker is placed inside the word at exactly the same position as in the physical book. This means that occasionally a word might be bifurcated by this marker.

| 5

Preface and Acknowledgements

In 2003, a German classmate and I listened to some HSK mock tests in my dorm at Peking University. Unfortunately, we did not comprehend the essential parts of the dialog we listened to, and not surprisingly, we could not answer the related items. Suddenly, my Mongolian roommate entered the room, and he burst out laughing when he listened to the tape. Obviously, we had missed something funny. My roommate explained what made him laugh:

A woman asks a guy if he also attended the concert last night. In fact, the guy was there as well and liked it very much. This irritates the woman because she thinks the concert was horrible, especially the last artist. It turns out that the singer is the wife of the guy. The woman embarrassedly says that the artist sang quite terrifically (with a voice like a bird), and the problem was only that she had just picked such an awful song, whereupon the man comments, So, you didnt like the song? I have spent half a year composing it. (cf. Essentials of HSK Listening, 2003 (2nd ed.), p. 53)

Afterwards, we listened to the tape again and understood at least the major parts. My classmate considered the whole dialog typical HSK crap and said that the test would be completely useless Some weeks later, I took the HSK and reached level 4 out of 8. Many other 5 | 6 Westerners achieved similar scores, and several complained about the HSK, e.g., that the test did not directly assess spoken language and that it was simply a multiple-choice test without any value for assessing the real Chinese proficiency. As a result, many did not take the HSK again. However, I noticed that my Chinese ability still had shortcomings, and I registered for the HSK again; in the following years I took the HSK several times, gradually improving my score. For me, the HSK provided a somehow fair measure of my Chinese language proficiency, and the test appeared useful to me.

Acknowledgements

First of all, I want to thank Andreas Guder and Erling von Mende for their support when I first started my project in October 2007 and when I conducted an initial survey among 35 HSK test takers in Hannover and Berlin. By May 2009, surveys had been conducted at all seven HSK test sites, in the end yielding 257 valid questionnaires. Several persons helped to carry out these surveys. Thus, I want to thank Heidi Brexendorff, Anja Warnecke-Bi, Kathrin Grnke, Li Xioyn , L Rng, H Chnchn , Julia Hauser, Carsten Krause, Nadine Reick and Dng Xiojng . I would also like to thank Peter Rhl, who programmed a webpage where test takers participated in the survey via the Internet.

I must express my deepest thanks to Rdiger Grotjahn and Henning Klter, who adopted my project after I came to Ruhr University Bochum. Rdiger Grotjahn gave me advice regarding validity theory and applying validation approaches. Without his mentoring, which started when I participated in his seminars about language testing and language assessment, research methodology, and statistics, this work would not have been possible. Henning Klter encouraged me to write this dissertation in English, and he helped me to submit it to the Faculty of East Asian Studies of the Ruhr University Bochum. I would like to thank Rene Beine and Mary Ann Poerner for proofreading and correcting my thesis. Finally, I thank Rdiger Grotjahn and Gnther Sigott for publishing this work in the series of Language Testing and Evaluation.

Florian Kaan Meyer
Beijing, January 2014

The whole dialog has been transcribed in Appendix A.

One can argue that (a) the situation in the dialog is not typical for the life of foreign university students in the P.R. of China and (b) the text was unauthentic because it is unclear if the woman knows the man or not. If she knew him (and the dialog sounded like this), why would she not know that he was a songwriter from the concert the day before?