1
Module 5119
Language Testing and Assessment

Module 5119
Language Testing and Assessment



WEEK 5

Testing Receptive Skills: Reading
Reading skills
Readers can take control of the input more easily
Readers determine the speed of the activity
Reading skills seen in terms of ‘bottom-up’ and ‘top-down’ skills
Bottom-up (systemic) skills: building up meaning from analyzing/decoding the form of the language used
(from words to clauses to sentences to paragraphs)
Top-down (schematic) skills: prior world knowledge (cultural or world knowledge) brought by the reader to the text
Types (genres) of reading
Academic reading
General interest articles (in magazines, newspapers)
Technical reports, professional journal articles
Reference materials (e.g., dictionaries)
Textbooks , theses
Essays, papers
Test directions
Editorials and opinion writing
Types (genres) of reading (cont.)
Job-related reading
Messages
Letters/emails
Memos
Reports (job evaluations, project reports)
Schedules, labels, signs, announcements
Forms, applications, questionnaires
Financial documents (bills, invoices)
Directories (telephone, office)
Manuals, directions
Types (genres) of reading (cont.)
Personal reading
Newspapers and magazines
Letters, emails, greeting cards, invitations
Messages, notes, lists
Schedules (train, bus, plane)
Recipes, menus, maps, calendars
Advertisements (commercials, want ads)
Novels, poetry
Financial documents (checks, tax forms, loan applications)
Forms, questionnaires, medical reports, immigration documents
Comic strips, cartoons
Developing reading tests
7
Stages of reading test development
Specifying skills (what candidates should be able to do)
As accurate and complete as possible
Selecting texts

Selecting tasks and/or items

Writing test specifications

Writing items/tasks


Activity 1: Identifying reading skills
8
Make a list of the skills and sub-skills that are
commonly associated with reading and testing
reading comprehension.










Categorizing skills
9
Expeditious (fast) operations
Macrostructure of a text
Top-down process
Goal-orientation

Careful operations
Close reading for detailed comprehension
Bottom-up process
Submissive reader





Activity 2: Reading skills in hierarchy
10
Read through the checklist of different operations
in the handout (Hughes, 2003). Place them in three
groups: 1) higher order 2) middle order 3) lower
order in terms of the level of ability or strategy
use implied.




Handout Week 4
Higher order (Macro skills)
Scanning text to locate specific information
Skimming text to obtain the gist
Identifying stages of an argument
Identifying examples presented in support of an argument
Middle order (Micro skills)
Identifying references of pronouns, etc.
Using context to guess meaning of unfamiliar words
Understanding relations between parts of text by recognizing indicators in discourse, especially for the introduction, development, transition, and conclusions of ideas.
Low(est) order (Grammatical and lexical abilities )
Recognizing the significance of the use of the present continuous with future time adverbials
Knowing that the word ‘brother’ refers to a male sibling
Distinguishing between the letters m, n, or b and d (##)
(##) only be useful for diagnostic purposes and can be carried out through informal observation, no call for formal testing
Criteria for text selection
12

Make a list of different criteria that you have
applied in selecting texts for testing reading
comprehension in your own testing practice.
Criteria for text selection (cont.)
Criteria for text selection (cont.)
Notes on text selection
Selecting a representative sample of texts
Appropriate length
Texts should cover all necessary operations
Should not bias anyone in terms of information
Avoiding texts which are culturally laden
Avoiding texts already read
Choose texts which interest test-takers, but will not overexcite or disturb them
Avoid texts made up of information which may be part of candidates’ general knowledge: S may provide correct responses without reading the passage
Cont’d
16
Texts and activities should mirror those in target situation
Reading purposes for texts should be maintained
Full genuineness is unattainable; practice of principled compromise
Avoiding contamination from other factors
Test specifications
Statement of the problem
Specifications
Content
Operations,
Types of texts,
Addresses, and
Topics
Format and timing
Sampling
Criterial levels of performance
Item and key writing
Pretesting
Writing items: test methods
18
Multiple choice questions (MCQ)
Gap filling
Short answer questions
Matching
Diagram-labeling
True/False/Not given
Summarizing





Multiple choice questions (MCQ)
Make a mark against one out of a number of alternatives
Alternative responses are written or take the form of illustrations
Choose the picture (A,B,C,D) which the following sentence describes:
The man with a dog was attacked in the street by a woman

Gap filling
22
Selective deletion gap filling
Example
Choose words from the following to fill in the numbered gaps:

Down, morning, is, not, hello.
Good morning everybody. Good (1), sir.
Sit (2) everybody. My name (3) Mr Hunt.

Gap filling
23
C-test
Example
James Watt was a great inventor who was born in Britain in the 18th century. He was the inventor of the steam engine.
Steam po— was t— first m— made po— used f— transport. Bef— this inve— we h— to u— animal po—.
Short answer questions (SAQ)
Open-ended reading comprehension questions
24
Short answer questions
E.g.
According to the author, what does the increase in divorce rates show about people’s expectations of marriage and marriage partners?
Answer:
They (expectations) are greater (than in the past)
Note: S has answer in his head after reading the text may not be able to express it well.
» guided short answers
Matching: Vocabulary matching task
Diagram-labeling task
True/False/Not given
Summarizing
How to evaluate summaries?
Procedures for test task design
32
Understanding the text

Taking notes of skills and features, main points, interesting pieces of information, stages of argument, examples
Writing items to match skills/tasks

Checking with colleagues for moderation

Piloting and item analysis













Practical advice on item writing
Not to include items that some candidates are likely to be able to answer without reading the text.
e.g., Inhaling smoke from other people’s cigarette can cause ….
Not to include items for which correct response can be found without understanding the text (unless that is an ability that you are testing).
i.e., Ss simply match a string of words in the question with the same string in the text.
Criterial level of performance
34
Pass marks
Performance criteria and standards
Notes on scoring
Errors of gram. , spelling or punctuation should not be penalized ‘cos the function of a reading test is to test reading ability
Giving feedback
Return the test to the student with one of the possibilities below:
A letter grade
A total score
An indication of correct/incorrect responses
Marginal comments
A whole class discussion of the results of the test
Individual conferences with each student to review the whole test
Practice
Follow the given procedures, construct a 12-item reading subtest based on the passage given. For each item, make a note of the skill(s) (including micro-skills) you believe it is testing.