Planning for Student’s Evaluation
&Test construction
(Blueprint)
ž
Outline:
·
Introduction.
·
Definition of Table of specification.
·
Purpose from TOS.
·
Benefits from TOS.
·
Factors considered when design TOS.
·
Guied lines to develop TOS.
·
Construction TOC.
·
Steps for preparing rubric.
·
Types of test items.
§ Objective test items.
§ Essay items.
·
General suggestion for writing test items&
assessment tasks.
·
How to measure achievement of ILOs in TOS.
·
References.
Intended learning
Outcomes (ILos)
A)Knowledge
and Understanding
a.
1.List 3 purpose of TOS.
b.
2.Explain importance of
TOS.
c.
3.Discuss criteria of
develop TOS.
d.
4. Discuss suggestion
for writing test items.
e.
5. Determine the factors
that affecting on the design TOS.
B)Intellectual skills
a.
1. constructing table of
specification.
C) Professional and practical skills
a.
1.Apply table of
specification .
b.
2.Measuring the
achievement of ILOs in TOS.
D)General and transferable skills
a. 1. Aware about different levels of
objectives with different weight.
b. 2.Appraise the development of TOS.
ž
Introduction:
A table of
specification (TOS) is the scientific expression specified to the plan for
writing Test items. A table of specification reveals what has been taught in
the teaching and learning process .In fact table of specification is a mirror
of mode of two dimensions instructional process like Content and Intellectual
procedure .At the same time table of specification Development and Usage are
continuing theme to discussion.
ž Definition:
ü
TOS is specification of the target
ILOs in different domains against specification of different tools , match
question, types of measurement tool against ILOs.
ü
A TOS, sometimes called a test
blueprint, is a table
that helps teachers align objectives, instruction, and ass2 is most commonly
associated with constructing traditional summative tests. When
constructing a test, teachers need to be concerned that the test measures an
adequate sampling of the class content at the cognitive level that the
material was taught.
ü Table of
specification is an instrument that teachers use to formulate a test or
exam. The table is aimed to establish a comparison and organize the number of
questions dedicated to each tier of Bloom’s taxonomy
ü TOS can help
teachers map the amount of class time spent on each objective with the
cognitive level at which each objective was taught thereby helping teachers to identify the
types of items
they need to include
on their tests.
ü A Table of Specifications is a two-way chart
which describes the topics to be covered by a test and the number of items or
points which will be associated with each topic. Sometimes the types of items
are described, as well.
ž
What is the Purpose of a
Table of Specifications?
Ø The primary purpose of a TOS is to
ensure alignment between the items or elements of an assessment and the
content, skills, or constructs that the assessment intends to assess.
Ø In order to
understand how to best modify a TOS to meet your needs, it is important to
understand the goal of this strategy improving validity of a teacher’s
evaluations based on a given assessment.
Ø TOS helps test constructors to focus
on issue of response content, ensuring that the test or assessment measures
what it intends to measure.
ž Importance
of TOS:
· For
teacher:
Ø Way of communication between teachers & each others.
Ø Teachers able to cover importace point of content.
Ø Determine the values of ILOs.
Ø Tool of evidence of teachers & st Achievement.
Ø A table of
specification (TOS) can be used to help teachers frame the decision making
process of test construction and improve the validity of teachers' evaluations
based on tests constructed for classroom use.(As map)
Ø A
Table of Specifications allows the teacher to construct a test which focuses on
the key areas and weights those different areas based on their
importance.
Ø A
Table of Specifications provides the teacher with evidence for test
content
Ø Tables of
Specifications can help students at all ability levels learn better.
·
For
the student:
Ø able to achievement of the ILOs.
Ø By providing
the table to students during instruction, students can recognize the
main ideas, key skills, and the relationships among concepts more easily.
Ø Teachers can
even collaborate with students on the construction of the Table of
Specifications- what are the main ideas and topics, what emphasis should be
placed on each topic, what should be on the test? Open discussion and
negotiation of these issues can encourage higher levels of understanding while
also modeling good learning and study skills.
· For the organization:
Ø
Too for communication inside of the educational
organization.
Ø
Improve communication between the staff.
Ø
Evidance for accreditation .
ž Factors to be considered
when design TOS :
1.
course
learning outcomes/objectives .
2.
topics
covered in class &it,s importace.
3.
amount
of time spent on those topics .
4.
methods of instruction .
5.
assessment
plan.
6.
Facilities
&resources.
7.
Level
of student achievement.
8.
understanding
of the student.
9.
Time
of the test.
10.
Way
of evaluation.
11.
Types
of the test.
12.
Experiences
of the teacher.
13.
Policies
of the organization.
Table: 1 Revised Taxonomy
|
Creating |
Placing
collectively ideas or elements to develop an original idea or engage in creative
thinking |
|
Evaluating |
Judging the
value of ideas, materials and methods by developing and applying principles
and criteria |
|
Analyzing |
Breaking
information down into its component elements |
|
Applying |
Implementing
abstractions in concrete circumstances |
|
Understanding |
Understanding
of given information |
|
Remembering |
Recall or
recognition of definite information |
ž Guide lines to develop
table of specifications :
ü Balance among goals selected for examination.
ü Balance among levels of learning;
ü The test format.
ü Time of the test.
ü Policy of the organization.
ü The total number of items.
ü The number of test items for each goal and level of learning.
ž
Construction TOC:
§ Step 1- Determine
the coverage
of your exam:(course content)
The first rule in making exams and
therefore in making a document called table of specification is to make sure
the coverage of your exam is something that you have satisfactorily
taught in class. Select the topics that you wish to test in the exam. It is
possible that you will not be able to cover all these topics as it might create
a test that is too long and will not be realistic for your students in the
given time. So select only the most important topics.
§ Step 2- Determine
your testing objectives for each topic area:
In this step, you will need to be
familiar with bloom’s taxonomy of thinking skills. Bloom has identified the
hierarchy of learning objectives, from the lower thinking skills of knowledge
and comprehension to the higher thinking skills of evaluation and synthesis.
Bloom’s Taxonomy has six categories:
(starting from lower level to highest) - (1) Knowledge, (2) Comprehension, (3)
Application, (4) Analysis, (5) Synthesis and (6) Evaluation
§ Step 3- Determine the duration for each content area:
The next step in making the table of
specifications is to write down how long you spent teaching a particular topic.
This is important because it will determine how many points you should devote
for each topic. Logically, the longer time you spent on teaching a material, then
the more questions should be devoted for that area.
§ Step 4- Determine
the Test Types for each objective:
·
Now that you have created your table of
specifications for your test by aligning your objectives to bloom’s taxonomy,
it’s time to determine the test types that will accomplish your testing
objectives. For example, knowledge questions can be accomplished easily
through multiple choice questions or matching type exams.
·
If you want to test
evaluation or synthesis of a topic, you will want to create exam type
questions or perhaps you will ask the students to create diagrams and
explain their diagrams in their analysis.
·
The important thing is that the test type
should reflect your testing objective.
§ Step 5- Polish
your terms of specification:
After your initial draft of the
table of specifications, it’s time to polish it. Make sure that you have
covered in your terms of specification the important topics that you wish to
test. The number of items for your test should be sufficient for the time allotted
for the test. You should seek your academic coordinator and have them comment
on your table of specification. They will be able to give good feedback on how
you can improve or modify it.
|
Subject Content |
Knowledge
& Comp |
App. |
Ana. |
Syn. |
Eval. |
Total |
|
Unit 1/ topics |
||||||
|
Unit 2/ topics |
||||||
|
Unit 3/ topics |
||||||
|
Unit 4/ topics |
||||||
|
Total |
|
Weight of Unit / topic |
Total
no. of grades |
Total
no. of questions |
ILOs |
Questions
/ Grades |
Course
content |
||||
|
Ev ------ |
Syn 5 |
An 10 |
App 15 |
K&U 20 |
|||||
|
|
|
|
|
|
|
|
Topic 1 4
sessions |
||
|
|
|
|
|
||||||
|
|
|
|
|
|
Topic 2 2
sessions |
||||
|
|
|
||||||||
|
|
|
|
|
|
Topic 3 2sessions |
||||
|
|
|
||||||||
|
|
|
|
|
Total
no.of questions |
|||||
|
|
|
|
|
Total no.
of grades |
|||||
|
%100% |
|
|
|
Weight of
ILOs |
|||||
ü Some
proponents stipulate that Knowledge & Comprehension account for 30
to 40% of the items.
ü Application and Analysis
account for between 25 and 30 %
ü Synthesis and Evaluation
can be between 30 and 40 %.
ž
Steps of preparing classroom tests and assessments:-
1-
Determine the purpose of measurement
2-
Developing specifications
3-
Selecting appropriate assessment tasks
4-
Preparing relevant assessment tasks
5-
Assembling the assessment
6-
Administering the assessment
7-
Appraising the assessment
8-
Using the results
Ø The purpose of classroom testing and assessment:-
-Classroom
tests and assessment can be used for variety of instructional purposes. These
can be best described in terms of their location in the instructional process.
·
Pre testing:-
-Tests and assessments may be given at the
beginning of instructional segment (unite or course) to determine:
·
Testing and assessment during instruction:
-These tests and
assessment provide basis for formative assessment.
- They used to monitor learning progress ,
detect misconception, encourage student to study and provide feedback to
student and teachers.
-They cover some predefined segment of
instruction (chapter or particular set of skills) and thus limited sample of learning
outcomes.
·
End of instruction testing and assessment:
-They done at the end of course to determine
extent to which intended learning outcomes and performance standard have been
achieved.
- They are used primarily for summative
assessment ( to certify accomplishment or assign grades)
-They serve function of both summative and
formative assessment
-Provide information for evaluating
instructional effectiveness.
-They cover all intended outcomes of course.
Ø Developing specification for tests and assessments
§ Building
table of specification involves :-
1-prepair list of instructional objectives
2-outlining the course content
3-preparing the two way chart
·
Knowledge – remembering facts, terms, definitions and
concepts.
·
Comprehension – understanding the meaning of material.
Explain, interpret, summarize, give examples, predict, translate.
·
Application – selecting a concept or skill and using it to
solve a problem. Compute, solve, apply, modify, construct.
·
Analysis – breaking down material into its parts and
explaining the ordered relations.
·
Synthesis – producing something new from material that
was broken down into its component parts.
·
Evaluation – making a judgement based on a set criteria.
|
Subject Content |
Kno.
|
Comp. |
App. |
Ana.
|
Syn. |
Eval. |
Total
|
|
Topic 1 |
2 |
1 |
2 |
1 |
1 |
1 |
8 |
|
Topic 2 |
3 |
2 |
2 |
2 |
2 |
3 |
14 |
|
Topic 3 |
4 |
3 |
2 |
2 |
2 |
3 |
16 |
|
Topic 4 |
2 |
3 |
1 |
1 |
2 |
3 |
12 |
|
Total |
11 |
9 |
7 |
6 |
7 |
10 |
50 |
|
20 (40%) |
13 (25%) |
17 (35%) |
|||||
§ Some proponents
stipulate that Knowledge & Comprehension account for 30 to 40% of the
items.
§ Application
and Analysis
account for between 25 and 30 percent.
§ Synthesis
and Evaluation
can be between 30 and 40 %
§ There are verbs
associated with the taxonomy which can be used to structure questions under
each cognitive level.
§ Weighting
increases depending on where in the taxonomy the question falls.
§ Objectives/
concepts which account for most of the teaching usually carry highest weight.
§ List topics
covered assigning them a percentage according to time spent on the topic.
Percentage should work back to 100%.
§ Determine how
many questions will comprise the test/ exam.
§ Divide your twenty-five
lessons among the four topics which will be taught. You should know which one
will have the most teaching time and which will have the least.
§ Next calculate
the percentage for each topic in terms of lessons to be taught.
§ You will
determine how many items you wish to have on your examination paper.
§ Once you have
done that you will now work out how many items there must be for each topic.
§ That is done by
finding the percentage, which you calculated previously, of the total number of
items which will constitute your paper for each topic.
§ Using 40% for
Knowledge & Comprehension, 25% for Application & Analysis and 35% for
Synthesis & Evaluation, calculate the number of items to fall under the
sections.
§ If decimals
occur take to the nearest whole number under Application & Analysis.
§ Now you are
ready to fill in your table with the information you have gathered.
§ Be mindful you
are just putting numbers in the spaces. You must ensure that your cognitive
levels are covered and the number of your items are not exceeded.
§ You may need to
re-arrange in order to get the correct configuration but both columns should
work back to the same.
ž
Types of test items:
1-
Objective
test items:-
-
It requires students to
select response from several alternatives.
-
To supply word or short
phrase to answer question or complete statement.
-
Categories of objective
items:
A)
Selection
type:
o Multiple
choice
o True-false
o Matching
B)
Supply
type:-
o Completion
o Short
answer
2-
Essay
items (performance assessment):
o
Extended response essay ( discuss-
describe- compare- summarize)
o
Restricted response essay (list-
enumerates- mention- short account)
ž
General suggestion for writing test
items and assessment tasks:-
1-Use test and assessment specification as guide: They serve as
aid selecting the types of items and tasks to prepare, for writing and for
determining how many items and tasks are needed for each sub domain of
achievement
2-Write more items and tasks than needed:
prepare more test items than needed will permit the weaker items and tasks to
be discarded during later review.
3-Write the items and tasks well in advance
of the testing date: Setting items and tasks aside for several days and
then reviewing them with fresh outlook will reveal any lack of clarity or
ambiguity and determine any defect
4-Write each test item and assessment tasks so
that the task to be performed is clearly defined and it calls forth the
performance described in intended learning outcomes: Clarity should consider when formulation of
questions , Use simple and direct language, using correct punctuation and
grammer and avoid unnecessary wording .During writing and review, compare task
student are asked to perform to learning outcomes.
5-Write each item or task at appropriate
reading level: Keep the reading difficultly and vocabulary level as simple
as possible to prevent these factors from distorting results.
6-Write each item or task so that it doesn’t
provide help in responding to other items;
For example name, date,or fact included in the stem of multiple choice
may be called short answer item.
7-Write each items so that the answer is one
that would be agreed on by experts or in case of assessment task responses
judged excellent by experts: This rule is easy to satisfy when measuring factual knowledge , but more
difficult when measuring complex outcomes in extended essay.
8- whenever test item or assessment task is
reviled, recheck its relevance: Reviewing items for appropriateness,
clarity, diffulty,and freedom from clues and bias . After reviewing items ,
check to be sure that it still provides relevant measures of intended learning
outcome .
ž
How to measuring achievement of ILOs
in TOS?
·
Direct
Measures of Learning Methods:
(Case Studies- Essay Questions- Projects Journals Reflective -
Presentation Seminar and Clinical experiences- Portfolio Behavioral –observations-
interviews -assignments -Self and Peer)
Ø Cognitive
domain measuring:
• Class
discussions – refreshing previous lectures
• Organized
class notes
• Tutorials
• Provide
sufficient information through charts
• Power point slides
• Real
time examples
• Self-check quizzes
• Project/problem
based learnings
• Conducting
course seminars and viva voce
• Practice questions with answers and “expert” explanations.
Ø Affective domain measuring:
• Feedback
from alumni on how to be successful.
• Lecturers-students
discussion on course value.
• Conducting
Quiz at regular intervals.
• Examples of professionals applying
the learned course knowledge in their lives.
• Conducting
course-wise guest lectures.
Ø Psychomotor domain measuring:
• Lab
courses for science classes
• Vocational courses
• Physical
education courses
• Training using specified equipment such as computers,
projectors, videos etc.
• Performing arts.
|
Types
of Student Learning Outcomes |
Application
to Direct Measures of Assessment |
|
Content: Knowledge
of a subject matter (Cognitive learning). |
Examples of direct measures in this category ·
Subject tests (ETS field tests,
Academic Profile, etc.); ·
Portfolios ·
Coursework in Capstone
Courses/Senior Projects. |
|
Skill Acquisition:
Comprehension of a topic, demonstration of a competency, etc. (Behavioral
learning). |
Examples of direct measures in this category ·
Juried performance (music, speech,
art exhibits, poster sessions,
etc.); ·
Portfolios ·
Assignments and coursework in
capstone courses; ·
Major projects (research projects,
major papers, case studies, group projects); ·
Mathematical reasoning tests
foreign language competency tests,
critical thinking tests, etc. |
|
Attitudes: Awareness,
interest, concern, etc. (Affective Learning). |
Examples of direct measures in this category ·
Survey of students’ responses to
value-laden issues; ·
Tests of students’ recognition and
understanding of ethical issues ·
Pre- and post-test measures of
changes in attitudes, values, or beliefs. |
|
Domain |
Description |
Keywords |
|
Remember Recalling important information |
Retrieve relevant knowledge form long term memory |
Define,
repeat, record, list, recall, name, relate, underline |
|
Understand Explaining important information |
Understand, translate, explain facts, concepts, principles, laws and
theories or comprehension |
Translate, restate, discuss, describe, recognize, explain, express,
identify, locate, report and review |
|
Apply Solving closed-ended problems |
Use facts, concepts, laws, theories, principles, knowledge and skills
to solve related problems |
Interpret,
apply, employ, use, demonstrate, dramatize, practise, illustrate, operate,
schedule and sketch |
|
Analyze Solving open-ended problems |
Compare and elaborate the similarities, differences and relationship
between one and the other |
Distinguish, analyse, solve, differentiate, appraise, debate,
calculate, experiment, test, compare, contrast, criticize, diagram, inspect,
question, relate, examine and categorize |
|
Synthesis Creating “unique” answers to
problems |
Merge, combine and integrate facts and ideas |
Compose, plan, propose, design,
formulate, arrange, assemble, collect, construct, create, set up, organize,
manage and prepare |
|
Evaluation Making critical judgments based on a sound knowledge base |
Prove, evaluate, verify, criticize, conclude or to give opinion on a
statement, invention, principles,
theories etc. |
Judge, appraise, evaluate, rate, compare, revise, assess and estimate |
ž
References:
·
Bloom BS, editor. Taxonomy of Educational Objectives: The Classification of Educational Goals: Handbook I, Cognitive
Domain. New York; Toronto:
Longmans, Green; 1956.
·
Anderson LW, Krathwohl
DR, Airasian PW, Cruikshank KA, Mayer
RE, Pintrich PR, et al.
Taxonomy for learning, teaching, and assessing a revision
of Bloom’s taxonomy of educational objectives. Abridged Ed. New York: Addison Wesley Longman, Inc.; 2001.
·
American Educational Research Association, American Psychological
Association, & National Council on Measurement in Education. (1999).
·
Anderson, L.W. (Ed.), Krathwohl,
D.R. (Ed.), Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R.,
Raths, J., & Wittrock, M.C. (2001).
·
Linn, R. L. & Gronlund, N. E.
(2000). Measurement and assessment in teaching. Columbus, OH:
Merrill.
·
Cambridge University Press. Blueprint. In: Cambrigde Dictionary. 2018.
Available at: https://dictionary.cambridge.org/dictionary/ english/blueprint
·
Sood R, Paul V, Mittal S, Adkoli B, Sahni P, Kharbanda O. Assessment in
medical education: trends and tools. New Delhi: KL Wig CMET, AIIMS; 1995.
·
Miller, M, Linn, R.& Grnlund ,N (2009). "Measurement and
Assessment in Teaching"(11th ed.).New york, san francisco.
·
Deyoug ,S (2004). Teaching nursing: a self
instructional Hand book,(2th ed).
London: Roultedge.
إرسال تعليق