Testing and Evaluation in the Translation Classroom

Testing and Evaluation in the Translation Classroom

It is not at all uncommon today for professional translators to be invited to teach a course at a university. Many translators, though flattered at being invited to teach, are hesitant to accept the position due to their lack of pedagogical knowledge. One particular problematic area is that of marking translations and making decisions on student competence. This paper presents the basic information professional translators need to know before they enter the classroom, and outlines possible testing strategies they might use to make their teaching experience enriching and valuable for themselves as well as their students.

One of the most challenging terms for professional educators is 'test.' Even seasoned instructors may not always feel at ease with putting a grade or a mark on a student's final paper. If an entire class does well, the instructor feels proud that work has been accomplished; however, if a large number of students do not perform well, instructors are disappointed and sometimes need to reevaluate the objectives of the entire course. Certainly, students show signs of stress and anxiety before exam periods. Most of us may recall the hollow feeling in our own stomachs the minute just before a test was distributed as well as the silence in the classroom when instructors handed back the corrected papers.

Instructors and curriculum designers today seem to be convinced that a more learner-centered, creative and flexible teaching system motivates students. They also see the necessity to adapt testing methods to the revised curricula and methodologies. Peer correction, self- and portfolio evaluation are becoming common in even the most traditional university settings. Instructors who emphasize a communicative type of testing may promote a more efficient learning environment. They certainly contribute to making tests less traumatic. Nevertheless, it seems that the instructor's testing methods do have a lasting effect on the learning experience, the students' attitude as well as the teacher's enthusiasm (Schmidt & McCutcheon, 1994:118.) Traditional testing is still a critical aspect of education; research in North America has shown that students who take frequent instructor-developed assessments scored higher on national tests (Linking Effective Teaching to Test Scores, 2001). This may be the case not only because of the value of testing, but because the tests are well thought out and allow students to apply the process of what was learned in class as well as the content of the instruction. Instructors must not overlook the importance of student motivation to do well as one important factor in the success of testing. In one survey, students themselves requested numerous quizzes and tests--testimony to the critical role testing plays in a university setting (Kfouri, 2003).

According to Maier and Warren (2000: 131), in university education, there are "various stakeholders" who hold a vested interest in making the system work.

Testing methods affect more than the simple student-instructor relationship in a translation classroom. The instructor's choice of testing strategies first of all sends a message to the individual student regarding competence in a particular skill or knowledge base. The individual student can then compare his or her result with those of the rest of the class. The department of translation will evaluate the level of the tests given in each of the courses and will likely make recommendations concerning the students' performance, the instructors' efficiency and the need to alter the syllabus. The companies, or clients that hire the students will make a favorable or unfavorable judgment of the graduate translator when they compare the quality of the translation to their expectations. If the quality is high, the translation program can take some of the credit; if the quality is low, the education of the student will be questioned. In the end, instructors who prepare quality tests and demand the highest quality from the students will raise the standards of the profession in general.

The figure below shows the various effects of an instructor's testing choices. The instructor is at the center of this quadrangle.

Companies hiring

University
Syllabus, Admin.

Students (individuals
and class)

Standards of the
Translation Profession

Instructors of translation need to become competent in test writing, but they must keep in mind that there is no perfect test and no foolproof grading or marking system.

Some universities request that instructors submit a copy of the midterm and final exams with the course syllabus. Though surprising to a novice instructor, this is actually quite logical, since instructors cannot test without having planned their course objectives or learning outcomes carefully. A midterm or a final exam should test whether or not students have reached the objectives. Before an instructor actually begins writing exams or makes decisions on how to evaluate a student's progress, it is helpful to have an overview of the basic terminology.

Key Terms

Measurement is a process that attempts to obtain a quantitative representation of the degree to which a student shows competence in a particular skill or area of knowledge (Ahmann & Glock, 1981:16.) In order to measure, instructors must have an instrument. The instrument an instructor uses to measure a student's competence has traditionally been the test. A test (oral or written) is made up of items. A student responds correctly or incorrectly to each item. The corrector may mark the test by counting or by judging. Counting correct answers is practical for evaluating receptive skills such as reading or listening. Judging requires that the examination answer key allow for a large number of responses. Instructors are relieved when students respond to the test items correctly. However, if the student does not answer an item correctly, the instructor must analyze further and investigate whether the incorrect answer is a mistake or an error. Even though many people use both terms interchangeably, scientifically speaking, a mistake is generally considered as a fault in performance; it does not occur systematically. An error, on the other hand, reflects a gap in the student's knowledge; it is systematic. An error is therefore more serious than a mistake because it indicates a lack of knowledge; both student and instructor must address the problem when the test is returned (Heaton, 1990).

Evaluation is also a process; it is the systematic process of determining the extent to which students reach the educational objectives set by the institution or standard-setting body that issues their diploma. Evaluation is part of a decision-making process through which the instructor collects information systematically through a test, analyzes that information and relates the results of each student or of the class in general to objectives in the course. Reflective evaluation necessitates the following procedure:

Instructor® prepares learning activities ® carries out instruction ® prepares testing instrument ® administers test ® judges students test performance ® evaluates methodology and questions students ® alters or retains methods or objectives.

Evaluation as shown above depends on the reliability of the test instrument.

Reliability refers to the test's consistency. If the same test were administered a second time under equivalent conditions, the same results should occur (Gage & Berliner, 1998: 519). If the test is not machine corrected, its degree of reliability may depend in a certain sense on the corrector. If the corrector has just read a stellar paper, and the one following is not of the same level even though it is above average, the student may not receive a completely fair mark (Heaton, 1990). Reliability in translation studies is an essential issue. A test of technical translation ability may render more reliable results than a literary translation test. For example, one word in a literary translation may have five to six different almost equivalent synonyms in the target language, each with a different connotation. Moreover, the student translator has to take a number of factors into consideration while taking a literary test. What were the cultural implications, for whom did the author intend the text? How well and how similarly the student and the corrector answer those and other questions will influence the reliability of the translated document and its correction.

Validity, on the other hand, reflects whether the test measures what it was supposed to measure (Ornstein & Lasley, 2000: 392). If, for example, students are asked to write an essay in a language class on the latest methods of imputting data into a database, and those students are not knowledgeable on that particular subject, that test will not be a valid judge of their language abilities. There is also some discussion as to the validity of oral exams since it is not sure how much a student's pleasant or not so pleasant personality impacts on the examiner (Heaton, 1990: 7).

Types of Assessment

Translation students will take a number of tests during their time in university. The tests they take in the university setting will also prepare them for the tests they take as part of their professional lives. Translators are regularly asked to prove their abilities by taking a test before they are hired.

A placement test is generally the first test a student translator will sit for at university. The purpose of the placement test is to classify the level of incoming candidates to a translation or any other skill-based program. A placement test can also be instrumental in the reorganization of a curriculum. According to the results, the department may have to implement remedial or intensive courses. On the other hand, more advanced classes may need to be set up if the student level is higher than preceding years. Placement tests are a practical way to assess the evolution in incoming students' talents from one year to the next. However, university placement tests do necessitate a large amount of research to be effective. The test writers must be aware of the curriculum from which the students are coming; they should also know the curriculum demands of the university. Placement tests must situate the entry level of the student. For example, if you are placing students in an Editing and Revision class sequence, the placement test must measure how well versed the students are already in copyediting, idioms, syntax, so as to assign them to the correct class level.

Diagnostic tests are tests designed to pick out student problems before it is too late in the year or the semester to do so. Their objective is different from placement tests; a diagnostic test is given so as to facilitate the student's learning, to encourage students to correct areas of weakness. For example, if a student was diagnosed with problems in Spanish grammar at the beginning of the semester, and still exhibits the same problems at the midpoint, a solution must be found. Some progress tests may also serve a diagnostic function (Heaton, 1990).

Progress tests are the most frequent tests instructors give. The objective of a progress test is to determine if the students have mastered material that has already been taught. In theory, if the teaching has been sufficient, if the syllabus is organized efficiently, if the test is well written and of course, if the students have been attentive, marks on a progress test should be high. If the marks are not all above 75 out of 100, then the instructor will have to determine why and alter the weekly course distribution (Heaton, 1990).In a translation classroom, where rote learning is not emphasized, progress tests apply the principles of translation. Progress tests are most often "open book" in translation classes. Students have access to notes, databases, dictionaries, etc. Open book tests are suitable in testing situations where the instructor is determining how competent students are in applying knowledge, not recalling it. Quizzes, graded homework, short projects, weekly or bi-weekly tests are all types of progress tests.

Achievement tests are meant to determine if the student has met the course objectives. If students were placed in the correct course level, benefited from the results of diagnostic tests and progress tests, the achievement test should reaffirm their acquisition of skills necessary to advance to a further level of study. Achievement tests are usually all-inclusive and occur at the end of the course. Their results should be examined closely so as to evaluate the program's strengths and weaknesses (Bahous, 1998; 39).

The types of tests above fall into two further traditional categories: formative and summative.

Formative assessment is the most common form of assessment in higher education and constitutes the bulk of instructors' efforts to evaluate students. Formative assessment takes place during the instruction period and is designed to guide instructors to adjust their teaching, if need be (Gage & Berliner, 1998:529). Progress tests also fall into this category, as do diagnostic tests. Feedback from formative assessment must be communicated to the student as soon as possible. Students react more positively to formative assessment if the results are analyzed by the instructor and the teaching style or class content is altered if need be. This is called the washback effect (Heaton, 1990: 16). In order to gain the students' trust in the value of evaluation in their overall education or the course itself, formative assessment should not be used as the only means to determine the final grade. Ideally, formative assessment is the ongoing process instructors and students use to gauge the success of the syllabus and to prepare for the second type of assessment, the summative.

Summative assessment contrasts with formative assessment first of all by its purpose. The purpose of summative assessment is to attribute value, and for that reason it is oftentimes more quantitative than the qualitative formative assessment. It also occurs at the midpoint and/or end of instruction so as to determine the extent to which syllabus objectives have been met. Achievement tests, final exams, oral or written, and research projects are examples of summative assessment. Grades or marks from summative assessment often provide a basis for passing a student or for repeating a class. The weight of summative assessment in the student's final grade or mark is oftentimes quite high; in some universities as high as 60%. Both types of assessment are necessary and complementary. However, if summative evaluation shows that the majority of the class is not at the level the instructor had targeted, then it has come too late and the formative assessment was also not sufficiently well planned (Heaton, 1990). It is for this reason that diagnostic tests must not take place too late in the semester; otherwise it will be too late to lift constraints to learning.

Process assessment is a relatively new assessment technique that is more formative than summative. It works most efficiently with long-term projects and is particularly applicable to higher-level translation studies. An instructor sets process assessment in place by first setting benchmarks the student must attain (Types of A&E, 2002, MIT). For example, in a translation practicum whose objective is to emulate the workplace, the instructor would begin by distributing the brief. A timetable for documentation, translation and revision would be set. Students would form teams and distribute tasks. In order to further simulate the workplace, a timetable for the final project would be assigned as well as an estimate of the projected costs to be incurred during the project itself. The assessment would take into consideration if the deliverables were produced and delivered on time and within the cost estimates. Success of the project is determined by the difference between the blueprint of the original project and the final product. Already used in business schools where case studies form the basis of the curriculum, it is easily adapted to translation classroom or internship work. (Types of Assessment and Evaluation, 2002, MIT).

Portfolio assessment is also a relatively new technique to aid students in tracking their progress. Not only do the students track their own level but also the instructor is able to judge the student's work in reference to past assignments. A portfolio is a file that students compile throughout the semester or course and in which they choose the work they have done and want to be marked for a final grade. Instructors can determine the minimum number of assignments per week, or each two weeks, to be included in the portfolio. The portfolio method is time consuming for instructors who have large classes, but the advantage is that instructors can gauge the progress of the student by actually consulting the work done by the student at the beginning of the course or in the middle rather than only consulting the marks in their book.

Test Items

Translation instructors need not depend only on a text as a basic test item.In order to assess in a formative or summative manner, instructors have a wide range of item formats to choose from. The basic types of item format are objective and subjective. In a simple format objective test, the items may be supply, true-false or alternative response, or matching. Multiple-choice and interpretive items are more complex forms of objective tests. Essay tests and their derivatives form the basis of subjective exams. Although translation instructors may not use all these types of items on a regular basis, it is useful to experiment with various means of determining how well your students apply the information you present. From the examples below, you may find some new ideas.

Supply or free-response items

Unstructured short answer and fill in items are the main types of free response test questions. They are used primarily in informal testing. The great advantage to these items is their ease of preparation and correction even if students do sometimes present answers that were not originally in your key.

Ex: Unstructured short answer

What is the main function of the human liver?

What is the medical specialty which deals with cancer patients?

Ex: Fill in

The following groups of words are not full sentences. In the space provided add whatever is necessary to make them complete.

He was in such _______he forgot his suitcase.

The deeper the roots are, ____________to pull the trees out.

The Two-Alternative Items

More commonly known as yes/no, true/false, such items measure how well students know facts and definitions, and if they can distinguish between fact and opinion. They are however difficult to write clearly and should not include terms such as never and always.

Ex: True/False

Place T or F in the space provided. If the statement is false, provide the correct answer on the line below the statement.

Consumers and producers share the burden of a sales tax. (T)

The seller of the product levies sales taxes. (F)

The government levies the tax; the seller collects the tax for the government.

Multiple Choice Items

Multiple choice items can be used to measure a variety of learning objectives such as vocabulary acquisition, analysis, application of principles, cause and effect association or the ability to interpret data (Ebel & Frisbie 1991:183). Early research in testing and measurement has shown that a multiple choice test with a given number of items can be expected to show as much reliability in the scores as a true/false test with twice that number of items (Ebel 1979:74). It is challenging to write valid multiple-choice test items. Professional test writers are expected to produce ten such items in an eight-hour day (Gage & Berliner 1979: 732). Whether to test through multiple-choice items or not is quite a controversial issue. Some instructors do not see the necessity of offering four alternatives; some instructors believe they encourage an unnecessarily passive attitude in the student. Actually, multiple-choice tests have more advantages than disadvantages. First, an instructor can build an item bank and alter particularly effective questions and use them more than once. Also, multiple-choice statements offer the instructor one means of being creative in the testing of translation skills.

Ex: Multiple Choice

Read the text below and answer the questions.

We are intensely competitive. If we think that we have any chance at all to move beyond bare survival, we are almost all ambitious. We worry about winning our honor, our pride, our integrity, our desire to be heard, our need to be right, who recognizes us, whether we are achieving enough, rich enough, good-looking, well-dressed, influential-the list is endless. We are easily jealous and « stupid » people call us arrogant when all we are is competent. We worry about status, position and whether we have clout. We are constantly trying to avoid those who would coerce us, manipulate us or use us. That we have often been wronged and seek revenge is much on the minds of many of us. Do people put us down or avoid us when we offer « constructive » criticism of how they live their lives ? If what I have written here-and I could go on and on-does not pertain to the way you live your life, then it may be that you are not driven by this need. But then maybe you are not of our species : Among us, even the humble compete for who can be humblest of all.

Taken from : Glasser, William, M.D. Control theory in the classroom. 1986 Harper Collins.

According to the author:

everyone has basically the same needs even if we do not all admit them

some people around us are not human

it is absolutely not normal to be jealous

it is abnormal to be ambitious

When the author uses the term "stupid" he really means that

arrogance is a positive characteristic

in reality, we all think we are among the best

there are many stupid people around us

very few people are actually competent

If you want to experiment with constructing multiple- choice questions, the following guidelines are valuable: (Gronlund 1985: 182)

The stem of the question should be meaningful in itself.

Not: WinZip: a. b. c. d. But, WinZip is a computer tool which: a. b. c. d.

State the stem of the question in positive terms.

Not: Which one of these translation memory tools is not useful?

But: Which one of these translation memory tools is most useful for free-lance translators?

Write all alternate answers in parallel form.

Not: The one disadvantage of the Excel program is:

The speed in processing large files

The installation problem

The fact that it is not compatible with other programs

The high price at the outset

But: a. The processing speed

The installation procedure

The compatibility issue

The financial question

Give only one correct answer

Not: Lake Michigan is found in:

Canada

United States

Indiana

Ohio

But: Lake Michigan borders on:

Ontario and Michigan

Indiana and Michigan

Ohio and Michigan

Quebec and Michigan

Dictation and Dicto-Comp

Dictation is quite useful in a translation classroom to test the receptive skills of listening and recognition and use of terminology. It is definitely not only a method of checking a student's spelling. After students have documented a text to be translated, or read parallel texts, they can benefit from dictation taken from one or more of the texts. The benefit of dictation after reading is that students practice putting into words what they have read perhaps passively. Students of interpreting skills benefit from dictations because the instructor can vary the speed of delivery, and can ask colleagues to deliver a dictation so students become accustomed to various accents. The following is a simple methodology for a classroom dictation that should not exceed 6 minutes. A student's attention span and focus fades after that.

Ask students to listen to the entire text once with pens down

Read the text again in logical segments. Start slowly so that student get used to the rhythm of the reading.

Pause between segments to give students time to write.

Read the text again at the end of the dictation so students can correct any errors.

As a means of creative dictation, you may use a small portion of the text students are to translate and dictate one section as you see it in the target language. If students need practice in one particular area of difficulty, such as numbers, choose a text that emphasizes this aspect, or create a text yourself.

Marking a dictation is very straightforward. Inform students in advance of the criteria you are using. For example, insertion of incorrect terms or incorrect verb agreement is a more serious error than simple spelling. Beginning with 10 points or 20 if the text is longer, take off one mark for every error.

Language instructors have been using Dicto-comp as a means to test student ability to remember main ideas of a text in chronological or logical order and as a test of comprehension. As a type of formative assessment, an instructor can gauge how much of the original text the students have understood by how well they are able to rewrite it in a logical order. Translation and interpreting instructors can use dicto-comp in both the L1 and the L2 of the student. It can be used after the students have prepared documentation for their translation but have not yet written the translation. The following is a simple methodology to try dicto-comp.

Read the text to the students several times. Students listen with pens down.

Then ask the students to write what they remember in a logical order staying as close to the original as possible. To correct the dicto-comp, provide students with the 5, 10, or 15 main ideas in the order of the original.

What is practical about this test is that translation students are initiated into the idea of translation units and can then move on to consecutive interpreting with greater ease. This type of test is particularly appropriate for instructors of consecutive interpreting.

 

Example dicto-comp text and its correction.

Directions to students. Listen to this text and write down the three main ideas in the order in which they are presented in the text.

If academic learning is not just about acquiring knowledge, is it really different from the acquisition of everyday knowledge? We learn a great deal about the world very successfully outside academic institutions, with no help from any didactic process. The tradition of pedagogy that stretches back to Dewey's rejection of the classical tradition of passing on knowledge in the form of unchangeable ideas, has always argued for the active engagement of the learner in the formation of their ideas. More recent exponents of the latter tradition are Vygotsky, Piaget, Bruner, all of who argue for the active engagement of the learner rather than the passive reception of given knowledge. These psychologists have had an effect in schools, especially at primary level, but in universities, with their continued reliance on lectures and textbooks, the classical tradition of 'imparting knowledge' still flourishes.

(Text taken from: Laurillard, Diana. Rethinking university teaching. 1993. Routledge.

Dewey rejected the classical tradition of passing on knowledge in the form of unchangeable ideas.

He and others such as Vygotsky, Piaget, etc. argue in favor of active engagement of the learner rather than passive reception.

Schools have adopted the thinking of these psychologists, but the classical tradition of imparting knowledge still flourishes in universities.

Subjective/Essay Testing

Essay type questions do not often apply to the testing of translation skills as such. Students of translation do not write about translation unless they have a particular reason to comment on a particular theory or critique a translation. Before you ask students to write a critique of a text, provide a format for them to follow. A common critique format is the following:

Paragraph One: Introduction (title, author, when and where is was published, intended audience)

Paragraph Two: Elements with which the student agrees and concrete proof why.

Paragraph Three: Elements of disagreement and proof why.

Paragraph Four: Conclusion (comments, suggestions)

Some instructors may need to ask short essay questions in a test for a course on Legal Terminology. A sample question may be:

Differentiate between 'tax law' and 'real estate law'. Give two examples of where laypersons may be confused.

Assessment and Grading/Marking

When instructors mark exams they usually do so based on one of two traditional options available. Norm-referencing assessment judges one student's performance based on the rest of the students in their group. The group is the norm. Students will be informed if they fall in the top or bottom third of the class, for example. In competitive testing situations, a norm-referencing assessment is used. When an institution wants to compare the test results of all the candidates and only take the top 10%, for example, they will use norm referencing. The candidates are competing against each other. If there are many high quality students, an average level student may not be admitted; if there are few high quality students, the average student will certainly have a higher probability of passing the test. Very often, universities may also restrict the number of students received in order retain an aura of "quality." (Maier & Warren, 2000: 132.

Criteria-referenced assessment involves evaluating whether the student can perform a task or not; instructors are not concerned with the comparison among students. In translation classes, criteria-referenced tests are more frequent. Criterion referencing may be fairer from a student's point of view since it compares the students' results with fixed criteria. Students are judged on how well they alone can perform a task (Heaton, 1990). For example, can they complete a technical translation within a fixed time period? In theory, all of the students may be able to do so.

Ipsative referencing compares a student's present performance with a previous one. Generally considered effective in special needs education and performance coaching, it may be beneficial in translation classes as it enables students to judge how much they have progressed within a fixed period of time (De Montfort University, 2003).

Instructor Assessment

There have been many suggestions made as to how to mark a translation. Certainly the type of translation whether technical or literary plays a crucial role in the type of correction you choose. The corrector also plays an important role. Some emphasize certain criteria above others. Students in a classroom must be informed of the criteria you are judging.

There are basically three options an instructor can choose from when correcting a translation.

General Impression

Although some experienced instructors are able to differentiate between a paper that is a 62/100 rather than a 67, for example, a general impression mark is not very beneficial to the student for it does not, in general, provide the reasons for the missing marks.

Error Count

A simple error count is not recommended as a method of marking a student's translation since it rarely gives points for content and does not take into consideration the seriousness of the errors.

Analytical Grid

Heaton (1990: 110) proposed an analytical grill for language courses. However, it can be easily adopted for a translation correction. An analytical grid allows the instructor to set clear criteria for correction based on simple arithmetic.

Correction

Criteria
5
4
3
2
1

Fluency /Flow






Grammar






Terminology






General Content






Mechanics
X
X

In this particular case the translation would be marked over 23 since the instructor chose to weight mechanics less than the other areas. When students are provided with a grid assessment, they are able to see where their weaknesses and strengths lie. Some instructors provide their students with a complete description of each number used on the grid. For example, a student who receives a 5 on the Fluency category would know that the instructor considers this quality work to be an almost native style of writing with varied sentence structure. Coupled with descriptive comments such as the examples below, a student will be able to rewrite the translation with a clear focus. Descriptive comments are similar to the "I" messages suggested by both communication and education specialists. (Cangelosi, 2000)

Your use of prepositions is incorrect almost 3/4 of the time; review before you write again.

You take an inappropriate amount of license in translating this technical text.

Consider your target audience before you translate.

There are too many examples of basic grammatical errors for me to evaluate this text. Begin again.

I feel that you have really gotten the feel of what the original author wanted to say.

Self-Assessment

Translation students are adults who have chosen to pursue a career in language

services. The majority knows that competition is quite stiff and in order to succeed they must excel. Asking students to assess their own progress is one way of initiating them to see their work objectively. Below is an example of a translation student self-assessment paper that can be given to the students at the beginning of the semester or course. A simple Likert Scale is used for facility.

Translation Student Self-Assessment

Directions: Respond to the following statements truthfully using the scale given to you.

Statements
Never
Often
Sometimes
Always

1.
I understand all that I read in my L1.





2.
I understand all that I read in my L2.





3.
I am confident that I will be an effective translator.





4.
I make serious comprehension errors when I translate.





5.
I make grammar mistakes when I translate.





6.
I feel comfortable working on a computer when I translate.

An instructor may add statements that are appropriate for the particular course or the maturity of the student translator.

Some students may show surprise at the mark they receive. A self-evaluation sheet filled out directly after an assignment may provide the student with helpful clues to their weaknesses. The example below can be modified to fit both the instructors and students' needs.

Assignment Evaluation
Yes
No

1.
I understood the text the first time I read it.

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

2.
I had to consult resources minimally.
______
______

3.
I devoted a lot of time to documentation.
______
______

4.
I felt that I was linking the major parts
of the text in a logical manner..
______
______

5.
I felt at ease translating this subject.
______
______

Peer Assessment

Students are effective revisers and evaluators of each other's work. They are even more effective when they help decide on the criteria for the assignment undertaken. For example, students can agree that errors easily corrected by Spell-Check would not be considered as serious, but that a "contresens" would be. They may not be asked to put a mark to the work, but they can find areas in the translation that are not clear or which they themselves translated differently. In fact peer assessment is an extremely useful learning experience. Here are some hints for peer assessment.

Have students work with one student with whom they feel comfortable and secure.

Once students have evaluated one partner's work several times, they should work with another student's work so as not to become used to their partner's errors.

Students should have completed the translation that they are evaluating.

A specific time limit and correction symbols are important to ensure consistency.

Ask students to evaluate the work in another color pen than you yourself use so as not to confuse the student.

Give students time to explain their reactions to the work orally as well as written.

Testing and Evaluation in a Academic Atmosphere

Some instructors feel that their prime role is to test their students' progress. They seem to test more than they actually provide opportunities to learn.

Remember that testing your class is as much a reflection of your own teaching as it is of the students' knowledge. A test may evaluate the effectiveness of your instruction. Do not be tempted to coach for a test, or teaching for the test as it is sometimes called. Teach in a way that prepares students to apply what they have learned in any situation, test or normal class work.

If, as in the case of many university courses presently, you are teaching with a team of teachers in what is called a "multi-section" course and are called upon to write a common exam for your students as well as the other instructors' students, remember the following:

Contribute items that have not been covered on your own class quizzes; this is not a fair evaluation of your students in comparison to the others.

Consult with the other instructors in advance as to what is to be covered on the exam.

Set up a common grading scale as well as the common exam.

Meet and exchange papers to make sure grading is consistent. For example, ask that all your colleagues bring three papers for discussion: the highest, the average and the lowest grades. Exchange the papers and discuss objectively.

You may even experiment with exchanging entire class sets of papers for truly objective grading.

Conducting an Exam

It is possible to have prepared a very valid exam and be dissatisfied by how the exam was conducted. Simple preparation can help you avoid any difficulties and keep the students calm and focused.

Arrive in class early on the days on which you give tests. Make sure that the furniture is set in an appropriate manner, that is, there is ample space between the students.

Make sure that your test has an explicit cover page on which your directions are clear. Do you allow scratch paper? Do you allow dictionaries? How much time is given for the test?

Distribute the test in a professional manner. If the class is large, and the rows are long, give one pile to the first person and have them take one and pass the rest back. Or give half and distribute from the half of the hall back.

Once the test has been distributed, ask the students to look through the test. If they have any questions at all, allow them one minute to ask. After that, no questions will be answered. If you do not allow for a question period, and make it clear that this is the only time, you will be bothered during proctoring.

Be very clear about your policy on cheating. Announce it before the test and be consistent. Most instructors use the following:

Cheating is not tolerated.

If cheating is suspected, your paper will be taken away and you will be asked to leave the exam room.

One of the best ways to avoid cheating is to never use the same exam twice. Some students are collectors of old exams and you are just inviting problems.

When you proctor, although it is among the worst jobs you will be asked to do at a university, do not be tempted to read the newspaper or chat with other proctors. Walk around the exam hall and show that you are taking this test seriously. When you do, the students will also.

Correct your exams quickly, within 48 hours if possible and post the grades, marks on your office door. Allow students the right to see their final exams during specified office hours. Make sure that you have double-checked your math and that there are no mistakes in your calculations. In any case, mistakes can occur. Invite the students to recount when you distribute the corrected exam.

Case Studies of Tests for Translation Courses.

It is tempting to give a text and simply request that it be translated. If the objective in testing is to evaluate the overall ability of the student then this is an appropriate method. However, instructors may wish to test specific skills.

The hardest part of writing a test is deciding how much material can be tested within a certain time frame. Many instructors have a tendency to write tests that are too long for the two-hour test period, for example. In order to circumvent this problem, observe your class as they work in a class situation. When you carry out activities in class, gauge the amount of time your class needs to complete the work.

 

Examples of Translation Tests and Quizzes

Timed Matching

Even though students are not required to memorize terminology, you may have requested that they have a basic knowledge of the terms used in a particular theme. One way to test basic knowledge on a theme is to give students terminology in the source and their equivalents in the disorder. Students are then allowed 4 minutes, or more (or less) depending on the length of the list to find the correct match.

Example:

Directions: Match the terms in Column A with the correct translation in Column B. Write your answer in the blank in Column B.

Column A
Column B

Cosmetics

1.
External use only
______
a.
la peau grasse

2.
Sun block
______
b.
le teint

3.
All skin types
______
c.
l'usage externe

4.
Complexion
______
d.
écran solaire

5.
Oily skin
______
d.
toutes types de peaux

In order to test the student's ability to apply the terminology, you may give the students sentences that must be translated within a certain time limit.

Example:

Directions: Translate these sentences in two minutes; then ask for Part Two.

Rinse off with warm water and follow with a cream of your choice.

Apply generously to the neck and face.

This cream will ensure you the softest skin ever.

Directions: You have two hours for this exam.

You may not use lexicons or dictionaries for Part One.

When you have finished part one, turn it in, and start Part Two.

You may use your documentation for Parts Two and Three.

Example:

Part One. 5 pts.

Read the following statements, and their translations. Correct the errors. Errors may be of language or of sentence formulation and meaning.

Saudis split over possible freeze of their US assets. (original)

Il est possible que les Etats Unis gelent les avoirs Saudiens.

Le droit à l'alimentation est aussi un formidable enjeu de démocratie. (original)

The right of food is also an enormous challenge for democracies everywhere.

Le comité chargé de la lutte contre le blanchiment a publié hier un avertissement pour mettre en garde les libanais contre des operations frauduleuses. (original)

The anti-money washing committee published an advertisement yesterday which told the Lebanese public that they better not participate in illegal money transactions.

There has been no dramatic shift in terms of getting out of the US. (original)

(Note: with reference to getting money out of the US)

Personne ne veut quitter les Etats-Unis avec son argent de poche.

En realité, ces accords n'ont pas profité aux pays en développement. (original)

Actually, these agreements did not profit to the developing countries.

Example: Translate the following extract into Arabic. /70 pts.

Comment on five difficulties you encountered. 20 pts.

Do the cultural elements here make the text difficult to translate? Why or why not? 10 pts.

"But there is something I must say to my people, who stand on the warm threshold which leads into he palace of justice. In the process of gaining our rightful place, we must not be guilty of wrongful deeds. Let us not seek to satisfy our thirst for freedom by drinking from the cup of bitterness and hatred. We must forever conduct our struggle on the high plain of dignity and discipline...

When we allow freedom to ring, when we le it ring from every village and every helmet, form every state and every city, we will be able to speed up that day when all God's children, black men and white men, Jews and gentiles, Protestants and Catholics, will be able to join hands and sing in the words of the old Negro spiritual: Free at last. Free at last. Thank God Almighty, we are free at last." Martin Luther King, Jr.

Example:

For a higher-level course, provide two translations of the same text, or part of one, and ask students to evaluate the strengths and weaknesses of each.

Testing will probably never be the high point of a teaching experience, but we can try to make our tests as creative as possible so that students learn both from their time in our classes and our testing sessions.

 

References

Ahmann, J.S. & Glock, M. (1981). Evaluating student progress: principles of tests and Measurements, 6th ed. Allyn and Bacon, Inc., Boston.

Assessment. ( 2003). De Montfort University. Online

Bahous, J. (1998). The reliability of supply items. Unpublished doctoral thesis.

Cangelosi, J. (2000). Classroom management strategies, gaining and maintaining Students'cooperation, 4th ed. John Wiley and Sons. USA.

Ebel, R. (1979). Essentials of educational measurement, 3rd ed. Prentice Hall, Englewood Cliffs.

Ebel, R. & Frisbie, D., (1991). Essentials of educational measurement, 5th ed. Prentice Hall of India, New Delhi.

Gage, N.L. & Berliner, D.C. (1998). Educational psychology 6th ed. Houghton Mifflin. Boston.

Gronlund, N.E. (1981). Measurement and evaluation in teaching. Macmillan New York.

Heaton, J.B. (1990). Classroom testing. Longman, New York.

Kfouri, C. (2003). "Classroom management in the university classroom" The Near and Middle Eastern Journal of Research in Education. under press.

- "Learning and Teaching Assessment." http://www.dmu.ac.uk/~james/teaching/assessment.

- "Linking Effective Teaching to Test Scores." Gifted Child Today. Winter 2001. V 24. Infotrac. Online. ASAP.

Maier, P., Warren, A. (2000). Integrating technology in learning and teaching. Kogan Page, London.

Ornstein, A. C., & Lasley, T., J. II. (2000). Strategies for effective teaching. 3rd edition. McGraw Hill, Boston.

Schmidt, C. & McCutcheon, J. (1994). "Verbal versus nonverbal cues in evaluations of teaching." The journal of research and development in education. vol.27, n.2, pp. 118-125.

Types of assessment and evaluation. Assessment and evaluation.
http://web.mit.edu/tll/assessment/types.htm