Assessment for Career Counselling
Methods of Assessment
A psychometric tool examines a sample of behaviour to understand the whole behaviour. All psychological tests require a person to perform some observable and measurable action. This could be a test, an inventory or a questionnaire. The test could be a written test, or the testee may be required to perform some action (e.g., drawing or giving a speech or creating a model). This performance is used to measure some personal attribute, trait, or characteristic. A psychological test measures a person’s performance on a set of items in order to come to conclusions about a broader personal attribute, trait, or characteristic.
There are two broad approaches to assessment: the quantitative approach and the qualitative approach.
Quantitative Approaches
Quantitative methods are psychometric in their approach. They are based on the assumption that individuals possess a unique combination of traits and qualities which can be accurately measured and quantified. The person is given a test and scores are obtained. Scores are expressed as quantities, usually in numbers (e.g., IQ = 121 or Linguistic aptitude = 23/40).
Methods Used
Quantitative methods are designed to quantify and give a numerical value to an observation. The most common tools used are:
- Tests
- Questionnaires
- Inventories
- Checklists
Usefulness and Limitations
Psychometric instruments are highly useful tools. However, their accuracy is reliable only if they are:
- Culturally and statistically validated for the group for which they are intended. A test developed for American students may not give accurate results if used with Indian students.
- Age appropriate. Tests are sensitive to age. A test developed for Grade 10 students may be not be useful with Grade 12 students.
- The test taker is able to give the test his or her “best shot”. Anxiety, fatigue, skepticism, playfulness, succumbing to socially acceptable responses, can all affect test performance.
- Administered by a qualified psychologist / counsellor. A given test usually requires the test giver to hold a license to administer the test.
Tests and Norms
- By itself, a student’s raw score has little meaning. A good test will provide norms to interpret a student’s performance.
- The term “norm” is short for normative. Norms are a statistical framework based on performance of a large number of students (called a normative sample) on the test.
- Norms make it possible to determine where an individual stands in comparison to his/her peers.
- Test norms can only be developed for tests that are standardized. To be effective, tests should be administered in a “standardised” manner: testing conditions, test instructions, scoring and interpretation are expected to be uniform across test-takers and testing environments.
- Finally, having norms is important, but not sufficient. A test is useful only if it has norms that are relevant to the persons being tested.
Check this website for more information on the importance of norms.
Qualitative Approaches
Qualitative approaches focus on quality rather than quantity and aim at describing rather than measuring or quantifying. The counsellor dialogues with the client and talks to him or her to gain deeper insights into the person’s interests and aptitudes. Qualitative data therefore is verbal data. This data is analysed to look for patterns and trends in the person’s orientations. The qualitative observer would not say, Linguistic aptitude = 23/40 but would describe the aptitude in terms of the person’s fluency with words, sensitivity to shades of meanings and so on. Therefore in the qualitative approach, there are no tests, no right or wrong answers, no scores or norms. It is through an intense verbal interaction and through asking the right questions, that the counsellor learns more about the client.
Methods Used
Qualitative methods are designed to collect information using non-mathematical techniques. The most common techniques used are:
- Interviews
- Observations
- Discussions
Usefulness and Limitations
- Qualitative methods allow matters to be considered in detail and in depth. Through a good interview the counsellor can go back in client’s history and gain rich information.
- Since interactions between the counsellor and client are not limited to a set of standardised questions, the direction of an interview can be guided and thereby the interviews are more responsive to the client.
- On the other hand, the quality of the data obtained relies heavily on the counsellor/assessor. The biases, idiosyncrasies and preoccupations of the assessor can influence the direction that the interview takes.
- Qualitative methods require a highly trained and experienced assessor, who is able to maintain objectivity while simultaneously eliciting subjective data.
- Another issue is that the results of qualitative research can’t be verified easily and one must rely on the opinions of the interviewer.
So which is the better method: Qualitative or Quantitative?
As seen in the figure below, both methods have their advantages and limitations.
- From the quantitative perspective: If the tests used by the quantitative approach are not reliable and valid, then the result will not be accurate
- From the qualitative perspective: If the interviewer is not experienced and alert all through the interaction with the student, the data obtained could be unreliable. Also, qualitative approaches are highly time consuming.
There is a possibility that the reliability of guidance given to the student could suffer in either case.
It is for this reason that the Jiva system of assessment takes a Mixed Methods approach. Read on to know more.
Mixed Methods Approach
The Mixed Methods approach is one of most modern and contemporary approaches to assessment. It attempts to circumvent the shortfalls of purely quantitative or qualitative approaches by blending quantitative and qualitative methods. Information collected using one method is verified and validated against information collected using the other method.
A mixed methods approach to assessment is not simply about collecting both quantitative and qualitative data, but using these in tandem to collect more comprehensive information as compared collecting information using just one of the approaches. Quantitative data (e.g., test data) is cross-referenced with the quantitative data (e.g., interview with the student) to build a multidimensional database of useful information. All attempts are made to limit the gaps of using any one method through data linkage and or integration. For example, a quantitative test may not be able to pick up what the student’s accomplishments are in daily life. The qualitative approach would be able to cover this gap through a well administered interview.
Methods Used
- Psychometric devices such as standardised tests, inventories and questionnaires with norm based scoring protocols.
- Qualitative tools such as guided interview formats, discussion schedules and observation protocols.
- A unifying theoretical foundation and algorithm based on which qualitative and quantitative data can be integrated.
The Jiva Mixed Approach to Assessment
The Jiva system uses a combination of quantitative and qualitative tools:
- Quantitative Tools: The Comprehensive Student Information Protocol (CSIP) is a quantitative psychometric tool that combines an interest inventory and an aptitude test. The CSIP has been developed by Jiva specifically for Indian students and has been standardised on a sample of over 15000 Indian students.
- Qualitative Tools: Jiva counsellors use a guided interview format with the student and parent to collect qualitative information. This covers the student’s academic history, preferences, hobbies and accomplishments.
- Both methods are embedded within the Multiple Potentials Framework. Click here to know more.
- Jiva then uses a special algorithm to combine both units of data.