The Feasibility of a Selection/Evaluation Program
for Vietnamese Technical Supervisors

The development of Vietnam as a self-sustaining and viable nation depends, to a great extent, on the technical proficiency of the Vietnamese people. Moreover, as the technical level of the nation develops it becomes vital to augment this work force with adequately trained supervisors and managers. The successful turnover of U.S. military bases to the RVNAF depends on the proper selection and training of effective technical leaders. Due to the expeditious pace with which base turnover must be accomplished, the development of supervisory potential among the Vietnamese will necessarily require immediate program development and implementation.

The Vietnamese technicians who will become supervisors and managers must not only be technically proficient, but must be skilled in leadership, organization, and planning. These extra-technical skills enable the supervisor to meet operational standards, maintain equipment, reduce waste, prevent accidents, maintain high morale, and hold absenteeism and tardiness to a minimum.

Since good supervision involves skills beyond technical ability, the identification of leadership and supervisory potential among technically trained individuals is an important first step to ensure training effectiveness and to reduce training time and costs. It is essential to identify supervisory potential before the risk is taken by placing individuals in advanced training or responsible positions. The fact that an individual is a good technician or scores high grades in the classroom is no guarantee that he will be an effective supervisor. Adequate identification and development of management and supervisory potential will insure reliable and expeditious national advancement.

A pilot study was conducted to determine the feasibility of a selection and evaluation program developed for use as a supervisor training program. The specific objectives of the study were to:

1. Determine which selection techniques are appropriate for Vietnamese personnel involved in supervisory training.
2. Design a screening battery that will effectively identify supervisory potential.
3. Develop guidelines for supervisor selection and evaluation among technical personnel.
4. Develop a procedure to evaluate the selection technique against operational criteria.
5. Identify areas of training appropriate to supervisor development.

Identification of management potential requires an appropriately designed screening technique based on cross-cultural standards and a clear concept of what is to be measured. Therefore, the selection procedure was developed with assistance from the Vietnamese Testing and Research Center in Saigon to insure proper cultural context. After careful analysis of cross-cultural variables and job functions, an assessment plan was developed based on a supervisory concept regarding characteristics listed below.

Personal characteristics. A supervisor must possess certain personal characteristics related to basic leadership ability. Although each culture has a fundamental character or personality pattern, there exists within each culture individual variations that are related to successful performance of particular functions. Notwithstanding the major cultural differences that exist between Vietnam and other nations, the same fundamental characteristics are required in most cultures for effective performance as a leader. The latter conclusion was reached with assistance from cognizant personnel within the Vietnamese military who are involved in leadership training. Therefore, fundamental to effective leadership are personal characteristics such as:

Negative characteristics are also related to effective leadership.

Although leadership capability involves other factors, the six positive and negative characteristics listed above are based on universal personality theories and are measurable with the techniques designed for multinational groups.

Technical Skill. A good supervisor should be at least as technically qualified as those he must manage. However, it is not unusual for a supervisor to rely on the expert knowledge of subordinates who are highly qualified in specific areas. The supervisor’s main task is to coordinate functions, make decisions, and insure that essential duties are performed. Therefore, the level of technical expertise required for a supervisor depends on the technical proficiency of his subordinates. For example, if a group lacks the necessary skills required to perform technical duties, then the supervisor must be highly qualified in order to provide guidance.

Motivation. A supervisor must be motivated toward or interested in the managerial role. Some individuals may possess strong leadership characteristics and be highly qualified technically, but may reject a supervisory position. An individual may feel more comfortable in the strict performance of technical duties for a variety of reasons. A qualified technician placed in a supervisor position against his desire will usually not function adequately. The result will be the acquisition of a poor supervisor and the loss of a good technician.

Reasoning ability. It is safe to assume that the individual with the most intellectual ability will function best as a supervisor, other factors being equal. A supervisor often encounters complex and unique situations that require logical analysis and sound judgment. In addition, the supervisor must be able to reason, associate abstract ideas, and plan for all contingencies. Therefore, basic intellectual ability is as fundamentally important as personal traits, technical skill, and motivation.

Determination of the feasibility of an assessment – evaluation plan requires the application of the design in an actual situation related to the overall program. It is necessary to apply the plan to a small population before general use in order to establish the validity and practicality of the total approach. Once the validity and viability of the program is established, the design can be used to screen and evaluate potential trainees in all functional areas.

A pilot study was conducted on students involved in basic technical training and advanced supervisor training. This population provided data for evaluation of the basic design, and also made possible comparative analysis across various skill and training levels.

The subjects for the present study were fifteen Vietnamese Navy (VNN) students involved in fire fighting training at the Vung Tau Training Center. Ten of the students were in their second week of the four week Fire Fighting (FF) course, and five students were in the second week of the Fire Officer (FO) course. The five FO Trainees had previously completed the FF course and were selected for advanced training on the basis of highest grades.

Tables 1 and 2 present a breakdown of pertinent demographic characteristics of the FF trainees and FO trainees respectively. The median age for both groups was 20.5 years. Educational level is about the same for both groups; the average education for the FF and FO trainees is 10.1 years and 10.0 years respectively. However, two exceptions with regard to education are important to note, i.e., trainee No. 10 with 6 years education and trainee No. 15 with 15 years of education.

Most of the trainees were not married and most were students prior to entering service. The exceptions to the latter are trainees No. 10 and No. 11. Only two students achieved a rank higher than Seaman level.

With the exception of trainee 15, both the FF and FO trainees represent a rather homogenous group.

A selection technique was developed based on the study objectives and technical supervisor concept. The basic design involved the following assessment battery:

Personal leadership-management characteristics. The Multicultural Preference Schedule (MPS) was used to identify individual characteristics related to leadership. The MPS has been effectively utilized in cross-cultural settings and has been established as a reliable measure of personality variables related to many areas of performance.

Technical skill. Assessment of individual technical skill and functional comprehension was based on formal training grades and school performance. Information regarding evaluation of actual on-the-job performance was not available, since students had not actually worked in the field prior to training assignment.

Motivation and attitude toward functional area. Assessment of each individual’s level of interest in and motivation toward supervisory and managerial positions was accomplished through an appropriately designed attitude scaling and direct measuring device. Although attitude and motivational assessment is a highly complex process, it is a very important aspect of selection. Moreover, the problem of motivation and attitude measurement is amplified by cultural factors. Therefore, items were designed to evoke a range of responses, through the use of direct and unobtrusive measures. Care was taken to insure proper cultural context, and to avoid interpretation errors related to social desirability of response and defensive bias.

Reasoning ability. The determination of basic intellectual ability was accomplished through modification of an existing culture-fair intelligence test, viz., the Digit Span Test (DST). Although this test measures a set of specific abilities, empirical studies reveal a high correlation between DST scores and general intellectual ability. Moreover, the DST test is appropriate for the present task. The specific skills measured by the DST are: visual acuity, motor coordination and speed, learning ability, concentration, stability under timed conditions, and approach to problem solving.

Demographic information. Finally, data were collected on each trainee’s general background, education, experience, and previous training. This data provided an information base which was used to determine group and individual significance related to areas of ability and/or performance success.

The above assessment battery represents an initial approach in the development of a screening/evaluation program. This approach is comprehensive in scope, however; comprehensiveness is essential in order to clarify problem areas and to determine which techniques and factors are most appropriate for the task involved.

The fifteen trainees were administered the test battery at the Vung Tau Training Center during their normal instruction period. Two Vietnamese instructors, the examiner, and an interpreter assistant were present during the administration of the battery.

All assessment techniques were administered in Vietnamese. No observable language or interpretation difficulties were encountered. The battery was administered in sequence. Total testing time was two hours and thirty minutes.

The results of the pilot study were better than expected. The findings for each category of measurement are given below.

Each Fire Officer candidate (FO) and each Fire Fighter candidate (FF) was assessed for personal characteristics through the use of Multicultural Preference Schedule (MPT). The candidates were then rated according to a leadership – supervisory criteria developed specifically for the pilot study and based on the concept outlined in Chapter One. An MPS pattern analysis was made for each candidate and score values were assigned based on significant mean variations.

Table 3 shows the results of supervisor assessment for the five Fire Officer candidates and the correlation between leadership scores, predicted success based on score values, and actual performance. The EPPS score values for the FO candidates range from six (6.0) through minus four (-4.0). Since adequate norms for Vietnamese leadership are not available, it is difficult to develop criteria for scores at this time. However, a Leadership-Supervisor score of six (6.0) is considered to indicate high average leadership capability based on U.S. standards. Moreover, a score falling in the negative range (-1.0 and below) is considered indicative of poor leadership potential. The leadership scores for the Vietnamese FO candidates therefore indicate relative potential when placed in rank order, i.e., the higher the score the more leadership capability the individual possesses. Whether a score of six (6.0) represents maximum leadership potential for the Vietnamese population will not be known until adequate standards are established based on large samples of the population. What is known at this point is, the highest score on the leadership scale indicates the highest potential of the particular group under study.

Of all the tests administered, the MPS leadership scale appears to be the best single predictor of supervisor training success. As Table 3 indicates, not only was it possible to predict failure or unsatisfactory performance, but the leadership scores (with the exception of one score), shows a direct relationship to class rank. The relationship between the leadership criteria and course performance is beyond chance expectation in light of the sample size and the fact that Vietnamese MPS norms have not been established. The candidate (No. 13) who did not satisfactorily complete the Fire Officer course, failed due to his inability to perform in a leadership capacity during practical exercises, not because of class work or written test scores. Candidate 13’s leadership score was –4.0 indicating very poor leadership potential. Therefore, candidate 13’s poor ability to function as a Fire Officer could have been predicted prior to training assignment.

The candidate (No. 14), who received a "marginal" score on the MPS Leadership Scale, satisfactorily completed the FO course. However, additional test results indicate that motivational factors may have been an important aspect of candidate 14’s success. The latter finding illustrates the importance of a comprehensive approach for selection. An assessment of candidate 14 with regard to motivation is covered below.

Since selection for supervisor training has previously been based on training grades or technical proficiency in the past, the leadership characteristics for the ten FF candidates will be covered under Technical Skill.

Technical skill for the Fire Fighter and Fire Officer trainees was measured by formal training grades and class performance only. The FO trainees were selected for advanced training based on classroom performance or grade point average (GPA) during the FF training course. Although technical skill, as measured by class performance and/or work performance, is an important factor for selection of individuals for supervisor training, there is some opportunity for error when using skill as a selector alone.

As revealed in Table 4 the FF trainees with the highest GPA did not necessarily score high on the MPS leadership scale. Therefore, some students, selected on the basis of grades alone will not function as effectively as Fire Officers as those selected by GPA and personal characteristics combined. This conclusion is supported by the results of FO candidates (Table 3) who were selected for advanced training based on their FF GPA alone. The FO trainee who scored "poor" on the Leadership Scale failed the FO course as indicated above.

Table 4 also shows that if five trainees of the present FF class are selected for advanced training based on GPA, their predicted success would be one (1) "Good", two (2) "Marginal", and two (2) "Poor". Furthermore, there is a considerable chance that the individuals rated "Poor" on the MPS leadership scale would not successfully complete the FO course, or would not function adequately as Fire Officers. High performance as a technician or student does not guarantee high performance as a supervisor.

A number of motivational items were included in the test battery. However, assessment of group and individual motivation involves a rather complex procedure due to the variable and intangible nature of motives. In addition, with the rather small group of trainees involved, it was difficult to determine significant relationships between responses and performance. Furthermore, assessment of motivational factors for selection of supervisors is best accomplished through validation against actual on-the-job performance. The only criteria available for estimation of motivation was class grades or school performance. A more valid assessment of motivation will be determined during follow-up studies. However, although many of the items will not be analyzed here due to the difficulties cited above, the results show some rather significant trends between responses to motivation items and performance.

Only two of the 15 trainees stated that achievement to the best of one’s ability was more important than maintaining social tradition. It can be assumed that these two trainees are somewhat more motivated to perform on the job than the rest of the group studied, since their attitude indicates a willingness to go beyond the boundaries of tradition in the interest of personal achievement. Both these men, candidates 15 and 14 were in the FO course and ranked first and second respectively in school performance.

With regard to attitude toward supervisory roles and functions, as measured by a combination of related items, candidates 14 and 15 again displayed the strongest positive attitude. Table 5 shows a comparison of FO candidates with regard to motivation, leadership ability, and performance.

The criteria for motivational assessment used in Table 5 were direct and indirect items. Again, due to the number of respondents, measurement is relative and reveals a differential of strong motivation or not strong motivation. The degree of weak or average motivation cannot adequately be determined at this time.

As Table 5 indicates, candidate 15 displayed a high motivation, scored highest on the MPS leadership scale, and achieved the highest class GPA. Candidate 14, on the other hand, displayed equally high motivation, scored "marginal" on the leadership scale, and achieved the second rank in class performance. The results of trainee 14 illustrate the importance of the motivational component of assessment. Although trainee 14 lacks some of the basic characteristics usually required for effective leadership, his high motivation appears to have overcome an inherent weakness. This is an excellent example of how motivation can offset the effects of limited personal development, and also illustrates the importance of motivational assessment as part of any selection program.

Another measure of motivation can be drawn from the results of the MPS leadership scale. One of the characteristics considered pertinent to leadership potential is an orientation toward achievement. With the exception of one candidate, all the trainees in the FF course who scored in the negative range on achievement orientation also scored in the negative range on leadership potential. Moreover, these trainees were in the bottom half of their graduating class. The lowest rated achiever ranked tenth in the class of ten. In the FO course the only negative achiever identified failed the course.

Although measures of motivation are rough at this point, the importance of motivational factors in any training or development program cannot be overemphasized. Without motivation there can be no success. An individual who is not motivated will not function effectively regardless of his intellectual ability, aptitude, personal characteristics, or technical skill level. What good is ability if it cannot or will not be applied? Furthermore, an unmotivated individual in a leadership role is not only a worthless entity in any program, but his behavior may infect the motivation of those in his charge.

Assessment of motivation seems rather elementary; however, due to the hidden qualities of attitudes and motives, adequate measurement requires sensitive techniques that go beyond the conscious awareness of respondents, and elicits the more prevailing predispositions of individuals or groups. In this respect, assessing motives is analogous to fishing. Effective fishing requires the use of specific lures for certain fish at various depths.

Finally, motivational assessment is important for group development programs as well as for the selection of individuals most likely to succeed in training. It is believed that anyone can be motivated. Once we discover what motivates individuals or groups, it is possible to increase interest through the use of incentive and generate optimum performance.

The results of intelligence testing were unsatisfactory due to procedural and translation difficulties encountered. In addition, all the materials were not available at the time of testing and no valid conclusions can be drawn at this time. It was obvious that some of the students did not understand the instructions as they were presented. Misinterpretation of instructions led to recognizable performance errors. Since the test was timed, there was no way to rectify the situation during this test session.

The purpose of a pilot study is to identify such procedural difficulties, and determine the feasibility of techniques before a large program of evaluation is launched. Since a few of the trainees were able to respond effectively to the reasoning ability test and, their results compared favorably to U.S. I.Q. standards, it appears that the Digit Span ability test is appropriate for Vietnamese trainees. However, modification of the instrument is necessary for future use.

The relationship between demographic data, leadership ability, and class performance is presented in Table 6. Again, while the group under study was small, some rather significant trends did emerge from the results.

Age. Since most of the trainees fall in the same age category (18 to 21 years) no relationship was found between this age range and leadership potential or class performance. However, candidate 15, who falls in the over 25 age category (27 years), displayed the highest leadership potential of all the trainees combined. In addition, trainee 15 ranked first in class performance in the FO course. These findings indicate that age (or rather maturity) may be an important factor for selection, if the individual’s age is significantly greater than the group average. Everything else being equal, and in consideration of cultural expectations, the individual who is three or four years older than the group should be selected for advanced training.

Rank. As Table 6 reveals there is a negative correlation between rank and leadership potential with this group of trainees. However, since only two of the trainees achieved rank above seaman (No. 1 and No. 6), and since the highest ranking trainee (No. 1) scored lowest in overall leadership potential, the results may be skewed due to small group analysis, and may lead to false conclusions at this time. Without adequate rank differential within the group and large samples some distortions are possible. Furthermore, there is no correlation between military rank and age in this group.

These results suggest that caution should be exercised when selecting individuals for advanced training based on rank alone. Some individuals may have achieved rank through means other than ability or time-in-grade.

Education. Within the FO group, the candidate (No. 15) whose educational background is advanced beyond high school level (15 years of formal school) ranked first in both leadership potential and class performance. The latter indicates a positive relationship between supervisory ability and educational level, if the educational difference within the group is considerable, i.e., three years of college compared to high school level.

However, within the FF group, the trainee (No. 10) with the lowest educational level (only 6 years of formal school) scored high on the MPS leadership scale and achieved the highest GPA. This reverse trend indicates either the results are skewed due to an unusual subject in a small group analysis, or that formal education is not as important for selection of supervisors as personal characteristics, skill level, and motivation.

Marital Status. With regard to marital status, only three of the 15 trainees were married at the time of testing. All the married trainees scored in the negative range on the MPS leadership scale, and two of these ranked below the 50^th percentile in class performance. This finding appears opposite from expected, since married men are usually considered more mature and stable than single men.

Analysis of specific personality characteristics of the three married trainees reveals that all display a very strong need for sympathy, encouragement, and affection, and all exhibit a tendency to avoid responsibility. Furthermore, the married trainees indicated a low desire to make decisions and to supervise others. This analysis suggests that these trainees chose marriage for emotional support and possibly dominance by their respective mates. The fact that motivation for marriage may be generated by relatively weak personality characteristics is not distinctive of any particular culture. This is especially true if marriage occurs at relatively young age.

The above results emphasize the fact that marital status, maturity, and leadership ability are not necessarily related, and technicians should not be selected for supervisor training based on marital status alone.

Combined Results. The overall findings of this pilot study reveal that the unsatisfactory performance of one of the five FO candidates could have been predicted prior to training assignment. In addition, the results of the FF candidates indicate that at least two of the students would prove unsatisfactory as Fire Officers if selected for advanced training based on GPA alone; the criteria previously used for selection. The predicted attrition rate based on these results would be one in five.

Using hypothetical figures, assuming the total cost of training one supervisor candidate in a four week course is $500.00 (considering salaries, equipment, and facilities) a total of $2,000.00 per month could be saved for every 20 man training effort. In a six month period, an appropriate selection program would result in a savings of $12,000.00.

The present study was an attempt to develop an appropriate selection-evaluation test battery for use with Vietnamese personnel involved in PAE’s technical supervisor training program. This initial approach was comprehensive in scope and based on a universal concept of leadership. Included in the selection battery were a number of factors related to effective performance of individuals who will be trained as technical supervisors in any functional area. Therefore, although this pilot study was conducted on Fire Fighter and Fire Officer candidates, the results have implications for power plant, engineering, and maintenance trainees as well. Furthermore, since this approach was based on universal concepts, the same method can be applied to all cultures to provide the necessary information for prediction of training success. All managerial and supervisor positions require the same basic characteristics for effective performance. These basic characteristics are: technical ability, leadership potential, reasoning ability, and motivation. Each of the above factors is considered of equal importance when considering candidates for advance training.

The purpose of a pilot study is to develop viable procedures, identify significant factors related to the objectives of training programs, and to develop hypotheses for further study. The present study yielded considerable data with regard to all aspects of supervisor success. However, the initial selection technique must be evaluated against adequate operational criteria. Supervisor trainees must be evaluated by follow-up studies during functional performance in the field to determine the predictive validity of the results, and to identify which factors are most significant and should receive primary consideration for future selection programs.

Field performance results will be correlated with initial testing results to establish criteria for future manager-supervisor selection and training programs. The findings of initial selection and evaluation programs will indicate which factors are most significant for selection. The final selection technique will be based on significant characteristics validated against successful performance in the supervisory role. Furthermore, information gained through the supervisor evaluation program will identify specific training problems and will be used to facilitate curriculum development. Figure 1 presents a diagram of the total selection-evaluation program.

Figure 1

RVNAF Supervisor Selection – Evaluation Program

The success of any selection program depends on the population from which candidates are selected. Therefore, it may not be possible to select individuals with optimum supervisor potential. The quality of the individuals selected is relative to the quality of the group from which they will be selected. What is possible, however; is selection of the best qualified individuals from the available group. If selection is adequate, than we can be assured of the best possible results from training efforts. Finally, when the available group is generally lacking in ability due to limited development, then the quality and value of selection becomes a considerable importance.

The findings of the present study support the following general conclusions:

1. The techniques utilized for assessment, based on universal concepts, are appropriate for Vietnamese personnel for selection and prediction of successful supervisor performance.
2. Selection of trainees for advanced or supervisor training must be based on a comprehensive or "whole person" concept related to the specific functions of the particular task involved. Using single factors, such as, previous training grades as a selection criteria may lead to unsatisfactory results and lost training effort.
3. All the areas of assessment seem important for supervisor selection, viz., personal characteristics, related leadership ability, technical skill, training grades, intelligence, and motivation. However, at present, personal characteristics or leadership potential appear to be the best single predictor of supervisor success.
4. A continuing program of data collection must be conducted to evaluate the results against operational criteria and establish significant relationships for future selection programs.
5. Through the use of an appropriately designed selection and evaluation program it is possible to decrease training time, increase training effectiveness, and decrease training costs.