A performance appraisal, also referred to as a performance review, performance evaluation,[1] (career) development discussion,[2] or employee appraisal, sometimes shortened to "PA",[lower-alpha 1] is a periodic and systematic process whereby the job performance of an employee is documented and evaluated. This is done after employees are trained about work and settle into their jobs. Performance appraisals are a part of career development and consist of regular reviews of employee performance within organizations.

Performance appraisals are most often conducted by an employee's immediate manager or line manager.[3] While extensively practiced, annual performance reviews have also been criticized [4] as providing feedback too infrequently to be useful, and some critics argue that performance reviews in general do more harm than good. It is an element of the principal-agent framework, that describes the relationship of information between the employer and employee, and in this case the direct effect and response received when a performance review is conducted.[5]

Main features

A performance appraisal is a systematic, general and periodic process that assesses an individual employee's job performance and productivity in relation to certain pre-established criteria and organizational objectives.[6][7] Other aspects of individual employees are considered as well, such as organizational citizenship behavior, accomplishments, potential for future improvement, strengths and weaknesses, etc.[6][8][9]

To collect PA data, there are three main methods: objective production, personnel, and judgmental evaluation. Judgmental evaluations are the most commonly used with a large variety of evaluation methods.[1] Historically, PA has been conducted annually (long-cycle appraisals); however, many companies are moving towards shorter cycles (every six months, every quarter), and some have been moving into short-cycle (weekly, bi-weekly) PA.[10][11] The interview could function as "providing feedback to employees, counseling and developing employees, and conveying and discussing compensation, job status, or disciplinary decisions".[10] PA is often included in performance management systems. PA helps the subordinate answer two key questions: first, "What are your expectations of me?" second, "How am I doing to meet your expectations?"[12]

Performance management systems are employed "to manage and align" all of an organization's resources in order to achieve highest possible performance[1] and to eliminate distractions procured from individual agents that neglect the companies goals.[5] "How performance is managed in an organization determines to a large extent the success or failure of the organization. Therefore, improving PA for everyone should be among the highest priorities of contemporary organizations".[13]

Some applications of PA are compensation, performance improvement, promotions, termination, test validation, and more.[14] While there are many potential benefits of PA, there are also some potential drawbacks. For example, PA can help facilitate management-employee communication; however, PA may result in legal issues if not executed appropriately, as many employees tend to be unsatisfied with the PA process, as well as, the misuse of PA's can incur apathy towards organizational goals and values.[1][15][16] PAs created in and determined as useful in the United States are not necessarily able to be transferable cross-culturally.[17]

Applications of results

A central reason for the utilization of performance appraisals (PAs) is performance improvement ("initially at the level of the individual employee, and ultimately at the level of the organization").[14] Other fundamental reasons include "as a basis for employment decisions (e.g. promotions, terminations, transfers), as criteria in research (e.g. test validation), to aid with communication (e.g. allowing employees to know how they are doing and organizational expectations), to establish personal objectives for training" programs, for transmission of objective feedback for personal development, "as a means of documentation to aid in keeping track of decisions and legal requirements"[14] and in wage and salary administration.[1] Additionally, PAs can aid in the formulation of job criteria and selection of individuals "who are best suited to perform the required organizational tasks".[6] An appraisal can be part of guiding and monitoring employee career development.[18] PAs can also be used to aid in work motivation through the use of reward systems.[6]

Potential benefits

There are a number of potential benefits of organizational performance management conducting formal performance appraisals (PAs). There has been a general consensus in the belief that PAs lead to positive implications of organizations.[19] Furthermore, PAs can benefit an organization's effectiveness.[18] One way is PAs can often lead to giving individual workers feedback about their job performance.[15] From this may spawn several potential benefits such as the individual workers becoming more productive.[20]

Other potential benefits include:

  • Facilitation of communication: communication in organizations is considered an essential function of worker motivation.[15] It has been proposed that feedback from PAs aid in minimizing employees’ perceptions of uncertainty.[18] Fundamentally, feedback and management-employee communication can serve as a guide in job performance.[15]
  • Enhancement of employee focus through promoting trust: behaviors, thoughts, and/or issues may distract employees from their work, and trust issues may be among these distracting factors.[21] Such factors that consume psychological energy can lower job performance and cause workers to lose sight of organizational goals.[15] Properly constructed and utilized PAs have the ability to lower distracting factors and encourage trust within the organization.[22]
  • Goal setting and desired performance reinforcement: organizations find it efficient to match individual worker's goals and performance with organizational goals.[15] PAs provide room for discussion in the collaboration of these individual and organizational goals.[23] Collaboration can also be advantageous by resulting in employee acceptance and satisfaction of appraisal results.[24]
  • Performance improvement: well constructed PAs can be valuable tools for communication with employees as pertaining to how their job performance stands with organizational expectations.[18] "At the organizational level, numerous studies have reported positive relationships between human resource management (HRM) practices"[15] and performance improvement at both the individual and organizational levels.
  • Determination of training needs: “Employee training and development are crucial components in helping an organization achieve strategic initiatives”.[15][25] It has been argued that for PAs to truly be effective, post-appraisal opportunities for training and development in problem areas, as determined by the appraisal, must be offered.[26] PAs can be especially instrumental for identifying training needs of new employees.[9] Finally, PAs can help in the establishment and supervision of employees’ career goals.[18]

Potential complications

Despite all the potential advantages of formal performance appraisals (PAs), there are also potential drawbacks. It has been noted that determining the relationship between individual job performance and organizational performance can be a difficult task.[25] The ones conducting performance appraisals, such as line managers also often face complexities (Tyskbo, 2020). Generally, there are two overarching problems from which several complications spawn. One of the problems with formal PAs is there can be detrimental effects to the organization(s) involved if the appraisals are not used appropriately. The second problem with formal PAs is they can be ineffective if the PA system does not correspond with the organizational culture and system.[15]

Potential complications that may arise:

  • Detrimental impact to performance improvement: It has been proposed that the use of PA systems in organizations adversely affects organizations’ pursuits of quality performance.[27] It is believed by some scholars and practitioners that the use of PAs is unnecessary if there is total quality management.[23]
  • Subjective evaluations: Subjectivity is related to judgement based on a supervisor's subjective impressions and opinions, which can be expressed through the use of subjective performance measures, ex post flexibility in the weighting of objective performance measures, or ex post discretional adjustment, all of which are based on factors other than performance measures specified ex ante. Traditional performance appraisals are often based upon a manager's or supervisor's perceptions of an employee's performance and employees are evaluated subjectively rather than objectively. Therefore, the review may be influenced by many non-performance factors such as employee 'likeability', personal prejudices, ease of management, and/or previous mistakes or successes. Reviews should instead be based on data-supported, measurable behaviors and results within the performers control.[28]
  • Negative perceptions: "Quite often, individuals have negative perceptions of PAs".[19] Receiving and/or the anticipation of receiving a PA can be uncomfortable and distressful[18] and potentially cause "tension between supervisors and subordinates".[20] If the person being appraised does not trust their employer, appraiser or believe that they will benefit from the process it may become a "tick box" exercise.[29]
  • Central Tendency: This is where the evaluator fails to make extreme ratings to either direction-low or high but remains at the intermediate. The evaluator fails to use extremes of the scale and uses the central points e.g. rating all employees as average.[30]
  • Inflationary pressure: this is where there is low differentiation with the upper range of the rating choices defining outstanding performance as 90 or above good as 90 or above, average or above 70 or above and inadequate performance as anything below 70 leaves one wonder for a promotion.[31]
  • Errors: Performance appraisals should provide accurate and relevant ratings of an employee's performance as compared to pre-established criteria/goals (i.e. organizational expectations).[32] Nevertheless, supervisors will sometimes rate employees more favorably than that of their true performance in order to please the employees and avoid conflict.[15] "Inflated ratings are a common malady associated with formal" PA.[33]
  • Legal issues: when PAs are not carried out appropriately, legal issues could result that place the organization at risk.[20] PAs are used in organizational disciplinary programs[18] as well as for promotional decisions within the organization.[15] The improper application and utilization of PAs can affect employees negatively and lead to legal action against the organization.
  • Performance goals: performance goals and PA systems are often used in association. Negative outcomes concerning the organizations can result when goals are overly challenging or overemphasized to the extent of affecting ethics, legal requirements, or quality.[34] Moreover, challenging performance goals can impede an employees’ abilities to acquire necessary knowledge and skills.[21] Especially in the early stages of training, it would be more beneficial to instruct employees on outcome goals than on performance goals.[15]
  • Derail merit pay or performance-based pay: some researchers contend that the deficit in merit pay and performance-based pay is linked to the fundamental issues stemming from PA systems.[26]

Improvements

Although performance appraisals can be biased, there are certain steps that can be taken to improve the evaluations and reduce the margin of errors through the following:

  • Training - Bringing awareness to the potential for bias by training the Evaluators to be aware of the difference of skills and abilities between employees and how to subjectively consider these traits.
  • Providing Feedback to Raters - Trained raters provide managers who evaluated their subordinates with feedback, including information on ratings from other managers. This has the potential to reduce leniency errors.
  • Subordinate Participation - By allowing employee participation in the evaluation process, there is employee-supervisor reciprocity in the discussion for any discrepancies between self ratings and supervisor ratings, thus, increasing job satisfaction and motivation.[35]
  • Use multiple raters to avoid the likely biasness to be found with using only one rater.
  • Conduct post appraisal interviews. interview employees after appraisal to get their comments, views and opinions on the whole exercise and general performance.
  • Use selective rating. Use people as raters on areas where they have job knowledge since no single person is knowledgeable in all areas.
  • Reward accurate appraisers. It is good practice to reward performance and hence it means those who appraise accurately have performed and should be rewarded.[36]

Effectiveness

Leadership development coach Jack Zenger urges companies to find alternatives to annual performance reviews, and says that research supports the following:[37]

  • Frequent discussions with employees are better than annual reviews
  • Talking about future goals is more productive than past performance, especially with clear targets, deadlines, and the participation of the employee
  • Negative feedback can cause defensiveness and worsen productivity
  • Positive feedback does little to improve productivity, though does improve the interpersonal relationship with the person giving the feedback
  • Neither managers nor employees like performance reviews
  • Higher-level employees receive performance reviews less frequently
  • Annual reviews are often justified on the grounds they are needed for salary changes, but they are not actually necessary, and empirically they would make little difference for most employees

Seniority and labor contracts

Labor union contracts sometimes specify that promotions, layoffs, and various perks are assigned in order of seniority, rather than based on performance. Historically, this was one means to prevent cronyism, nepotism, and corruption, and could also be seen as a form of solidarity. If employers have a reliable way to distinguish productive from unproductive workers, then firing the worst employees and hiring replacements would be one way to increase the overall productivity of the firm, and possibly increase profits or lower consumer prices. Some labor contracts specify a way to take merit into account when firing or promoting, often involving performance reviews. For example, union rules might require an employer to warn a poorly performing employee and have a probationary period before firing. The records generated by performance reviews might be consulted during binding arbitration to decide whether a firing was justified.

Resistance from managers

Managers who have had unsatisfactory experiences with inadequate or poorly designed appraisal programs may be skeptical about their usefulness.

  • Some managers may not like to play the role of a judge and be responsible for the future of their subordinates.
  • They may be uncomfortable about providing negative feedback to the employees.
  • This tendency can lead them to inflate their assessments of the workers' job performance, giving higher ratings than deserved.

Conducting

Human resource management (HRM) conducts performance management. Performance management systems consist of the activities and/or processes embraced by an organization in anticipation of improving employee performance, and therefore, organizational performance.[38] Consequently, performance management is conducted at the organizational level and the individual level. At the organizational level, performance management oversees organizational performance and compares present performance with organizational performance goals.[26] The achievement of these organizational performance goals depends on the performance of the individual organizational members.[26] Therefore, measuring individual employee performance can prove to be a valuable performance management process for the purposes of HRM and for the organization.[26] Many researchers would argue that "performance appraisal is one of the most important processes in Human Resource Management".[16]

The performance management process begins with leadership within the organization creating a performance management policy.[26] Primarily, management governs performance by influencing employee performance input (e.g. training programs) and by providing feedback via output (i.e. performance assessment and appraisal).[39] "The ultimate objective of a performance management process is to align individual performance with organizational performance".[40] A very common and central process of performance management systems is performance appraisal (PA).[26] The PA process should be able to inform employees about the "organization's goals, priorities, and expectations and how well they are contributing to them".[40]

When they are conducted

Performance appraisals (PAs) are conducted at least annually,[41] and annual employee performance reviews appear to be the standard in most American organizations.[10] However, "it has been acknowledged that appraisals conducted more frequently (more than once a year) may have positive implications for both the organization and employee."[15] It is suggested that regular performance feedback provided to employees may quell any unexpected and/or surprising feedback to year-end discussions.[16] In a recent research study concerning the timeliness of PAs, "one of the respondents even suggested that the performance review should be done formally and more frequently, perhaps once a month, and recorded twice a year."[16]

Other researchers propose that the purpose of PAs and the frequency of their feedback are contingent upon the nature of the job and characteristics of the employee.[42] For example, employees of routine jobs where performance maintenance is the goal would benefit sufficiently from annual PA feedback. On the other hand, employees of more discretionary and non-routine jobs, where goal-setting is appropriate and there is room for development, would benefit from more frequent PA feedback. Non formal performance appraisals may be done more often, to prevent the element of surprise from the formal appraisal.[10][42][43]

Methods of collecting data

There are three main methods used to collect performance appraisal (PA) data: objective production, personnel, and judgmental evaluation. Judgmental evaluations are the most commonly used with a large variety of evaluation methods.[8]

Objective production

The objective production method consists of direct, but limited, measures such as sales figures, production numbers, the electronic performance monitoring of data entry workers, etc.[8] The measures used to appraise performance would depend on the job and its duties. Although these measures deal with unambiguous criteria, they are usually incomplete because of criterion contamination and criterion deficiency. Criterion contamination refers to the part of the actual criteria that is unrelated to the conceptual criteria.[8] In other words, the variability in performance can be due to factors outside of the employee's control. Criterion deficiency refers to the part of the conceptual criteria that is not measured by the actual criteria.[8] In other words, the quantity of production does not necessarily indicate the quality of the products. Both types of criterion inadequacies result in reduced validity of the measure.[8] Regardless of the fact that objective production data is not a complete reflection upon job performance, such data is relevant to job performance.

Happy-productive worker hypothesis

The happy-productive worker hypothesis states that the happiest workers are the most productive performers, and the most productive performers are the happiest workers.[44] Yet, after decades of research, the relationship between job satisfaction and job performance produces only a weak positive correlation. Published in 2001 by Psychological Bulletin, a meta-analysis of 312 research studies produced an uncorrected correlation of 0.18.[45] This correlation is much weaker than what the happy-productive worker hypothesis would predict.

Personnel

The personnel method is the recording of withdrawal behaviors (i.e. absenteeism, accidents). Most organizations consider unexcused absences to be indicators of poor job performance, even with all other factors being equal;[44] however, this is subject to criterion deficiency. The quantity of an employee's absences does not reflect how dedicated he/she may be to the job and its duties. Especially for blue-collar jobs, accidents can often be a useful indicator of poor job performance,[8] but this is also subject to criterion contamination because situational factors also contribute to accidents. Once again, both types of criterion inadequacies result in reduced validity of the measure.[8] Although excessive absenteeism and/or accidents often indicate poor job performance rather than good performance, such personnel data is not a comprehensive reflection of an employee's performance.[8]

Judgmental evaluation

An employee filling in an evaluation form

Judgmental evaluation appears to be a collection of methods, and as such, could be considered a methodology. A common approach to obtaining PAs is by means of raters.[1] Because the raters are human, some error will always be present in the data. The most common types of error are leniency errors, central tendency errors, and errors resulting from the halo effect.[1] Halo effect is characterized by the tendency to rate a person who is exceptionally strong in one area higher than deserved in other areas. It is the opposite of the Horns effect, where a person is rated as lower than deserved in other areas due to an extreme deficiency in a single discipline.[46] These errors arise predominantly from social cognition and the theory in that how we judge and evaluate other individuals in various contexts is associated with how we "acquire, process, and categorize information".[1]

An essential piece of this method is rater training. Rater training is the "process of educating raters to make more accurate assessments of performance, typically achieved by reducing the frequency of halo, leniency, and central-tendency errors".[1] Rater training also helps the raters "develop a common frame of reference for evaluation" of individual performance.[47] Many researchers and survey respondents support the ambition of effectual rater training.[16] However, it is noted that such training is expensive, time-consuming, and only truly functional for behavioral assessments.[16]

Another piece to keep in mind is the effects of rater motivation on judgmental evaluations. It is not uncommon for rating inflation to occur due to rater motivation (i.e. "organizationally induced pressures that compel raters to evaluate ratees positively").[1] Typically, raters are motivated to give higher ratings because of the lack of organizational sanction concerning accurate/inaccurate appraisals, the rater's desire to guarantee promotions, salary increases, etc., the rater's inclination to avoid negative reactions from subordinates, and the observation that higher ratings of the ratees reflect favorably upon the rater.[1]

The main methods used in judgmental performance appraisal are:[1]

  • Graphic Rating Scale: graphic rating scales are the most commonly used system in PA.[1] On several different factors, subordinates are judged on 'how much' of that factor or trait they possess. Typically, the raters use a 5- or 7-point scale; however, there are as many as 20-point scales.[1]
  • Employee-Comparison Methods: rather than subordinates being judged against pre-established criteria, they are compared with one another. This method eliminates central tendency and leniency errors but still allows for halo effect errors to occur.[1] The rank-order method has raters ranking subordinates from "best" to "worst", but how truly good or bad one is on a performance dimension would be unknown.[1] The paired-comparison method requires the rater to select the two "best" subordinates out of a group on each dimension then rank individuals according to the number of times each subordinate was selected as one of the "best".[1] The forced-distribution method is good for large groups of ratees. The raters evaluate each subordinate on one or more dimensions and then place (or "force-fit", if you will) each subordinate in a 5 to 7 category normal distribution.[1] The method of top-grading can be applied to the forced distribution method.[48] This method identifies the 10% lowest performing subordinates, as according to the forced distribution, and dismisses them leaving the 90% higher performing subordinates.
  • Behavioral Checklists and Scales: behaviors are more definite than traits. The critical incidents method (or critical incident technique) concerns "specific behaviors indicative of good or bad job performance".[1] Supervisors record behaviors of what they judge to be job performance relevant, and they keep a running tally of good and bad behaviors. A discussion on performance may then follow. The behaviorally anchored rating scales (BARS) combine the critical incidents method with rating scale methods by rating performance on a scale but with the scale points being anchored by behavioral incidents.[1] Note that BARS are job specific. In the behavioral observation scale (BOS) approach to performance appraisal, employees are also evaluated in the terms of critical incidents. In that respect, it is similar to BARS. However, the BOS appraisal rate subordinates on the frequency of the critical incidents as they are observed to occur over a given period. The ratings are assigned on a five-point scale. The behavioral incidents for the rating scale are developed in the same way as for BARS through identification by supervisors or other subject matter experts. Similarly, BOS techniques meet equal employment opportunity because they are related to actual behavior required for successful job performance.[49]

The frequency of an evaluation, and policies concerning them, varies widely from workplace to workplace. Sometimes, an evaluation will be given to a new employee after a probationary period lapses, after which they may be conducted on a regular basis (such as every year). According to the 2014 Performance Management survey, 96% of employers perform annual performance evaluations and 44% of employers perform a 90-day performance review for new employees.[50]

Peer and self assessments

While assessment can be performed along reporting relationships (usually top-down), net assessment can include peer and self-assessment. Peer assessment is when assessment is performed by colleagues along both horizontal (similar function) and vertical (different function) relationship. Self-assessments are when individuals evaluate themselves.[1] There are three common methods of peer assessments. Peer nomination involves each group member nominating who he/she believes to be the "best" on a certain dimension of performance. Peer ratings has each group member rate each other on a set of performance dimensions. Peer ranking requires each group member rank all fellow members from "best" to "worst" on one or more dimensions of performance.

  • Self-assessments: for self-assessments, individuals assess and evaluate their own behavior and job performance.[1]
  • Peer assessments: members of a group evaluate and appraise the performance of their fellow group members.[1] There it is common for a graphic rating scale to be used for self-assessments. Positive leniency tends to be a problem with self-assessments.[8] Peer assessments from multiple members of a group are often called crowd-based performance reviews, and solve many problems with peer assessments from only one member.[51]
  • 360-degree feedback: 360-degree feedback is multiple evaluations of employees which often include assessments from superior(s), peers, and one's self.[1]
  • Negotiated performance appraisal: The negotiated performance appraisal (NPA) is an emerging approach for improving communication between supervisors and subordinates and for increasing employee productivity, and may also be adapted to an alternate mediation model for supervisor-subordinate conflicts. A facilitator meets separately with the supervisor and with the subordinate to prepare three lists. What employees do well, where the employee has improved in recently, and areas where the employee still needs to improve. Because the subordinate will present his or her lists first during the joint session, this reduces defensive behaviors. Furthermore, the subordinate comes to the joint session not only prepared to share areas of needed improvement, but also brings concrete ideas as to how these improvements can be made. The NPA also focuses very strongly on what employees are doing well, and involves a minimum of twenty minutes of praise when discussing what the employee does well. The role of the facilitator is that of a coach in the pre-caucuses, and in the joint sessions the supervisor and subordinate mostly speak to each other with little facilitator interference.[52][53]

In general, optimal PA process involves a combination of multiple assessment modalities. One common recommendation is that assessment flows from self-assessment, to peer-assessment, to management assessment - in that order. Starting with self-assessment facilitates avoidance of conflict. Peer feedback ensures peer accountability, which may yield better results than accountability to management. Management assessment comes last for need of recognition by authority and avoidance of conflict in case of disagreements. It is generally recommended that PA is done in shorter cycles to avoid high-stakes discussions, as is usually the case in long-cycle appraisals.

Research has shown that the source of the feedback (either manager or peer) does not matter in influencing employees' subsequent innovative or extra-role behaviors after the feedback is received.[54] As long as the feedback is provided, the source does not matter.[55]

Principal - Agent Framework

The Principal-agent framework is a model describing the relationship of information held between an employer and an employee. It is used to forecast responses from employees and strategies at finding resolutions against misaligned incentives that interfere with the goals of the employer. The model makes two assumptions: the principals wants agents to work for the principal's best interest, but agents possess different goals than the principals; and, the agents have more information than the principals resulting in the asymmetry of information between the two parties. This paradigm creates adverse selections and moral hazards for the hiring company in deciding how to effectively minimize the potential threat of shirking; disruption to daily operations; and loss in output margins due to actions of the employee.[5]

Incentive Conflict Resolutions

Incentive pay leads to the increase of agents awareness of their own actions and seek to maximize their pay by considering the best possible actions that can be taken for the success of the firm and actively explore several options to minimize opportunity costs. The issue with this form of resolution is the firm must compensate the agents for bearing a risk premium and inequitable pay.

Fixed payment ensures a safer, standardized mode of contract that delivers reassurance in spite of performance fluctuations and external environment volatility. However, lack of motivation occurs more readily and incurs shirking and adverse selections.[56]

Organizational citizenship behavior

Also referred to as contextual behavior, prosocial behavior, and extra-role behavior, organizational citizenship behavior (OCB) consists of employee behavior that contributes to the welfare of the organization but is beyond the scope of the employee's job duties.[8] These extra-role behaviors may help or hinder the attainment of organizational goals. Research supports five dimensions of OCB: altruism, conscientiousness, courtesy, sportsmanship, and civic virtue.[57] Researchers have found that the OCB dimensions of altruism and civic virtue can have just as much of an impact on manager's subjective evaluations of employees’ performances as employees’ objective productivity levels.[58] The degree to which OCB can influence judgments of job performance is relatively high. Controversy exists as to whether OCB should be formally considered as a part of performance appraisal (PA).

Interviews

The performance appraisal (PA) interview is typically the final step of the appraisal process.[1] The interview is held between the subordinate and supervisor. The PA interview can be considered of great significance to an organization's PA system.[10] It is most advantageous when both the superior and subordinate participate in the interview discussion and establish goals together.[1] Three factors consistently contribute to effective PA interviews: the supervisor's knowledge of the subordinate's job and performance in it, the supervisor's support of the subordinate, and a welcoming of the subordinate's participation.[10] The objective of performance appraisal is to assess the training development needs of employees.

Employee reactions

Numerous researchers have reported that many employees are not satisfied with their performance appraisal (PA) systems.[16] Studies have shown that subjectivity as well as appraiser bias is often a problem perceived by as many as half of employees.[16] Appraiser bias, however, appears to be perceived as more of a problem in government and public sector organizations.[16] Also, according to some studies, employees wished to see changes in the PA system by making "the system more objective, improving the feedback process, and increasing the frequency of review."[16] In light of traditional PA operation defects, "organizations are now increasingly incorporating practices that may improve the system. These changes are particularly concerned with areas such as elimination of subjectivity and bias, training of appraisers, improvement of the feedback process and the performance review discussion."[16]

According to a meta-analysis of 27 field studies, general employee participation in his/her own appraisal process was positively correlated with employee reactions to the PA system.[24] More specifically, employee participation in the appraisal process was most strongly related to employee satisfaction with the PA system.[24] Concerning the reliability of employee reaction measures, researchers have found employee reaction scales to be sound with few concerns through using a confirmatory factor analysis that is representative of employee reaction scales.[59]

Researchers suggest that the study of employees' reactions to PA is important because of two main reasons: employee reactions symbolizes a criterion of interest to practitioners of PAs and employee reactions have been associated through theory to determinants of appraisal acceptance and success.[59] Researchers translate these reasons into the context of the scientist-practitioner gap or the "lack of alignment between research and practice."[59]

Schultz & Schultz notes that opposition to performance appraisals generally don't receive positive ratings from anyone involved. "So employees that will be directly affected by the Performance Appraisals are less than enthusiastic about participating in them". When an employee knows that their work performance has been less than perfect it's nerve-racking to be evaluated. Employees tend to be hostile knowing they could be given bad news on their performance.[60]

Most managers prefer to begin with positive information and then add bad news or suggestions for improvement at the end. However, employees are most satisfied when bad news is addressed early in the interview and positive information is saved until the end, so that the meeting ends with a positive feeling.[61]

Hidden Cost of Control

While performance appraisals are fundamental in the assessment of employees, frequent testing can result in the deterioration of employee performance, thus impacting overall business operations. The agent's perception of these 'control' devices are that they signal mistrust to the individual and reduce working autonomy. If these management practices are employed without consideration of the emotional response to said devices, then the agent's willingness to engage in the company's ambitions are greatly reduced as suggested in empirical studies.[62]

There are federal laws addressing fair employment practices, and this also concerns performance appraisal (PA). Discrimination can occur within predictions of performance and evaluations of job behaviors.[1] The revision of many court cases has revealed the involvement of alleged discrimination which was often linked to the assessment of the employee's job performance.[63] Some of the laws which protect individuals against discrimination are "the Title VII of the Civil Rights Act of 1964, the Civil Rights Act of 1991, the Age Discrimination in Employment Act (ADEA), and the Americans with Disabilities Act (ADA)."[1] Lawsuits may also results from charges of an employer's negligence, defamation, and/or misrepresentation.[1] A few appraisal criteria to keep in mind for a legally sound PA is to keep the content of the appraisal objective, job-related, behavior-based, within the control of the ratee, and related to specific functions rather than a global assessment.[63] Some appraisal procedure suggestions for a legally sound PA is to standardize operations, communicate formally with employees, provide information of performance deficits and give opportunities to employees to correct those deficits, give employees access to appraisal results, provide written instructions for the training of raters, and use multiple, diverse and unbiased raters.[63] These are valuable but not exhaustive lists of recommendations for PAs.

The Employment Opportunity Commission (EEOC) guidelines apply to any selection procedure that is used for making employment decisions, not only for hiring, but also for promotion, demotion, transfer, layoff, discharge, or early retirement. Therefore, employment appraisal procedures must be validated like tests or any other selection device. Employers who base their personnel decisions on the results of a well-designed performance review program that includes formal appraisal interviews are much more likely to be successful in defending themselves against claims of discrimination.[64]

Cross-cultural implications

Performance appraisal (PA) systems, and the premises of which they were based, that have been formed and regarded as effective in the United States may not have the transferability for effectual utilization in other countries or cultures, and vice versa.[17] Performance "appraisal is thought to be deeply rooted in the norms, values, and beliefs of a society".[65] "Appraisal reflects attitudes towards motivation and performance (self) and relationships (e.g. peers, subordinates, supervisors, organization), all of which vary from one country to the next".[66] Therefore, appraisal should be in conjunction with cultural norms, values, and beliefs in order to be operative.[67] The deep-seated norms, values and beliefs in different cultures affect employee motivation and perception of organizational equity and justice. In effect, a PA system created and considered effectual in one country may not be an appropriate assessment in another cultural region.[66]

For example, some countries and cultures value the trait of assertiveness and personal accomplishment while others instead place more merit on cooperation and interpersonal connection. Countries scoring high on assertiveness consider PA to be a way of assuring equity among employees so that higher performing employees receive greater rewards or higher salaries.[66] Countries scoring low on assertiveness but higher in interpersonal relations may not like the social separation and pay inequity of higher/lower performing employees; employees from this more cooperative rather than individualistic culture place more concern on interpersonal relationships with other employees rather than on individual interests.[66] High assertive countries value performance feedback for self-management and effectiveness purposes while countries low in assertiveness view performance feedback as "threatening and obtrusive".[66][68] In this case, the PA of the high assertive countries would likely not be beneficial for countries scoring lower in assertiveness to employ. However, countries scoring lower in assertiveness could employ PA for purposes of improving long-term communication development within the organization such as clarifying job objectives, guide training and development plans, and lessen the gap between job performance and organizational expectations.[69]

Developments in information technology

Computers have been playing an increasing role in PA for some time (Sulsky & Keown, 1998). There are two main aspects to this. The first is in relation to the electronic monitoring of performance, which affords the ability to record a huge amount of data on multiple dimensions of work performance (Stanton, 2000). Not only does it facilitate a more continuous and detailed collection of performance data in some jobs, e.g. call centres, but it has the capacity to do so in a non-obvious, covert manner. The second aspect is in mediating the feedback process, by recording and aggregating performance ratings and written observations and making the information available on-line; many software packages are available for this. The use of IT in these ways undoubtedly helps in making the appraisal process more manageable, especially where multiple rating sources are involved, but it also raises many questions about appraisees' reactions and possible effects on PA outcomes. Mostly, the evidence so far is positive.[49]

Rater errors

Mistakes made by raters is a major source of problems in performance appraisal. There is no simple way to eliminate these errors, but making raters aware of them through training is helpful. Rater errors are based on the feelings and it has consequences at the time of appraisal.[70][71]

Varying standards

  • Problem: When a manager appraises (evaluates) his or her employees and the manager uses different standards and expectations for employees who are performing similar jobs.
  • Example: A professor does not grade the exams of all students in the same standards, sometimes it depends on the affection that the professor has towards others. This affection will make professor give students higher or lower grades.
  • Solution: The rater must use the same standards and weights for every employee. The manager should be able to show coherent arguments in order to explain the difference. Therefore, it would be easier to know if it is done, because the employee has provided a good performance, or if it because the manager perception is distorted.

Recency effects

  • Problem: When the manager rates an individual above what the performance actually merits due to only considering the very latest performance and not taking into consideration a sufficient enough period for quality assessment.
  • Example: When a professor gives the course grade based just in the performance of the student only in the last week.
  • Solution: In order to avoid that, the manager can employ methods that track dominant traits as well as minor traits to understand adaptation over time. Total strength can be understood as the sum of the relative strengths.

Primacy effects

  • Problem: When the person who evaluates gives more weight according to information the manager has received first.
  • Example: During an evaluation the manager gives a higher score due to the initial impressions the employee made during their first few weeks, and is overlooking recent performance issues.
  • Solution: When the manager has to make a decision, it is better not to do it according to what he or she remembers, but should be based on all the relevant and documented data of the employees performance.

Central Tendency

  • Problem: When the manager evaluates every employee within a narrow range, as the average because he or she is dismissing the differences in the performance that employees have done.
  • Example: When a professor because the average of the class tends to grade harder. Therefore, if the performance of the class average is quite high, the professor will evaluate them more highly. On the contrary, if the average of the class is lower, he or she would appraise lower.

Leniency

  • Problem: Rating of all employees are at the high end of the scale.
  • Example: When the professor tends to grade harder, because the average of the class.

Strictness

  • Problem: When a manager uses only the lower part of the scale to rate employees.
  • Example: When the professor tends to grade lower, because the average of the class.
  • Solution: try to focus more on the individual performance of every employee regardless the average results.

Rater Bias[72]

  • Problem: Rater's when the manager rates according to his or her values and prejudices which at the same time distort (distorsionar) the rating. Those differentiations can be made due to the ethnic group, gender, age, religion, sexuality or appearance of the employee.
  • Example: Sometimes happen that a manager treats someone different, because he or she thinks that the employee is homosexual.
  • Solution: If then, the examination is done by higher-level managers, this kind of appraising can be corrected, because they are supposed to be more partial.

Halo effect

  • Problem: When a manager rates an employee high on all items because of one characteristic that he or she likes.
  • Example: If a worker has few absences but the supervisor has a good relationship with that employee, the supervisor might give to the employee a high rating in all other areas of work, in order to balance the rating. Sometimes it happens due to the emotional dependability based on the good relationship they have.
  • Solution: Training raters to recognize the problem and differentiating the person with the performance they do.

Horn effect

  • Problem: This is the opposite to the halo effect and horns effect occurs when a manager rates an employee low on all items because of one characteristic that he or she dislikes.
  • Example: If a worker performs well but at certain times, he or she loves telling jokes, but his or her supervisor dislikes jokes, the supervisor might give the employee a lower rating in all other areas of work. Sometimes it happens when they do not have a close relationship and manager does not like the employee.
  • Solution: Is the same as in the halo effect. Training raters to recognize the problem and differentiating the person with the performance they provide.

Contrast

  • Problem: The tendency to rate people relative to other people rather than to the individual performance he or her is doing.
  • Example: At school, if you are sat down where all the chatty people are and you are silent but you do not pay attention and you do not do your homework, because you are drawing; when teacher gets angry with the group, you might be excluded of the bad behavior they have just because you are silent; but not because you are doing a good performance. Therefore, according to the group, you are not that chatty, but you are either doing the proper performance. However the rater will only get the idea that your behavior is not as bad as other, thus, you will be rate higher.
  • Solution: The rating should reflect the task requirement performance, not according to other people attitude.

Similar-to-Me / Different-from-Me

  • Problem: Sometimes, raters are influenced by some of the characteristics that people show. Depending if those characteristics are similar or different to the evaluators, they would be evaluated differently.
  • Example: A manager with higher education degree might give subordinates with higher education degree a higher appraisal than those with only bachelor’s degrees.
  • Solution: Try to focus on the performance the employee is doing regardless the common characteristic that you have

Sampling

  • Problem: When the rater evaluates the performance of an employee relying only on a small percentage of the amount of work done.
  • Example: An employee has to do 100 reports. Then, the manager takes five of them to check how has the work been done, and the manager finds mistakes in those five reports. Therefore the manager will appraise the work of the employee as a "poor" one, without having into account the other 95 reports that the manager has not seen, that have been made correctly.
  • Solution: To follow the entire track of the performance, not just a little part of it.

We have been looking one by one at the possible solutions to each of the situations, which are also complicated to put into practice, thus here we have a general solution that could be applied to all the possible rating errors. It is difficult to minimized rater errors, since we are humans and we are not objective. Moreover, sometimes, we are not aware of our behavior of having preferences towards people but there are some tools in order to have a more objective information as using available technology to track performances and record it which enables the manager to have some objective information about the process.

Consultant Marcus Buckingham and executive Ashley Goodall, reporting on a large-scale Deloitte performance management survey on Harvard Business Review, went as far as to say that, contrary to the assumptions underlying performance rating, the rating mainly measured the unique rating tendencies of the rater and thus reveals more about the rater than about the person who is rated. They referred to this as the idiosyncratic rater effect. In view of this effect, they advocate a radically different approach to performance management. In their scenario, 360-degree feedback and similar time-intensive exercises are replaced by team leaders' "performance snapshots" that focus on what they would do with each team member rather than what they think of that individual, and yearly appraisals of past performance are replaced by weekly check-ins among team leader and team member, preferably initiated by the team member, that focus on current and upcoming work.[73]

See also

Notes

  1. Often also called employee performance appraisal or employee performance review

References

  1. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Muchinsky, P. M. (2012). Psychology Applied to Work (10th ed.). Summerfield, NC: Hypergraphic Press.
  2. MIT Human Resources Archived 17 June 2012 at the Wayback Machine
  3. (Tyskbo, 2020])
  4. (Evans & Tourish, 2017)
  5. 1 2 3 Waterman, Richard (1998). "Principal-Agent Models: An Expansion?". Journal of Public Administration Research and Theory. 8 (2): 173–202. doi:10.1093/oxfordjournals.jpart.a024377.
  6. 1 2 3 4 Manasa, K. & Reddy, N. (2009). Role of Training in Improving Performance. The IUP Journal of Soft Skills, 3, 72-80.
  7. Abu-Doleh, J. & Weir, D. (2007). Dimensions of performance appraisal systems in Jordanian private and public organizations. International Journal of Human Resource Management, 18(1), 75-84.
  8. 1 2 3 4 5 6 7 8 9 10 11 Muchinsky, P. M. (2006). Psychology applied to work (8th ed). Belmont, CA: Thomson Wadsworth.
  9. 1 2 Broady-Preston, J. & Steel, L. (2002). Employees, customers, and internal marketing strategies in LIS. Library Management, 23, 384-393.
  10. 1 2 3 4 5 6 Cederblom, D. (1982). The performance appraisal interview: A review, implications, and suggestions. Academy of Management Review, 7(2), 219-227.
  11. Josh Bersin. "Time to Scrap Performance Appraisals". Forbes. Retrieved 6 May 2013.
  12. Richard Charles Grote (2002). The Performance Appraisal Question and Answer Book: Survival Guide for Managers. 28-29.
  13. Muczyk, J. P. & Gable, M. (1987, May). Managing sales performance through a comprehensive performance appraisal system. Journal of Personal Selling and Sales Management, 7, 41-52.
  14. 1 2 3 DeNisi, A. & Pritchard, R. (2006, July). Performance appraisal, performance management, and improving individual performance: A motivational framework. Management and Organization Review, 2(2), 253-277.
  15. 1 2 3 4 5 6 7 8 9 10 11 12 13 Schraeder, M. Becton, J., & Portis, R. (2007, Spring). A critical examination of performance appraisals. The Journal for Quality and Participation, 20-25.
  16. 1 2 3 4 5 6 7 8 9 10 11 Sudarsan, A. (2009). Performance appraisal systems: A survey of organizational views. The Icfai University Journal of Organizational Behavior, 3(1), 54-69.
  17. 1 2 Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations. Thousand Oaks, CA: Sage.
  18. 1 2 3 4 5 6 7 Spinks, N., Wells, B., & Meche, M. (1999). Appraising appraisals: computerized performance appraisal systems. Career Development International, 4(2), 94-100.
  19. 1 2 Pettijohn, L., Parker, R., Pettijohn, C., & Kent, J. (2001). Performance appraisals: usage, criteria, and observations. The Journal of Management Development, 20, 754-771.
  20. 1 2 3 Jenks, J. M. (1991). Do your performance appraisals boost productivity? Management Review, 80(6), 45-47.
  21. 1 2 Kanfer, R. & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690.
  22. Mayer, C. M. & Gavin, M. B. (2005). Trust in management and performance: Who minds the shop while the employees watch the boss. Academy of Management Journal, 48(5), 874-888.
  23. 1 2 Kikoski, J. F. (1999), Effective communication in the performance appraisal interview: Face-to-face communication for public managers in the culturally diverse workplace, Sacred Heart University Government Faculty Publications and Public Personnel Management, 28(2), 301-322.
  24. 1 2 3 Cawley, B. D., Keeping, L. M. Levy, P. E. (1998). Participation in the performance appraisal process and employee reactions: A meta-analytic review of field investigations. Journal of Applied Psychology, 83, 615-633.
  25. 1 2 Twomey, D. & Harris, D. (2000). From strategy to corporate outcomes: Aligning human resource management systems with entrepreneurial intent. International Journal of Commerce and Management, 10, 43-55.
  26. 1 2 3 4 5 6 7 Selden, S. C., Ingraham, P. W., & Jacobson, W. (2001). Human resource practices in state government: Findings from a national survey. Public Administration Review, 61(5), 598-607.
  27. Soltani, E. (2005). Conflict between theory and practice: TQM and performance appraisal. The International Journal of Quality and Reliability Management, 22, 796-818.
  28. Daniels, Aubrey C. Designing a Compensation Program That Motivates and Produces a Profit-Driven Workplace. PMeZine.com
  29. McGivern, G. & Ferlie, E. (2007) ‘Playing Tick Box Games: Interrelating Defences in Professional Appraisal’, Human Relations, 60 (9) 1361-1385.
  30. Team, MBA Skool. "Central Tendency - Meaning & Definition". MBA Skool. Retrieved 5 October 2022.
  31. Editor, Staff (12 July 2022). "What Is Inflation Rating Performance Appraisal? - QuestionAnswer.io". questionanswer.io. Retrieved 5 October 2022. {{cite web}}: |last= has generic name (help)
  32. Amsterdam, C. E., Johnson, R. L., Monrad, D. M.,& Tonnsen, S. L. (2005). A collaborative approach to the development and validation of a principal evaluation system: A case study. Journal of Personnel Evaluation in Education, 17(3), 221-242.
  33. Martin, D. C. & Bartol, K. M. (1998). Performance appraisal: Maintaining system effectiveness. Public Personnel Management, 27(2), 223-230.
  34. Schweitzer, M E., Ordonez, L., & Douma, B. (2004) Goal setting as a motivator of unethical behavior. Academy of Management Journal, 47, 422-432.
  35. Schultz & Schultz, Duane (2010). Psychology and work today. New York: Prentice Hall. p. 153. ISBN 978-0-205-68358-1.
  36. "Improving Performance Evaluations | Office of Financial Management". ofm.wa.gov. Retrieved 5 October 2022.
  37. "What Solid Research Actually Says About Performance Appraisals". Forbes. Archived from the original on 7 May 2023.
  38. DeNisi, A. (2000). Performance appraisal and performance management: A multilevel analysis. In K. Klein & S. Kozlowki (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 121-156). San Francisco, CA: Jossey-Bass.
  39. Molleman, E. & Timmerman H. (2003). Performance management when innovation and learning become critical performance indicators. Personnel Review, 32(1).
  40. 1 2 Den Hartog, D., Boselie, P., & Paaiwe, J. (2004). Performance management: A model and research agenda. Applied Psychology: An International Review, 53(4), 556-560.
  41. Selden, S. & Sowa, J. (2011). Performance management and appraisal in human service organizations: Management and staff perspectives. Public Personnel Management, 40(3), 251-264.
  42. 1 2 Cummings, L. L. & Schwab, D. P. (1978). Designing appraisal systems for information yield. California Management Review, 20, 18-25.
  43. Katz, Ralph. Motivating Technical Professionals Today. IEEE Engineering Management Review, Vol. 41, No. 1, March 2013, pp. 28-38
  44. 1 2 Staw, B. M. (1986). Organizational psychology and the pursuit of the happy/productive worker. California Management Review, 28(4), 40-53.
  45. Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001). The job satisfaction-job performance relationship: A qualitative and quantitative review. Psychological Bulletin, 127(3), 376-407. doi:I0.1037//0033-2909.I27.3.376
  46. Evaluationforms.org (28 March 2013). "Performance Appraisal". Retrieved 17 April 2013.
  47. Gomez-Mejia, L. R., Balkin, D. B., and Cardy, R. L. (1998). Managing human resources (2nd ed.). Prentice-Hall, New Jersey.
  48. Smart, B. D. (2005). Topgrading: How leading companies win by hiring, coaching, and keeping the best players. New York, New York: Penguin Group.
  49. 1 2 Fletcher, Clive. Performance appraisal and management: The developing research agenda. Journal of Occupational and Organizational Psychology74 (Nov 2001):p.482.
  50. 2014 Performance Management Survey http://performancereviews.net/survey/
  51. Rehn, Karen. "What are Crowd Based Performance Reviews? Why use them?". HHStaffing. Retrieved 21 January 2015.
  52. Party-Directed Mediation: Facilitating Dialogue Between Individuals (Chapters 12-14, on-line 3rd edition, 2014), by Gregorio Billikopf, University of California.
  53. Party-Directed Mediation (Chapters 12-14, on-line 3rd edition, 2014), from Internet Archive (3rd Edition, multiple file formats including PDF, EPUB, Kindle, and others)
  54. Eva, N., Meacham, H., Newman, A., Schwarz, G., & Tham, T. L. (2019). Is coworker feedback more important than supervisor feedback for increasing innovative behavior?. Human Resource Management, 58(4), 383-396. doi:https://doi.org/10.1002/hrm.21960
  55. Eva, N., Meacham, H., & Schwarz, G. (2018, July). Is Co-Worker Feedback More Important than Supervisor Feedback for Increasing Extra-Role Behaviors?. In Academy of Management Proceedings (Vol. 2018, No. 1, p. 11942). Briarcliff Manor, NY 10510: Academy of Management. doi:10.5465/AMBPP.2018.11942abstract
  56. Millar, Gary (2005). "Solutions to Principal-Agent Problems in Firms". Handbook of New Institutional Economics: 349–370. doi:10.1007/0-387-25092-1_15. ISBN 1-4020-2687-0.
  57. LePine, J. A., Erez, A., & Johnson, D. E. (2002). The nature and dimensionality of organizational citizenship behavior: A critical review and meta-analysis. Journal of Applied Psychology, 87(1), 52-65.
  58. MacKenzie, S. B., Podsakoff, P. M., & Fetter, R. (1991, October). Organizational citizenship behavior and objective productivity as determinants of managerial evaluation of salespersons’ performance. Organizational Behavior and Human Decision Processes, 50(1), 123-150.
  59. 1 2 3 Keeping, L. M.; Levy, P. E. (2000). "Performance appraisal reactions: Measurement, modeling, and method bias". Journal of Applied Psychology. 85 (5): 708–723. doi:10.1037/0021-9010.85.5.708. PMID 11055144.
  60. Schultz, Duane P. Schultz, Sydney Ellen (2009). Psychology and work today (10th ed., International ed. ed.). Upper Saddle River, N.J.: Pearson Education. pp. 108–109. ISBN 9780205705870.
  61. Pink, Daniel H. (2018). When: The Scientific Secrets of Perfect Timing. New York: Penguin Random House. pp. 160–165. ISBN 9780735210622. OCLC 1001431465.
  62. Falk, Armin (2006). "The Hidden Costs of Control". The American Economic Review. 96 (5): 1611–1630. doi:10.1257/aer.96.5.1611.
  63. 1 2 3 Malos, S. B. (1998). Current legal issues in performance appraisal. In J. W. Smither (Ed.), Performance appraisal: State-of-the-art methods for performance management, (49-94). San Francisco: Jossey-Bass.
  64. Schultz & Schultz, Duane (2010). Psychology and work today. New York: Prentice Hall. p. 129. ISBN 0-205-68358-4.
  65. Seddon, J. (1987). Assumptions, cultures and performance appraisal. Journal of Management Development, 6, 47-54.
  66. 1 2 3 4 5 Chiang, F. & Birtch, T. (2010, November). Appraising performance across borders: An empirical examination of the purposes and practices of performance appraisal in a multi-country context. Journal of Management Studies, 47(7), 1365-1393. doi:10.1111/j.1467-6486.2010.00937.x
  67. Sparrow, P., Schuler, R., & Jackson, S. (1994). Convergence or divergence: Human resource practices and policies for competitive advantage worldwide. International Journal of Human Resource Management, 5, 267-299.
  68. Sully de Luque, M. & Sommer, S. (2000). The impact of culture on feedback-seeking behavior: An integrated model and propositions. Academy of Management Review, 25, 829-849.
  69. Cardy, R. & Dobbins, G. (1994). Performance appraisal: Alternative perspectives. Cincinnati, OH: South-Western.
  70. Patricia. Performance appraisal rater errors. Retrieved 24 April 2014, from http://smartchurchmanagement.com/performance-appraisal-rater-errors/
  71. Robert L.Mathis & John H. Jackson. (2010). In Michele Rhoades, Susanna C. Smart, Ruth Belanger & Rod Ellington (Ed.), Human resources management
  72. The Office of Human Resources at Dartmouth College. (15 April 2010). Common rater errors. Retrieved 24 April 2014, from http://www.dartmouth.edu/~hrs/profldev/performance_management/rater_errors.html
  73. Marcus Buckingham, Ashley Goodall (April 2015). "Reinventing Performance Management". Harvard Business Review. Retrieved 6 February 2016.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.