Process of Evaluation of Effectiveness and Shortcomings of Training Sessions

Evaluation ‘is a process that critically examines a program. It involves collecting and analyzing information about a program’s activities, characteristics, and outcomes. Its purpose is to make judgments about a program, to improve its effectiveness, and/or to inform programming decisions’ (Patton, 1987).

Why to Evaluate a Training programme?

• Evaluation helps in controlling and correcting the training programme
• It improves programme design and implementation
• It demonstrates programme effectiveness

There are two categories of evaluations that can/should be done, one is ‘Pre-Evaluation’ done during the program development and implementation stage. Another one is ‘Post-Evaluation’ which will tell us to what extent the training was effective and achieving goals of the programme. The former one includes ‘Needs Assessment’ & ‘Process of Implementation’ while the latter one includes ‘Outcome Evaluation’ and ‘Impact Evaluation’.

Kirkpatrick suggests four level evaluations to which Hamblin divides the fourth level into two to distinguish between effect on organization and economic effects. These five levels will elaborate the categories mentioned above:

1. Reactions: Trainees reactions to the overall usefulness of the training
2. Learning: Evaluated on the basis of quantity of content learned and time taken
3. Behaviour: Includes evaluation of the manner and extent to which the trainee has applied learning in his/her job
4. Organization: Measures the impact of the training in achieving organizational objectives
5. Ultimate Value: Measurement of the ultimate result of the economic contributions

Methods of Evaluation:

1. Questionnaires: To obtain opinions, reactions, views of the trainees
2. Tests: To find out if anything was learnt by standard tests
3. Interviews: To find usefulness of training offered
4. Studies: Comprehensive studies on opinions, judgements of trainers, superiors and peer groups
5. Human Resource Factors: On the basis of Employee Satisfaction, grievances, absenteeism etc.,
6. Cost Benefit Analysis: Cost of conducting a training programme Vs. Value
7. Feedback: collection and implementation of feedback report

Possible Shortcomings and how to overcome?

Shortcomings / Issues	Methods to Overcome
Poor Participation	Ensure Instructional Objective is clearly stated and use few Ice breakers
Confused / unable to follow	Tell them what you told meaning ‘Reiterate’ & check your pace, be clear and audible, check by asking questions
Low energy and enthusiasm, seem to be dry and dull	Involve Trainees by asking to share their experiences, use humor, make learning fun
Poor Understanding of the concept/subject	Use Hands on Training to Demonstrate and apply teaching points
Session is not lively, no encouragement	Engage participants in the learning process by group activities
What’s in it for me, type feeling	Create Win-Win environment to build self-esteem and self-worth
Trainees are often easily offended to the act of disapproval of their activity work	Disapproval should be done in a positive way and should be seen as a means of improving
Participants feeling uncomfortable/inconvenient	Check basic arrangements, change seating arrangements, room temperature etc.,
Poor interaction, not asking questions, not involving in activities	Encourage to ask questions, conduct activity,
Participant offended	Avoid telling jokes, avoid negative feedback, avoid targeting,
Power point turned out to be ineffective	Be ready with alternate strategy like using case studies, role plays, and experiential learning techniques.
Strong Resistance behaviour	Make them realize and accept the benefits of learning. Why to learn and what’s in it for them?
Poor acceptance	Build your credibility, prove your hold on subject, behave like you are there to learn together, be their friend and well wisher
Complicated stuff, hard subjects, statistical information, hard policies, difficult skills etc.,	Make the content easier by breaking it into small segments, present in an attractive manner, avoid jargons, avoid lengthy sentences, reduce speed, avoid distractors, facilitate rather than teaching
Participants are tough	Always be on the lookout for what works best for your participants

Distinction between Training, Development and Learning and the Role of Training & Development in Learning

There is much confusion in understanding the terms ‘Training’, ‘Education’, ‘Development’ and ‘Learning,’ to the point where they are often used interchangeably. In the context of HRD, however, it is often necessary to define and delineate each of these in order to clarify the associated activities and desired outcomes within an organisation and that aspects of each are necessary to ensure full employee potential.

Training may be defined as “a planned process to modify attitude, knowledge or skill behaviour through a learning experience to achieve effective performance in any activity or range of activities. Its purpose, in the work situation, is to develop the abilities of the individual and to satisfy current and future manpower needs of the organisation”

Development is ‘a long-term process designed to enhance potential and effectiveness. It is also defined as the growth or realisation of a person’s ability, through learning, often from planned study and experience’ There is more theory involved with such education and hence less concern with specific behaviour than is the case with training. The intent of Development is to provide knowledge and understanding that will enable employees to carry out non-technical organization functions more effectively such as problem solving, decision-making etc., Thus, Training is meant to operatives and Development is meant for managers.

Learning Dimension	Training	Development	Education
Who	Non-Managers	Managers	By schools and colleges
What	Technical mechanical operations	Theoretical conceptual ideas	Theory based Knowledge
Why	Specific job related information	General knowledge	Increases general knowledge and understanding of employee
When	Short Term	Long Term	Long Term

Education can be defined as “activities which aim at developing the knowledge, skills, moral values and understanding required in all aspects of life rather than knowledge and skill relating to only a limited field of activity.’ is wider in scope and more general in purpose when compared to training”.

Learning, in contrast to training, is generally defined more holistically, as ‘a process that encompasses training as well as education (Jensen, 2001)’. According to John Sloman (2005: 2), learning can be described as ‘a self-directed, work-based process leading to increased adaptive capacity.’ This process is an ongoing, lifelong journey that may not always be clearly planned or even intentional and can be considered the heart of HRD. Learning process includes the acquisition of skills as well as insights or factual knowledge, and is at play whenever ‘people can demonstrate that they know something that they did not know before (insights and realizations and facts) and/or when they do something they could not do before (skills)’-Mumford

Training for instance, can be associated with ‘learning by doing’ whereas education is more synonymous with ‘learning by thinking’; development involves ‘learning thinking, doing and feeling’ absolutist definitions are not really helpful in understanding the role of training, development, education and learning in the Context of HRM/HRD but in practice such distinctions have occurred and still do occur.’

Role of Training and Development in Learning:

Training efforts must invariably follow certain learning-oriented guidelines with strategic focus on needs of Learning like:

1. Modelling:
2. Motivation
3. Reinforcement
4. Feedback
5. Spaced Practice
6. Whole Learning
7. Active Practice
8. Applicability of Training
9. Environment
10. Areas of Training
1. Knowledge
2. Technical Skills
3. Social Skills
4. Techniques

Planning and Implementation of Career Development Programmes

Today's employees are more career conscious than ever demanding more in terms of personal growth and development. Organizations that fail to meet employee individual needs will be losing valued employees.

A Career Development Program seeks to match to needs of the employee with those of the organization with the major components being counseling and training. Career Development Programs are essential because of changes in the workforce like, work-life balance, diversity, and focus on quality; advances in management theory like, employee motivation; changes in managerial styles and the increasing complexity of technology.

The most important aspect of career Development is that the employee must accept his/her responsibility for development. The career development actions are:

• Job Performance
• Exposure
• Resignations
• Change of job
• Career Guidance
• Internal Mobility

1. Career Development Objectives

1. To improve the effectiveness of the organization
2. To maximize the employee efficiency
3. To ensure discipline
4. To adopt organizational changes

2. Career Development Planning & Evaluation Phase

1. Define what career development means for the organization (by considering views of management vs. employees vs. supervisors and HRD)
2. Determine the organization’s needs for career development
3. Identify goals to be achieved through the program
4. Design a program to meet the identified needs, including finalizing of Internal or External coaches

3. Career Development Implementation Phase

1. Pilot the program with a specific team or department, evaluate results, and revise if necessary
2. Launch and promote the program to employees identified
3. Continue to evaluate results and, if a survey was done at the beginning of the program, survey again in 6-12 months to measure results

As the implementation is being done, both at the pilot level and at the organization-wide level, following are some of the steps that should be considered:

1. Assessment:  of skills, knowledge, values, interests, and work style
2. Exploration of options:  promotions, transfers, rotation and enrichment
3. Identification and resolution of barriers:  if the employee seems to lack education or training or have other barriers to their desired career development within the company, coaching addresses those
4. Goal setting and planning:  specifying what the employee wants to achieve and helping them get there through a Career Development Plan, often working with their direct supervisor
5. Strategizing:  support systems, mentoring, interviewing, training available resources, and timelines

4. Career Development Action plans:

1. The actual steps necessary to prepare the employee for further career growth must be noted down.
2. It must be realistic and measurable as to allow both the employee and organization to evaluate the employee's progress.
3. It must be specific and contain achievable developmental objectives, as well as the resources necessary to achieve these goals.
4. It must be flexible enough to enable the employee to reassess individual needs and desires throughout their careers.
5. A time frame for accomplishing these activities needs to be established and periodic follow-up must be conducted.
6. Continuous monitoring and evaluation are essential, especially in the early part of the employee's career. Employee development is multifaceted and must be viewed as a long-term process as there is no "one best way" to develop employees. 

What is an OD (Organizational Development) Intervention?

The term interventions refer to the range of planned programmatic activities clients and consultants participate, during the course of an organization development programme. Organization Development is “an effort, planned, organization-wide and managed from top, to increase organization effectiveness and health through planned interventions in the organization’s processes using behavioural-science knowledge”.

How to identify an Appropriate Intervention?

Plan an OD Strategy:

1. Desired Objectives
2. Specific Interventions aimed at achieving objectives
3. Time Frame
4. Monitoring, Review & Evaluation System

The strategy must specify contingencies as well as primary interventions and take into account the power and influence dynamics of the Organization. There are six general steps to building a strategy:

1. Defining the Change Problem
2. Determining appropriate OD Objectives
3. Determining the systems and subsystems readiness and capacity to change
4. Determining key subsystems
5. Assessing resources
6. Selecting an approach and developing an action plan for reaching objectives

To understand the type of Intervention required to the identified problem, typology of interventions based on the following questions had to be made.

1. Is the Intervention directed primarily toward individual learning, insight, and skill building or toward group learning?
2. Does the Intervention focus on task or process issues?
3. What interventions are most commonly used to improve their effectiveness?

Focus on the Individual	Focus on the Group
Role Analysis Technique (RAT) Education Career Planning Job Enrichment MBO	Techno structural Changes Survey Feedback Confrontation Meeting Team-Building Sessions Intergroup Activities Grid OD Phases 2,3 Sociotechnical Systems
Life Planning Coaching & Counseling of Individuals Group Dynamics, Planned Change Stranger T-Groups Third Party Peace Making Grid OD Phase 1 Gestalt OD Transactional Analysis	Survey Feedback Team-Building Sessions Intergroup Activities Process Consultation Family T-Group Grid OD Phases 2,3 Gestlat OD

Implementation of  OD Intervention       –             Level of Team Structure and Function

Example: Team Building (based on the working experience with OD Consultant in an exercise conducted for Sr.Management Team of our company recently, initiated by our strategic partner company)

Key Objectives:

• To review and improve the Team effectiveness
• To identify problem areas of team behaviour and corrective actions

Developmental Focus:

• Team Performance
• Team Goals , Goal setting process
• Team norms, culture and feedback processes

Participants:

• The team Leader and all Team members

When to Implement:

• Useful as an early step for the Senior Management team in an organization
• The initial sessions to be followed by a session 3 to 9 months later to evaluate progress on action plans

Time Scale:

• Five to Seven days based on the Team Size

Details of Process of Implementation:

1. Contract Session
• Discussing Needs and Expectations
2. Pre-work Session
• Developing 10 to 20 Interview questions
3. Information Collection
• Conducting Individual Interviews
4. Information Feedback
• Distributing copies of handouts to all group members for review
5. Information Analysis
• Identifying key Strengths and Weaknesses
6. Agenda Development
• Rank ordering key weaknesses / problems according to importance
7. Problem examination
• Discussing each key weakness / problem in depth
8. Option Generation
• Generating ideas for solving each key weakness / problem identified
9. Action Planning
• Developing a written plan of action for each key weakness / problem identified
10. Assessment
• Evaluating the meeting in terms of content and process

Review / Follow-up session

Barriers of Effective Communication

In general, it is the process by which individuals share and cooperate is called as Communication. In organizational perspective communication may be viewed as “flow of material, information, perception, and understanding between various parts and members in the organization”. It includes all aspects of communication upwards, downwards, lateral, speaking, writing, listening, reading, methods, median, modes, channels, networks, flow, interpersonal, intra-personal, inter organizational & intra organizational issues, as quoted by Vardhaman & Halterman.

Communication process can be described as an interpersonal process of sending and receiving symbols with meaning attached to it. Information is enclosed in a package and is channelled and imparted by a sender to a receiver via some medium.

Barriers of Communication: There are number of Interpersonal and intra personal barriers and environmental barriers that explain why the message decoded by the receiver is often different from the one that is intended by the sender.

Physical Barriers:

The competing stimulus- e.g. loud music, traffic noise.

Environmental stress- High temperature and humidity, Poor ventilation.

Subjective Stress- Sleeplessness, ill health, mood variations.

Filtering: It refers to a sender manipulating information so that it will be seen more favorably by the receiver.

Limitations of Sender’s capacity: Sender is often unable to put across his message in a language and form understandable to receiver because of wrong medium or uses medium in a confusing manner.

Limitations of Receiver’s capacity: It depends on his span of attention, his intelligence level, his understanding level of the subject and his memory.

Selective Perception: The receiver selectively sees and hear based on their needs, motivations, experience, background and other personal characteristics.

Information Overload: When the information we have to work exceeds our processing capacity as indicated by research.

Barrier of All-ness: It is the attitude of people with closed minds. They assume they know everything about a subject and are not prepared to believe that they can be mistaken.

Either/Or Orientation: We interpret messages in extremely negative or positive ways, wrong or right, black or white terms.

Snap Reactions: The listener responds speedily to the communicator’s message pronouncing it favorable or unfavorable even before communication is complete.

Tendency to Evaluate: Premature evaluation and judging tendency is a serious barrier. Resistances to change- New ideas that do not support our own views are resisted outright.

Defensiveness: When people feel that they are being threatened, they tend to react in ways that reduce their ability to achieve mutual understanding.

Linguistic & Cultural Barriers: Sender’s success is determined in the way in which he handles written and spoken words. Words are Symbols. Language is ambiguous.

Mechanical Barriers: These are raised by channels employed for interpersonal group or mass communication. E.g. Jargons, wrong placement of speakers, too small font size in magazine.

Other barriers include Personal expertise, Rules and Regulation Arrogance, Timing, Respect and confidentiality.

Organizations owe their existence to Communication. Most problems in business are caused by poor communication. So, understanding and mastering the art of effective communication is very important to every member of organization.

Conditioned Vs Instrumental Learning

“Learning could be defined as a relatively permanent change in behavior that occurs as a result of prior experience.” – E.R.Hilgard

”Learning has taken place if an individual behaves, reacts and responds as a result of experience in a manner different from the way he formerly behaved.”- McGehee

Classical conditioning, in contrast to operant conditioning, is a type of conditioning where an individual responds to some stimulus that would invariably produce such a response. This is also called Pavlovian conditioning after Ivan Pavlov, who worked out its fundamental principles through his studies of salivation in dogs, which he found could be elicited by neutral stimuli, such as ringing a bell, that had been repeatedly presented before the presentation of food. Another name for this form of learning is ‘Respondent Conditioning’.

Operant conditioning is voluntary or learned behaviour and it is determined, maintained and controlled by its consequence. It is also called Skinnerian conditioning after B.F. Skinner, who worked out its fundamental principles. Another name is Instrumental Conditioning, since the learned responses, which operate on the environment, are instrumental in either attaining some subsequent desirable reward or avoiding-escaping some subsequent aversive/punishing event.

Classical & Instrumental Conditioning Distinction: In classical conditioning one tries to increase the probability of a response to some neutral stimulus by pairing that stimulus with a following stimulus that already produces the response Initially, US - - ->UR, then CS, US - - ->UR, finally CS - - -> CR

In operant conditioning, one is trying to increase the probability of a response in the presence of some stimulus by following the desired response with a reinforcing stimulus. Alternatively, you might be trying to decrease the probability of a response (CR) in the presence of some stimulus (DS/CS) by following the undesired response with a punishing stimulus (P):DS/CS - - - -> CR -----> R or P

Note that in operant conditioning, unlike classical conditioning, there are response consequences, i.e., there are reinforcement or punishment contingencies--the reinforcement or punishment (R or P) being dependent/contingent upon the response (CR) occurring.

In contrast, in classical conditioning the unconditioned stimulus (US, which is like a reinforcement or punishment) follows the conditioned stimulus (CS) during training regardless of whether or not the conditioned response (CR) occurs. Here the CR, which is usually reflexive, is brought under the control of a stimulus event (CS) that precedes the response, rather than one that follows it.

Conscious application of operant conditioning principle often called as ‘Behaviour Modification’ helps in making a particular behaviour learnt and modified. This is Popularly called as ‘OB Mod’.

Example : The management of Emery Air Freight wanted packers to use freight containers for shipments whenever possible for economic savings. When packers were asked the reply was that they containerized 90 percent but the actual utilization is 45 percent. In order to make packers learn and change this behaviour, the management established a program of feedback and positive reinforcements. At the end of the first day the utilization rate jumped to 90 percent and was held to that level. In a three year period customer service, freight delivery systems, scheduling and other personal services were improved saving $2 million. The example of Emery Air Freight illustrates behaviour modification through operant conditioning.