Cross cultural coaching

Evaluating coaching assignments

Why evaluate?

Evaluation is important because it:

• Allows you to demonstrate return on investment
• Educates clients and their line managers about what good coaching looks like
• Enables you to intervene if the relationship isn’t working
• Informs how you and other HRBPs use individual coaches in future
• Enables you to make improvements in the overall process of using external coaches

When to evaluate

• After first two sessions (is it working?)
• At about the half-way point, informally, to ensure that progress is happening
• Immediately after end of assignment, or before renewal of contract, if assignment extended – a maximum of three weeks after the final coaching session is recommended
• After 12-24 months (have changes stuck?)

What to evaluate

You might think that evaluating external coaching is simply a matter of asking the client whether they found it useful and /or of establishing whether a specific performance goal has been achieved. However, it is more complex than that, because:

• Feedback from clients, who are naive about effective coaching, is unreliable
• Coaching goals frequently evolve into something very different as coach and coachee explore issues in depth
• Many of the most valuable outcomes of coaching are often not predicted at the start

The tables below show some of the practical ways you can measure the effectiveness of external coaching at the end of the assignment, in terms of changes for the coachee and for the coachees’ team. These are not exhaustive lists — it is important that you are creative in thinking about what could usefully be measured and that you aim where possible for a mixture of “hard” (SMART) and “soft” goals. Hard goals might be an improved score in a 360 feedback, or fewer absentee days in the team, or hitting tough targets for project delivery. Soft goals might be increased self-confidence, or feedback relating to how others perceive the coachee’s attitude.

Be prepared to alter the measures to meet with changed goals. The mid-point, informal evaluation is usually a good place to do this.

Table 1: Measures of impact on the individual

Table 2: Measures of impact on the coachee’s team

While you may suggest measures, the decision about which ones to use rests with the coachee and their line manager.

You will probably find that you have more potential measures than are practical. In deciding upon the measures to use, consider:

• What will this measure tell us that will be useful to:
  • The coachee?
  • Their boss?
  • HR?
• How important is this measure to them?
• How credible will it be to each of the stakeholders?
• What are the priority measures?
• How difficult will it be to gather this data?

How to evaluate

You have a number of choices, which are not mutually exclusive. These include:

• bringing together coach, coachee and line manager for an evaluation conversation at the end of the assignment
• surveying stakeholders (e.g. coach, coachee, coachee’s peers, coachee’s boss) – see generic template below
• gathering data informally
• requiring coachee and/or coach to produce Statement of Return on Investment, in which they gather their own evidence of where coaching has and has not had a significant impact

Longer term evaluation

Re-evaluating after 12 to 24 months is useful for a number of reasons:

• It tells you whether changes have stuck, so you can offer additional help if needed
• It helps coachees keep their newly learned behaviours and thinking patterns at front of mind, reinforcing changes
• It encourages coachees to seek further coaching to tackle other development needs that have emerged
• it allows you to gather data on trends and identify recurring issues amongst a number of coachees. These issues may be addressed with solutions other than coaching, or with team coaching

Using 360 feedback to evaluate external coaching

Good practice guidelines include:

• Ensure that the coachee has an input into the design of the questions
• Ensure that the coachee chooses who they wish to give feedback (people, whose opinion they will respect)
• Keep it simple – maximum of six questions
• Gather both quantitative and qualitative data (ask for examples of both positive/ negative behaviours)
• Recognise that poor bosses often score better than good ones, because respondents are afraid to be honest with the former!

Questions you can use in evaluations

These questions can be used either in informal feedback conversations or as on-line surveys.

Questions to coachees at the start of the assignment

What changes are you trying to achieve in:

• Leadership behaviour?
• Personal performance?
• Team performance?
• Thinking/ decision-making processes?
• Specific skills (e.g. communication, risk-management)?
• Other?

How will this change be recognised by:

• You?
• Your coach?
• Your team?
• Your peers?
• Your bosses?

Who will you invite to give feedback?

(These should be people, who have an opportunity to observe and who are stakeholders in the change.)

Questions to coachees at the mid-point of the coaching assignment

Questions at this stage are most useful when they focus on the quality of the coaching relationship.

On a scale of 1-5 (% being high), how true is each of the following statements?

1. We are both clear about the coaching agenda and learning goals
2. We are able to confront and discuss difficult issues openly
3. Our discussions are creative and reflective
4. Our discussions are constructively challenging
5. Our discussions have depth
6. We trust and have confidence in each other
7. We are both well-prepared for coaching sessions
8. We review the relationship regularly and discuss how to improve it
9. We revisit and re-evaluate the relationship goals from time to time
10. I leave the coaching session feeling motivated to action
11. I feel the coaching sessions are having a significant impact on my thinking and decision-making

Questions to coachees at the end of the coaching assignment

General

• To what extent have you achieved your original goals?
• To what extent have you achieved other goals that emerged during the coaching process?
• What development needs will you want to address over the next 12 months?
• Are you now using more of a coaching style within your team?
• How has the coaching added value for PMI?
• Would you recommend this coach to colleagues?

Performance goals

I have:

• Increased my productivity
• Gained or improved leadership skills/ knowledge
• Gained or improved technical skills/ knowledge
• Met key targets, about which I was concerned
• Improved performance of my team

Enabling goals

I have:

• Reduced my stress levels
• Gained confidence in my ability to do my job role
• Gained greater understanding of my own motivations
• Gained greater understanding of my strengths and weaknesses
• Gained a clearer sense of purpose
• Clarified work-related goals and priorities
• Clarified personal (non-work) goals and priorities
• Stronger commitment to work-related goals
• Stronger commitment to non-work goals
• Established a wider range of resources, on which I can call to help my development
• Improved my reputation
• Improved the quality of my relationships with:
  • Bosses
  • Peers
  • Direct reports
  • Customers

Questions to coachee’s direct reports

• To what extent have you observed positive change in your boss, with regards to:

Where appropriate, this data can be compared with 360 feedback ratings on these competencies/ behaviours at the beginning of the coaching relationship.

Questions to external coaches at mid-point of the assignment

• To what extent do you feel the client is committed to the coaching relationship?
• Do you feel that the goals of the relationship have been sufficiently clarified?
• In what ways do you consider you are helping the client?
• Do you still feel that coaching is the right intervention for this client at this time?
• Do you feel the client is making sufficient progress?
• Do you have any concerns about the assignment?

Note: It is important not to induce or appear to induce the coach to break client confidentiality.

Questions to external coaches at end of the assignment

• To what extent do you feel the client’s initial goals have been met?
• What additional goals emerged during the assignment and to what extent have these been met?
• On a scale of 1-10, how effective has this coaching assignment been?
• In what ways has it added value for the client? For PMI?
• To what extent do you feel that the coaching relationship has been supported by PMI?
• What, if anything, could have been done by the company or the client’s line manager better to support the coaching assignment?
• What do you recommend takes the place of the external coach for the coachee’s continued development over the next 12 months?

Questions to coachee’s line manager at the beginning of the coaching assignment

• What changes do you want to see in the coachee’s
  • Leadership behaviour?

• • Personal performance?
  • Team performance?
  • Thinking/ decision-making processes?
  • Specific skills (e.g. communication, risk-management)?
  • Other?

• How will you recognise this change?
• What specifically will you do to support the change?

NB Research shows that bosses often do not notice improvements, unless they are specifically looking for them. They tend, instead, to notice evidence that confirms their previous opinion!

Questions to coachee’s line manager at the midpoint of the coaching assignment

• Do you have any concerns about the coaching assignment?
• Do you perceive that the coachee is making progress?

Questions to coachee’s line manager at the end of the coaching assignment

• To what extent have you perceived changes in the client?
• To what extent has coaching the client resulted in positive outcomes for you?
• Has the assignment changed your view of coaching as a method of leadership development and, if so, in what ways?
• What, if anything, could have been done by the company or by you better to support the coaching assignment?
• What do you recommend takes the place of the external coach for the coachee’s continued development over the next 12 months?

Collating data from coaches

The purpose of collecting data is:

• To enable Korn Ferry to monitor the quality of coaching and identify any needs, which the coach may have, for additional support or development (or in extreme cases, removal from the coach pool)
• To identify significant trends amongst the client population, which HR needs to address (for example, concerns about career structures, or recurrent sources of severe stress)
• To inform discussion between PMI and Korn Ferry about the external coaching process

Data from coaches may come from three main sources:

1. Feedback collected from coaches’ survey responses to PMI at mid-point and end-point of each relationship. (The coach should provide this data to both PMI and Korn Ferry.)
2. Issues arising from supervision (within the constraints of client confidentiality). It is the coach’s responsibility to provide to the client manager within Korn Ferry a summary of any relevant issues arising from supervision.
3. Issues arising from coaches’ final debrief on the assignment to Korn Ferry. This should cover the same ground as (1) plus
   • What learning has the coach acquired from this assignment?
   • What further development needs have they identified in their role as an external coach?
   • What do they feel they did most and least well in assisting the client?
   • What advice, if any, can they offer to other coaches about coaching with PMI?
   • Would they welcome further opportunities to coach within PMI?

Only the first of these data sources will be available to HRBPs / PMI in their raw form. However, both PMI and Korn Ferry need data on how effective each coaching assignment has been. They also need aggregate data from all assignments over a period, to assess the effectiveness of coaching overall and make process improvements.

Periodic meetings between Korn Ferry and PMI will review coach provision and feedback from stakeholders, including coaches. Korn Ferry will also review stakeholder feedback for each assignment with the coach concerned. These continuing dialogues between all partners are essential for building a genuine coaching partnership.

© David Clutterbuck, 2015

Prof David Clutterbuck
Coaching and Mentoring International Ltd
Woodlands, Tollgate,
Maidenhead,
Berks, UK. SL6 4LJ

www.coachingandmentoringinternational.org
e-mail: info@coachingandmentoringinternational.org
Company registration number : 08158710