The Home Office employs 31,000 staff and leads on immigration and passports, drugs policy, crime policy and counter-terrorism and works to ensure visible, responsive and accountable policing in the UK.
Led by Guy Bailey (pictured right), the department has recently implemented a new internal communications measurement initiative through the development of a unique staff engagement dashboard. The project saw the Home Office working on three particular layers of evaluation: the first one being internal communications and engagement measures; the second one high-level staff engagement priorities, and the third being an overall engagement index.
“I took over the role about 18 months ago” Guy recalls. “At that time we were already doing a lot of work around surveys as well as all the usual channel evaluation techniques such as monitoring the numbers of hits and comments on the intranet. A good bit of data, but that was not enough; what we were doing was not tailored to day-to-day work and not everyone was aware of how to make our internal channels more effective across the whole organisation.”
Understanding the need for a fresh approach in order to deliver real value to staff, Guy worked with his line manager, James Harding, to change the way measurement was implemented inside the business.
“The internal communications team helped to draft the staff engagement strategy for the board. By putting together the key elements of the new evaluation methodology we could get buy-in from senior managers.”
Team Briefing
Guy explains that one of the requirements for the new dashboard was to pull all the data into one document. There were eleven metrics that the Home Office had been working on, and the idea was to amalgamate all this data into a cohesive summary: “in a dashboard, what we are trying to do is to combine the reach and richness of all these metrics.”
Guy takes as an example ‘team briefing’. In the dashboard, you have the opportunity to combine a variety of information: half of the weighting is given to the percentage of staff who received the briefing (‘reach’), the other half to the percentage of people who understood the key topics discussed in the team briefing (‘richness’). These figures are then combined to give an evaluation of the overall effectiveness of the team briefing metric. In other words it’s about measuring quantity and quality.
The other ten metrics have been developed on this ‘reach’ versus ‘richness’ matrix.
“It’s not simply one evaluation and one data set; it can be two or three dimensional depending on what you are evaluating,” says Guy.
While some are simply quantitative measures, because qualitative data is more resource intensive to collect, the majority are combined. Guy stresses it’s the way of pulling all the metrics together and making sense of all that data that makes the real difference.
Leadership Communications
The second level of the dashboard considers the metrics in a total different way.
To explain this, Guy recalls the ‘team briefing’ metric and the concepts of ‘richness’ and ‘reach’ described earlier.
Thanks to the new dashboard Guy could apply those concepts in a different context such as leadership evaluation.
“One of our board priorities for employee engagement was leaders’ visibility, and another was clarity of leadership communications.
Since I had already divided these elements in two segments, I realised I could combine them into one metric. In this context, richness would be about clarity of communications, with half of that metric going towards the clarity of leadership communications. The reach – staff who get it – is instead about senior leaders getting out and about and making sure team briefings are delivered. Therefore, the other half of the metric would go towards leadership’s visibility. In a similar way the five level two engagement priorities are a combination of either whole first level metrics or an element of the metric as described and one case a combination of five different level-one measures.”
Employee Engagement
“My original aim was to have something that charted how well the organisation was doing from an employee engagement perspective. This would include the internal communications channels and how effectively these help to drive staff engagement. This is what level three demonstrates”, says Guy.
At this level, all the level one and level two evaluaton is combined into one overarching graph using some complex benchmarking and interpolation techniques. By presenting these three layers together the Home Office can monitor, evaluate and satisfy multiple internal customers with differing requirements.
“Senior managers can use the dashboard to see how they are doing in terms of employee engagement. Also they can get the narrative and all the details behind the dashboard’s engagement results.
“Communication channel owners can look at individual metrics and see how the channels they are looking after are performing over time.
“And a lot of potential action-planning comes out of it”, stresses Guy.
At the end of each month the internal communications team get access to level one charts with detailed stats for each channel. Underneath that information, channel owners can record actions required along with an explanation for trends and changes from previous months. This additional information gives the dashboard data context and robustness.
This level 3 summary data is also published on a large whiteboard chart so that everyone is able to understand how internal channels are doing in terms of concrete value delivered and staff engagement. This is long-term data, which the Home Office was unable to track before implementing the dashboard.
“Now key staff in the Home Office, such as our communication account managers, are able to use staff feedback. They can get access to the trends shown on the dashboard, and see how their part of the business is doing in responding to staff feedback. Consequently, they can act on the comments, fears and concerns staff may have.”
The rationale behind the new dashboard may appear to be complicated with all those metrics and multiple layers. “In reality” Guy explains “it has proved to be very successful, and useful in many situations, especially in making sure that concrete and effective actions follow employee feedback.”
Sharing through stories
Guy looks after the internal communications aspect of the intranet, and used this channel to share the results of other evaluation such as internal surveys which he also manages. The way he does it is through articles. “We aspire to a culture of ‘we said, we did’. In line with our values, I wanted to help instigate the sharing of data and support an attitude of not being afraid of what the figures were saying.”
The internal communications team translate figures and stats into a narrative, comprehensible to everyone.
As an example Guy describes the current transformation programme the Home Office is going through. Their latest survey on channels was all about understanding if the transformation messages were getting across, if employees were feeling comfortable with the changes, as well as all the usual questions an organisation would ask when facing transformational changes.
On top of that, Guy’s team used the dashboard to benchmark all the data and metrics to review previous quarters. Then they shared that data with directors, so they were able to understand where staff were on the transformation journey.
Guy’s team take an open and honest approach to their work and publish full survey results on the intranet. These articles actually show the business what the changes mean for staff, picturing employees’ feeling about the changes over time.
Measuring internal social media
In terms of internal social media, Guy candidly admits that the Home Office is not using it as well as it could: “We are not brilliant in terms of internal social media. There is an element of mistrust with staff having access to social media. I think this mistrust comes down to a lack of understanding of what social media really is. However there is also an element of operational staff not wanting to take up the opportunity to use this sort of channel but a pilot should find out how popular it may be and then a decision can be made as to whether it should be adopted.”
The organisation – is currently looking at Yammer and things are likely to change. In terms of interactive communications the Home Office is currently making use of forums and web chats hosted on the intranet. When it comes to measuring the effectiveness of those channels, Guy measures the number of comments, the number of responses, but also looks at the kind of responses and combines the data using a weighting index.
“To measure the effectiveness of forums I start by measuring the ratio of responses to comments . That is the key thing I measure. No one told me to do that, it just seems to me the more logical thing, the more effective way to work out whether the forum is successful or not. A popular topic could result in hundreds of comments but true engagement is about how these comments are dealt with and what changes happen as a result. The number of comments on the forum is also used as part of another metric which captures the volume of feedback from the business which is the reach aspect of the forum.”
However, he goes beyond that. Everything is weighted and combined together into a feedback measure.
“I think that the real evaluation of whether a channel is effective, is two way. If you have thousands of people commenting on a director’s message, that is going to look like a very successful forum. If you measure just the number of comments, then that looks brilliant… But what I do with the metrics is to differentiate all the different kind of comments and feedback. My team are fantastic in differentiating all the different kind of comments. We measure the number of people who dial in to things, the number of people who say something on forums, the number of people who like or dislike comments, the number who post comments and the number of people who like articles and comment on them. We collect all of these measures and more. Then we weight them all. For example if someone has produced a 300-word article for the intranet this is surely worth more than someone liking a comment – therefore the article is weighted by say 50. This allows the figures to be combined together into a meaningful engagement quantity.”
Rewarding employees for sharing their ideas
To reward staff for their input, feedback and ideas on the intranet, the Home Office runs narratives based on web chats and conversations originated on that channel.
“For example, the directors agree to run a forum, let’s say for a period of three weeks. Some very good staff suggestions come up during those conversations. While it can be a challenge to answer to all the questions when the forum is live, we do follow-up stories.
“Importantly, we publish the actions that have taken place based on employee discussions online. So staff can see the results of their inputs. These stories show ongoing changes resulting from the inputs of our staff. They can see what kind of ideas and comments the business has taken forward and feel their input is making a tangible difference.”
Understanding the relevance of internal communications evaluation
The outcomes from developing and adopting the new dashboard are very positive. These included influencing Board strategy, improving internal channels, sharing best practice and above all in instilling a culture of evaluation across the whole team.
“The best thing about the dashboard is that everyone in our internal communications team understands the importance of evaluation, which has become an important part of what we are doing as communicators,” says Guy.
“At the beginning there was a some resistance; people in communications do not always recognise the importance of numbers. Luckily, I have had a very strong line manager who believed in this product. And now people can see results.
“Clearly there has been a big difference in terms of actions. We are able to tailor communication activities for specific channels and areas of the business and at the same time break down the silo mentality which is common in large, complex organisations.”
——————————————————————————————–
Picture at the bottom: Guy talks through the monthly level three results internal comms channel manager – Vicky Pennington
This article originally appeared on simply-communicate
No Comment