The human side of intranets Part IV

User opinion can be overshadowed by technological hype when measuring intranet performance, but it is actually the most important consideration in establishing how successful your system really is.
Part IV: Gauging intranet user reaction.

By Paul Chin

How do you truly measure the success of an intranet? This is a rather subjective issue as individual organisations have different criteria with which to gauge success. An organisation based on technological innovation may place more emphasis on its tools; while others that rely on the tight-knit cooperation of their employees, place more importance on knowledge sharing than technology. But there is one common denominator in every organisation that will dictate the ultimate success or failure of its intranet: user reaction.

In this final part of my series on the human side of intranet development and management, I will be looking into the ways in which intranet owners can gauge user responses and reactions to their system.

The importance of positive user reaction
An intranet as a whole is made up of three core components: technology, content and community. Unfortunately, too many intranet owners still think that the systems are purely technology-driven. They place the highest importance on technology, followed by content and, finally, the user community. The fact of the matter is that this view should be reversed. Technological innovation will mean nothing if the content is useless and valuable content is meaningless without people to put it to good use. Yet, for many organisations today, the focus is still placed on the vehicle rather than those driving it.

An intranet’s primary function – regardless of its design and technology – is to connect an organisation’s employees together within a ‘virtual knowledge network’, where they can coordinate their efforts and share information in a centralised environment. But, if the system goes against the grain of organisational culture or functions contrary to the manner in which employees are used to working, this network will quickly unravel – if it even gets off the ground.

The quality of use that employees get out of an intranet is the true definer of success. In this sense, an adequately designed intranet with a large and satisfied user-base could be considered much more of a success than a fancy, cutting-edge intranet with little content and an indifferent user community.

User satisfaction plays a larger role in determining success than most intranet owners give it credit for. It is crucial to understand an intranet’s user base and how it interacts with the system. Failure to do so will result in a system that is out of touch with those it is meant to support.

Measuring user reaction: Methods
There are many ways to measure user response to an intranet, but no single method will tell the whole story. Some will provide nothing but hard facts while others can help graph user behaviour. But one thing is for sure: we are well past the point of simply looking at log files.

Gauging true user reaction goes far beyond looking at raw web-server logs and counting the number of times users access the site or hit a particular page. It needs to include information such as:

What types of users are accessing the intranet?
Which users or departments are accessing specific content types?
When are users accessing the intranet?
Which content receives the most traffic?
How are users putting this content to use?
Are users satisfied with the system and the quality of content offered?

Fully understanding intranet user behaviour and reaction to the system necessitates the combination of two methods of measurement: passive and active. Passive methods provide information on the where, when, and who, while active methods provide information on the why.

Passive methods
Passive methods are used to measure intranet usage and behavioural patterns, and are labelled this way because they take place behind the scenes and do not require any interaction with the user community. These methods allow intranet owners to gather hard, factual statistics about who is accessing what and when, without disrupting users’ day-to-day work. This type of approach is useful because the responsibility of gathering usage information is placed directly onto the shoulders of those who will be processing and analysing the data as well. There is no reliance on any unknown factors, such as those that can be found in the active methods to be discussed in the next section.

Web analytics – the process of analysing user behaviour towards a website or intranet – is a perfect example of a passive method of measuring intranet usage. It usually involves only intranet administrators and high-level content owners. Web-analytics tools allow administrators to process the raw data collected from web servers to generate value-added, user-friendly reports in the form of charts, graphs and tables.

However, when using passive methods it is important to make a firm distinction between casual visitors and actual users. Casual visitors that access the site out of curiosity and click from one page to the next are probably not staying around long enough to make productive use of the content. Such casual hits can inflate the statistical usage data that is collected. This is precisely why passive methods alone will not be enough to give you a true indication of overall user response.

Active methods
Active methods are used to measure user satisfaction rather than gather statistical data on usage. Unlike passive methods, active methods require a high level of interaction with the user community – which will provide all of the feedback data. Measurements can be obtained through a simple, free-form contact page or a full-blown online user survey.

While a contact page is the quickest and easiest way to gather user feedback, it is also the most difficult with which to quantify the results. Since it is based on free-form feedback users can write just about anything, from a single word expressing satisfaction to a thunderous five page diatribe on why they think the intranet is little more than a glorified piece of driftwood.

A more effective way to put active methods into practice is to create an online user questionnaire. This provides intranet administrators and content owners with a template to gauge user reactions to various aspects of the intranet – from site design to content quality.

But there is also a downside with active methods, and it is one of human nature. There is no telling how many people will actually take the time to fill out the questionnaire. And unfortunately for those running the survey, people are far more likely to voice their opinion when they are disgruntled with something than when they are happy. An intranet that performs well may be taken for granted as users bask in quiet satisfaction. However, even minor imperfections can raise the ire of a once calm user community to pitchfork-wielding mob status. So it’s very possible for a vocal minority of unsatisfied users to overshadow a quiet majority of satisfied users.

To balance out the scale, intranet owners need to get the largest sample size possible. Links to the online questionnaire should be placed on the intranet’s home page and other high-traffic areas to increase exposure. Some organisations even offer incentives for completing and submitting a questionnaire.

Designing a user survey
A lot of animosity will be created if users are made to feel as though they have no say in the direction of their intranet, and are at the mercy of someone who doesn’t fully understand their needs. This happens far too often in many organisations. Put simply, IT delivers what it thinks users want rather than listening to what they need.

Surveys are a perfect way to give the user community a voice. This is especially important as intranets are meant to be community driven. Survey feedback – both positive and negative – can then be used as the basis for future intranet upgrades and enhancements.

Although user surveys will vary depending on the organisation and the intranet, most should be divided into sections to cover the three key intranet components:

Content – quality, relevance and timeliness of intranet content;
Functionality – site navigation, usability and feature set;
Design – overall site design, layout and cosmetics.

However, many organisations commit a very common mistake when running a user survey: they make it far too long. Intranet owners often end up listing every question that comes to mind – sometimes even repeating the same one subconsciously by rewording it. One well-worded question is more effective than five vague ones.

Tips on running an intranet survey

The survey questionnaire should be short and to the point. Users are unlikely to sit around answering a lengthy survey when they have other matters to attend to;
Keep each question concise. If users are forced to read and re-read vague or verbose questions they will quickly tire of the exercise;
The tone and language of the survey should be kept casual and conversational to make it seem less like work. If users have to struggle through something that sounds like a legal document, they won’t bother with it;
The bulk of the survey should be based on multiple choices, such as worded responses (for example strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) or a numerical scale (rating from one-to-five);
Keep multiple choice answers consistent to avoid disorienting users. For example, if true/false multiple choices are used on one question, don’t change to a yes/no response on another;
Offer optional free-form text responses at the end of the survey to allow users to enter comments on aspects of the intranet not covered by the multiple choice questions. For example, ‘If there is one thing I would change about the intranet it would be…’ or simply, ‘Additional comments’
Break the survey up so that no more than five to ten questions appear on one page;
Show users which page of the survey they’re on (for example ‘page 2 of 5’) so that they will know how much more there is to complete;
Allow anonymous submissions and ensure confidentiality of users’ responses. This will increase the likelihood of truthful responses;
Add an optional field for different user departments. This can be used as an identifier to graph usage demographics while still maintaining user anonymity;
Run the survey when the organisation is likely to be at maximum staff. Don’t do it during December/January when people will be on seasonal leave or during the summer when a high number are on holiday;
Always show users the outcome of the survey so that they will know something actually came of it;
Try to avoid running paper-based surveys if possible – they will require too much manual processing.

Technology doesn’t speak

Intranet success should be measured by usage levels and the satisfaction that users get from the system, rather than on the technology used to build it. Far too many intranet owners still believe that intranets are centred on technology. Although it does play a large role it is merely a facilitator: technology is a means to an end, not the end.

In the wider sense, an intranet is the sum total of the tools, content and users. A system that is developed solely for the sake of technology makes about as much sense as praying to your toaster.

An understanding of users’ views, opinions and usage behaviours is essential in defining the future direction of any intranet. It is this understanding that will help intranet owners take the emphasis away from the technology and begin to focus on user-centred development.

But it is important to keep in mind that the process of gauging user response is a two-way street – when users talk, you have to listen.

Paul Chin is an IT consultant and freelance writer.