Gone fishing

Fish4, the homes, cars and job-ads website, has recently made a big investment in enterprise-search technology that will serve as a launch pad for a host of new products and services. Enterprise Information spoke to Fish4's chief technology officer Richard Yeo about the company’s mission to deliver the best possible end-user experience.

Enterprise Information (EI): It’s no secret that online consumers are a pretty fickle bunch – if they can’t find what they need on a website very quickly, then they’ll quickly be off to a competing site. What sort of pressure does that put on Fish4 as a company?

Richard Yeo (RY): Well, it puts a great deal of pressure on us. One of our main goals, in fact, is to provide an excellent online experience for users so that they don’t feel the need to defect to one of our competitor’s sites. That’s true whether a user is looking for a job, a car or a new home.

But equally, we want to provide users with choice. The company’s services, Fish4cars, Fish4homes and Fish4jobs have been developed by four regional newspaper groups – Newsquest Media Group, Northcliffe Newspapers, Trinity Mirror and Guardian Media Group Regional Newspapers. Between them, these members represent about 60 per cent of the UK’s regional press, with around 650 daily and weekly regional newspapers. That’s a lot of classified ads. Fish4cars carries one of the largest used-car databases in the country and is updated daily. And Fish4jobs is the leading recruitment site in the UK, with more than 40,000 jobs advertised.

So the challenge for us is not only to give users great choice, but also to deliver results that are carefully tailored to suit their individual preferences and needs.

EI: So when you recently set out to procure ‘next-generation’ search technology, were you hoping to achieve general improvements across existing services or to launch new ones?

RY: A bit of both, actually. Our original remit was to find an enterprise-search product that could improve response rates and results across the whole site. But then a particular project came up, a curriculum vitae (CV) matching service, which is now called Castanet.

Basically, when a recruiter posts a job, the new technology works by automatically ‘reading’ and comparing all of the text in all of the CVs on the Fish4 CV database and selecting the best matches. These top matches are then e-mailed to the recruiter immediately, giving them a clear headstart in their hunt for the best candidates.

The idea for Castanet came from our chief executive Joe Slavin – he sees it as a way that Fish4 can differentiate itself from other job sites, by providing a market-leading candidate delivery service to recruiters. With traditional CV-matching software, recruiters are expected to log in and spend their time entering various keywords in order to initiate Boolean searches. However, these tend to throw up a large number of results, not all of them relevant to the search in hand.

By contrast, Castanet is much more advanced than that. It is far faster and produces much more relevant results. Not only that, it also gets round the problem that many recruiters have – they sign up for an online CV database and are given passwords, but these are subsequently lost, or the password holder leaves the company. Castanet requires a lot less effort on behalf of the recruiter. Every time they submit an advert, for example, they receive ten possible candidates within a matter of minutes.

EI: So how did you find a product that would work well for Castanet but also serve as the basis of other new products and services that you have planned for the future?

RY: It was a long and thorough process. We looked at a number of vendors, including FAST, Autonomy, Endeca and one or two others. One of our major requirements was document-matching capabilities – the ability to analyse a document and find others like it – which cut out a few products straight away. Not all search products can perform that kind of function, so that requirement narrowed the field somewhat.

After a lengthy procurement process, we selected FAST on the basis that not only could it power Castanet (which is now up and running successfully), but it was also suitable for a re-engineering of our CV database that we have planned for the first quarter of 2007. In fact, as 2007 goes on, more and more projects including the main site will use FAST’s enterprise-search technology.

Another thing in FAST’s favour was that the company was prepared to back its claims with financial guarantees. We told the team at FAST, ‘We want to do this many searches per second’, and they told us what hardware specification we would need and offered to underwrite their promise that the technology could handle the sorts of volumes we were talking about.

EI: Tell us a bit more about that database project.

RY: Well, I can’t say too much but I can tell you that, by rewriting the CV database to utilise the search technology, recruiters and large employers will be able to use all the power of FAST to find suitable candidates for their vacancies. Basically, they’ll be able to paste in an entire job specification, rather then a handful of keywords, and click ‘search’. They’ll then have the ability to ‘slice and dice’ results like never before and, when viewing a CV, they’ll be given the option to see other CVs like that one. There’s all sorts of improvements envisaged – it’s pretty exciting. One of the main benefits, though, is that by rewriting the search infrastructure of the CV database, it will be able to handle one million-plus CVs, compared to the 180,000 CVs it currently stores.

EI: What do you mean by ‘slice and dice’?

RY: Well, when you’re searching huge volumes of unstructured information for a handful of highly relevant results, you’re going to need assisted or guided navigation. Using the new search engine, as users search, they can see in real-time how many adverts are fitting their requirements. So if they’re looking for Java programmers, for example, they would type in ‘Java’ and navigators within the search technology will show them how many ads contain the term ‘Java’ in various different categories, such as part-time, full-time, contract and permanent; or by location such as London, Dublin, Glasgow and so on; or by industry sector – financial services, retail and so on. Very few enterprise-search products can guide users through results in this way.

EI: Presumably a search product as sophisticated as this requires a pretty concerted configuration effort to get it up and running?

RY: At this level of sophistication, enterprise-search products simply don’t come out of a box – you don’t just plug them in and away you go. With Castanet, we probably started with our most complex project and that involved a lot of coding outside of the FAST software to get it integrated with our CV database. We spent a lot of time, too, working with FAST’s professional service people.

We did a lot of the taxonomy work ourselves, though, and we’re putting in a lot of ongoing effort into relevancy and results quality. For example, we’ve trained up people internally on FAST and we’re looking to recruit someone whose entire working life will be focussed on looking after taxonomies and making sure that we are producing the best results in response to users’ searches.

As I’ve said, we want the user experience to constantly improve so that we stay ahead of the competition. When you implement a large search project and need a high level of accuracy, you must employ someone to ensure you’re getting the best out of it. This is how we will differentiate ourselves as a business for years to come and if users get stuff back that’s not really relevant, then it reflects badly on our business.

EI: So what will come after the CV database rewrite?

RY: Essentially, we plan to use FAST to power everything at Fish4. We’re in the process of rapid expansion, with plans to add more than 200 employees and a range of exciting new products within the next two years. We’ll even be using FAST for various back-office functions – anything, in fact, where someone is trying to find information.

One potentially valuable use of the technology, for example, could be on Fish4homes – it would be great to enable people to search for property by free text rather than just a postcode. In that way, they could find apartments that have a garden, developments that have their own gym or houses that are in the ‘catchment area’ for a particular school. You could even search for a property by particular features or attributes – by typing in ‘Grade II listed’, for example.

EI: Do you have any advice for other companies looking to implement sophisticated search functions?

RY: Plenty! This was a serious investment for Fish4 – we spent an awful lot of money on it and it’s not something we embarked upon lightly. I think the main thing is that buying enterprise search isn’t like buying any other software product.

I’ve got a lot of experience in procuring software, but search technology is simply not standards-based. It’s not like buying a relational database that’s based on the SQL [structured query language] standard. You’re not just ticking boxes for features and functions, because every product out there on the market is radically different from the next. There are no standards and they all use entirely different algorithms.

So your requirements have to be documented really accurately, in bullet-point format. We also look very hard at the way people search our site all the time – we know how they type things in, for example, and what results they’re most interested in. Once that’s all written down, I’d advise companies to think long and hard about their requirements and about the different products available. And if they have a very big site, they should do a ‘proof of concept’ – that’s a must. More than anything else, more than other technologies, a proof of concept is vital. It’s the only way you can be really sure you’re buying the right enterprise-search product.