Welcome to the first of a series of 3 blog articles covering what is arguably one of the most exciting areas of Microsoft to be a part of these days - helping people navigate, find, use and share information using enterprise search solutions. In this series I would like to share with you my view on what enterprise search is all about, why it's important and how to get going with enterprise search in your organisation.
This enterprise search series will cover 3 topics which I hope will give you more food for thought and get you thinking a little bit differently about what search in the enterprise really means. I have been in many customer and partner discussions about enterprise search and have found that there are many variations in opinions about how they can deliver what is needed to help business and IT people easily find information in the enterprise.
The 3 articles will cover these topics:
- Internet Search != Enterprise Search - The Enterprise Search Challenges
- The Microsoft Approach to Enterprise Search - Microsoft's Enterprise Search Strategy
- The Road Ahead - Microsoft's Enterprise Search Roadmap and Vision for the Future
So lets get going with the first in the enterprise search series:
Internet Search != Enterprise Search
Have a question for you: when you're on the internet and you need to find information, where do you go to search for it? Yup..... Live Search..... right? :)
Ok but seriously, you definitely know where to go to find information on the web. Now let's ask this question: when you're at work and you're looking for some information about "that project you've just started working on" or "the new product your company is about to launch" - where do you start looking for it? You're probably not sure where to start looking right now or you're about to ask someone near you who you think might know where to start looking. Or you just don't know..... Right?
This is exactly the problem in the enterprise, there is no clarity on where to look for stuff and there is no proven and repeatable way for people to find information inside the firewall. This is because information is scattered all over the place in many different information silos. Indeed, this is the reason why many leading organisations are spending so much time thinking about how to solve the problem of finding information. Before we get into how Microsoft can help to solve this problem (later in this series,) lets take a look at what the challenges really are. We'll look at the challenges in 3 logical areas: IT challenges, data challenges and user challenges.
IT challenges
Ranking and relevance - first up and definitely most important aspect about search from a user perspective is relevance and ranking but this challenge fits squarely in the IT space and how IT addresses this challenge... Internet search engines all use a similar algorithms for ranking the relevance of the results they give back to you when you run a search and this is the most common misunderstanding about the difference between internet search and enterprise search. When people are creating content for the web they are usually thinking about how people are going to find the information through a search engine and typically put a fair effort into the meta tags that go into the HTML header or metadata fields of a page or document. Another important aspect of web content is linking - the more your page is linked to from another page or the more your page links to other pages has a direct impact on the ranking your page will achieve from the internet search engine's ranking algorithm. This is very suitable and works extremely well for internet search.
However, for the enterprise this does not work because people at work don't create that intranet pages, documents or presentations with the intention of their content being found by the internet search engine and they certainly don't link to other documents or vice versa. This fundamental difference in the way people create and share content at work renders the traditional internet search engine's algorithm irrelevant.
Security - this is another fundamental difference between internet and enterprise search. Think about this: nearly all information on the web is accessible anonymously. Contrast this with information in the enterprise which is by definition all secure. This presents new challenges in ensuring that the search solution you choose is able to respect the security model of data in all your information silos and not just the common ones like file shares and content management systems.
Scalability and management - so granted the end users don't care about this stuff but IT really do because they have to run and manage the solution. The notion of one size fits all doesn't suit enterprises who seek flexibility in how they architect and grow their systems. Likewise, having multiple management interfaces and skill sets for managing solutions drives up cost of ownership.
Data challenges
Structured and unstructured data - searching content on the web is pretty straight forward when you start comparing this to how many information silos exist within just one typical company. On the web everything is accessible over HTTP and the content is usually HTML, PDF, Word, Excel or some other very common file format. In the enterprise there is usually a need to search the document libraries like File Shares, SharePoint or FileNet, the CRM data, the ERP system, a couple of bespoke written applications on Oracle or something else and maybe another type of solution like a 3rd party library.
This presents significant challenges for IT because connecting to all of these data silos and respecting their data and security models without an extensible platform makes it very difficult if not impossible to achieve and if the search solution can't do this, it usually offers less value to the business than if it could connect to all the silos and make information across the enterprise truly searchable.
Expertise location - analysts say that around 80% of a corporate's knowledge is tacit which means it's inside people's heads. This is one of the key drivers for a knowledge management system and culture in the enterprise. It's about getting stuff out of people's heads and into the more permanent corporate memory. Generally KM tends to do well when it's planned and implemented properly but the challenge of finding information still exists and so expertise location is equally, if not more important that searching data already in the enterprise.
In my view there are two ways of achieving expertise location. The one is through asking people to manage their own profiles manually - such as their MySite on SharePoint - the enterprise equivalent of the well known (unless you've been under a rock lately) :) Facebook profile. The smarter and more sustainable way is by using technology to build profiles for people automatically and of course Microsoft is working hard at this right now and you will see some exciting stuff coming in the Office System 14 release next year!
User challenges
Single UI and login and unified results - this is one of the areas that can really make a difference for the users and adoption of search. When we talk about search at Microsoft the aim is to either improve productivity or enable new business models and in the former, how can users be more productive in finding information when they have to use multiple applications to find what they're looking for. Imagine trying to find information about a customer using CRM, ERP, your desktop and the intranet web content as your sources but running individual searches across these repositories. Contrast this with visiting "the intranet" and running one search - or running one search from your desktop and getting all the results back.
The same is true for login to systems. The user is already logged in. Why not just access to the items in the results without being challenged to log in. This is an area that will always need more work as it's not just a search problem. In IT we've been talking about single sign-on for many years now and it's gotten a lot better but not yet perfect...
So these are some of the most common challenges around enterprise search and I hope you're starting to see that searching the enterprise is a different ball game when you compare it to good old internet search where the content is uniform and security is not such a challenge.
Look out for my next article in this series: The Microsoft Approach to Enterprise Search - Microsoft's Enterprise Search Strategy coming to an RSS reader near you soon...