Post on 28-Jan-2015
description
Enterprise Search in SharePoint 2010
Bob GermanTechnology ArchitectMicrosoft Technology Center | Boston
Agenda
SharePoint Search OptionsSharePoint Search EngineFAST for SharePoint
Social SearchContent AcquisitionFederation
Scale – content and query scale becomes a non-issue
User experience –guide users with rich, visual navigation
Social and people search – search unlocks the value of the people and expertise
Relevance –improving our matching and ranking capabilities and language support
Platform –tools to customize, extend, or build new applications.
Management & reliability –reliable crawling, scriptable admin, improved monitoring.
Rich Content Processing–extract metadata from my enterprise content
You Asked, We Listened!
Quick, easy, powerful search
(for free!)
Complete intranet search
High-end search delivered through
SharePoint
A Family of Search Solutions
Family Values
• Common UI Framework
• Social search features and integration
• SharePoint platform integration
• End user and site administrator enablement
• Common Connector Framework (BDC)
• APIs and developer experience
• Admin & deployment capabilities
• Operations advantages (SCOM, scripting)
SharePoint Search
End Users Content +
Profiles
FAST Search for SharePoint
FAST Search Extends SharePoint ServerSuperset of capabilities; common base functionality and platform
QueriesPeople
Unique Social Search from SharePoint Server
Common connectors and content APIs
Common query and federation APIs
Best High-End Content Search based on FAST ESP
Common developer experience and IT Pro Foundation Content
SharePoint Search Engine
What’s new in 2010?
SharePoint 2010 Shared Service Applications
SSP
SharePoint 2007 Model
http://hrweb/
SearchUser
Profiles
Excel Calc
Corp Farm
Business
Data
http://hrweb/
Corp Farm
SharePoint 2010 Model
http://itweb/ http://itweb/
Search
User Profiles
Excel
Calc
Visio
3rd party Servic
e
HRBusine
ss
Data Office
Web Apps
IT Searc
h
SharePoint Search SA
Each Search Service Application (SSA):
Administration component , databaseQuery Topology
Query Components (1+)Property Databases (1+)
Crawl TopologyCrawl Components (1+)Crawl Databases (1+)
Services (Query, Admin)
Default Search Configuration
Out of the Box:1 Admin component1 Crawl component1 Query componentDatabases on drive specified in post-setup configuration
Fully functional:Crawl contentDevelop customizations
CrawlAdmi
n
Property DB
Admin DB
Crawl DB
Query
(other SharePoint DBs)
Component Scale-out
Query:SegregatePartition for scalePassive mirrors provide failoverActive mirrors increase throughput
Crawl Additional crawl componentsCrawl Database
Crawl
Query
Admin
Crawl DB
Property DB
Admin DB
Crawl DB
Crawl
Crawl DB
Crawl
Crawl
100 million items!
Query Q2Q1
Q2 Q1
Admin DB
Q6
Q5
Q5
Q6
Q4
Q3
Q3
Q4
Q2
Q1
Q1
Q2
Manageability – Monitor Crawl
“Levels” of reports
Current rateCrawl rate per typeCrawl processing
Filter by SSA, timeOptimize crawl resources to achieve freshness goals.
Manageability – Monitor Query
“Levels” of reports
Current rateOverall Query LatencyDetailed Backend Query Latency
Optimize query latencyEnable verbosequery latency, trend
FAST Search Engine
High-end Search for SharePoint
Go Beyond the Search BoxVisual, Conversational Search
Thumbnails
Sorting on any property
Similarity Search
Visual Best Bets
Scrolling PowerPointPreviewsRefinement with
countson any property
FAST for SharePointArchitecture
DocumentProcessing
Indexing SearchQuery &Result
ContentConnectors• SharePoin
t• FAST• CCF
SharePointAPI’s
Administration
FAST Search Engine Shared with SharePoint Search Engine
SearchCenter
Users
Content PipelineAdvanced Content Processing Out-of-the-Box
Configurable content processing pipeline
Transform and process content automatically
Strong linguistic coverage80+ languages detected45 languages with advanced linguistics features
Property Extraction to create new metadata
Prebuilt extractors: Person, Location, CompanyGeneric extractors to extendExtensibility
Processing & refinement
…
FormatConversio
n
LanguageDetection
EntityExtraction
Configurable
Stages
Mapper
REDMOND, Wash., and OSLO, Norway — Jan. 8, 2008
Microsoft Corp. (Nasdaq “MSFT”) today announced that it will make an offer to acquire Fast Search & Transfer ASA (OSE: “FAST”), a leading provider of enterprise search solutions, through a cash tender offer for 19.00 Norwegian kroner (NOK) per share. This offer represents a 42 percent premium to the closing share price on Jan. 4, 2008 (the last trading day prior to this announcement), and values the fully diluted equity of FAST at 6.6 billion NOK (or approximately $1.2 billion U.S. ). FAST’s board of directors has unanimously recommended that its shareholders accept the offer.
Location
Company
Company
Company
Enable Conversational SearchDeep Refiners with Exact Counts
Search results can be refined based on:● Metadata from SharePoint (such as Author)
● Enriched meta data based on entity extraction (such as
Company)
Custom refiners can be made by:● Enabling new Managed Properties to be used as refiners
● Configuring the Refinement Web Part to display the new
refiner
User Context MattersAlan Brewer, Sales
What should I know about selling ERP consulting?
Renee Lo, Engineer
What should I know about implementing ERP?
Large Scale Deployment
Add columns to extend document capacityAdd rows to extend query capacity or fault tolerance
Indexer/Search
Indexer/
SearchRow 0
Indexer/
Search
Indexer/
Search
Indexer/
Search
Indexer/Search
Row 1
Column 0
1
Indexer/
Search
Indexer/
Search
2 3
Social Search
Go Behind the Search BoxConnect with People and Expertise
Phonetic and nickname matching
Improved result layout and hit-
highlightingRecently authored
content
Refine by query type, and many
other pivots
Sort by relevance, name
or social distance
Vanity search
Leveraging the Managed Metadata Service for better People Search
Out of box:Responsibilities, Interest, Skills, Schools, are all taxonomy propertiesAutomatic refinement on relevant pivots
ExtensibilityAdding a new taxonomy property to the profile store
Click through improves relevanceThe wisdom of the masses helps the masses
Documents are boosted by ‘popularity’: if many users find a document useful, it must be useful.
Search suggestionsHelping users form better queries
Pre-query suggestions
Post-query suggestions
Social tags
Recall
Ranking
Refinement
Results
Results with social tags are returned.
Results with social tags are boosted.
Users can filter results on social tags.
Top social tags are displayed as metadata on each result.
Content Acquisition
Feeding the Search Engines
Making Content Searchable
Use Query FederationQuery across external indexesBased on OpenSearch standard(http://www.opensearch.org/)
Use Indexing ConnectorsCrawl, enumerate and create local index of contentUse out-of-box connectors or write custom connector
Shared Out-of-box Indexing Connectors in SharePoint 2010SharePoint content (Protocol Handler -
PH)File shares (PH)Web sites (PH)People profiles (PH)Lotus Notes™ (Connector Framework - CF)Exchange public folders (CF)External systems(CF)Documentum™ content (CF)
Fast Search out-of-box Connectors
Enterprise crawlerGood for large scale crawling across multiple nodesSupports dynamic data including JavaScript
JDBC™ connectorSimple configuration using SQL commandsSupports push based crawlingSupports change detection through checksums
SharePoint Connector
Accesses data through SharePoint web service (using Windows credentials)Supports full crawl through enumeration of content and incremental crawl through change logBuilt-in support for security trimming (Windows and pluggable)
Connector Framework
Easy to buildWrite managed code instead of PH interfaceBuilt connectors shared across SharePoint Search and FAST search‘No-code’ solution for Databases and Web Services
Easy to manageUse existing user interfaces for crawl configurationFaster iteration on connectivity using SharePoint Designer (SPD)
Shared advantages across SharePointCan use other BCS features such as external lists
Connector Framework Architecture
Search Engine/IndexSearch BCS
Runtime
WCF Databases
.NET BCS
Connector
Execution call routing through BCSConnectors using BCS model file
Custom BCS
Connector
Search user queries
BCSProfile Pages or External Lists
Search results page – results
based on metadata
Search page points to Profile Page or custom page to show search result
Fetch data from repository
Protocol Handlers
Enabling Connectivity
Every indexing connector needsA BDC model file to express connection information and the structure of the backendA BDC Connector for code to execute when accessing the backend
Out-of-box BDC ConnectorsDatabaseWCF (Web Services).NET (custom code)
Federation
Combining Search Results
Eliminate Compromise
Web
Files
Databases
Lotus Notes
Documentum
Easy to Build
OpenSearch Federation
Enterprise Content Mgmt
Systems
Line of Business Apps
Custom
Apps
Asset Mgmt
systems
Indexing Connectors
Easy to Connect
Easily Connect to Corporate Assets that live Outside of SharePoint
SharePoint
Exchange Server
…
“Out of the Box” FederationPeople Search and Related SearchesBing and other OpenSearch providersWindows 7 and IE8 searching SharePoint 2010Rich RSS feeds and federation generator
How OpenSearch federation works
HTTP request with query in the URL:http://www.site.com/srchrss.aspx?q={searchTerms}
RSS / Atom results:RSS results with <title>, <link>, <description>Best sources also include: <pubdate>, <author>, <category>, <media:thumbnail>Optionally include custom metadata: <recordid>, <projectname>, <contactnumber>
HTTP request
RSS/Atom Results
Search Provider
Search Client
Deploying a federation connectorUse or create a searchable feed
Use an existing searchable feed or create your own.Anything searchable – takes a query as a URL parameter.Must return results in RSS or Atom formatted XML.Creating a searchable feed is simple. Just code a basic ASPX page.
OpenSearchPage
Search Provider
Search Client
Make Search Pervasive!Deploy OpenSearch (OSDX files) Everywhere
SharePoint 2010:
Import the file.
Add it to a Search Center.
Windows 7:
Pull: Add a link to the OSDX file on the search center or team site.
Push: Deploy the OSDX file via Group Policy.
IE8
Link to Search Provider
Discoverable Search Provider
Extend and Create Search Web PartsThe OOB Web Parts are no longer sealed!
Complete enterprisesearch solution
High-end search delivered through SharePoint
End UserGo Beyond the Search Box
Great out of box search experience
Most visual and dynamic user experience
IT ProEliminate Compromise
Enterprise class search system integrated
with SharePoint
Highly capable and easy to manage
DeveloperDo more with Search
Open and flexible search platform
The power to build a whole new class of search
applications
Microsoft Enterprise Search - ProductivityProducts for Every Search Need
FAST Search for SharePoint and ESPA new choice; a different world
Bringing high-end search to the mainstream market
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after
the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Great OOB search experience Most visual and dynamic user experience
One-stop search center to find answers fast
Excellent OOB relevanceFederated resultsPC, phone, or browser access
Same capabilities, plus:Broader, better language support Deeper query syntax
Refinement to explore information quickly
Metadata based refiners Related searchesView in Browser
Same capabilities, plus:Deep refiners with countsSimilarity searchSorting on any property
Social search to connect with people and expertise
Phonetic & nickname matchingRelevance and refinersIntegrated presence
Same capabilities
Search gets better with use
Click-through drives relevanceColleague & Expertise suggestionsQuery suggestions
Same capabilities
Visual cues for rapid recognition of information
Thumbnails & Scrolling PreviewsVisual Best Bets
Contextual to meet the needs of diverse groups
User context from user profileMultiple relevance profiles
Go Beyond the Search BoxEnd User perspective
Enterprise class search system integrated with SharePoint
Highly capable and easy to manage
Industrial strength scale-out and performance
Enterprise Scale-out (to 100M docs)Full Fault ToleranceNative 64 bit; Hyper-V support
Same capabilities, plus:Extreme scale-out
Easy deploymentWizard - driven installationConsolidated search dashboard
Same capabilities
Enterprise-class manageability
PowerShell supportSCOM support Full search reporting
Same capabilities
Secure, broad connectivity
Full set of connectors OOB Easy to add new sources via BDCStrong security
Same capabilities, plus:Enterprise Web Crawler
Advanced content processing OOB
Content Processing pipelineMetadata Extraction
Easy to configure high-end user experiences
Easy setup of User Context, Visual Best Bets, PromotionGive end users control over sorting, ranking, navigation
Eliminate CompromiseIT Pro perspective
Do More with SearchDeveloper Perspective
The power to build a whole new class of search applications
Same capabilities, plus:
Public web parts with high end
capabilities
Add custom property extractors
Extend content processing
Include external data in
relevance
Build multiple relevance profiles
Extend user context
Use advanced query capabilities
to create powerful applications
Open and flexible search platform
Public web parts
New connector framework
Integrated with Business
Connectivity Services (BCS)
Integrate search with BI,
Workflow,
Social, & Collab
BDC tooling built into
SharePoint Designer
Application tooling in VS2010
Query Object Model
Concepts
Search Technology Concepts
Content Sources - Host the content we want to return in main results
Content
Content
Content
Federated Source
Crawling - Traverse URL space to record items in search catalog
Indexing - Extract information from items to enable efficient matching
Query Servers - Accept query requests from users and return results
Search Center - UI for users to issue queries and interact with results
Query Federation - Return results from non-SharePoint Indexes
Crawler
Indexer
Query Servers
Index Files
Indexing Connectors - Know how to process different content sources
Index Partition - Subset of the overall index