Apache Solr Reference Guide Covering Apache Solr 5

535 Pages·2015·8.74 MB·English

TM
Apache Solr Reference Guide
Covering Apache Solr 5.0
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Apache Lucene, Apache
Solr and their respective logos are trademarks of the Apache Software Foundation. Please see the Apache
Trademark Policy for more information.
Apache Solr Reference Guide
This reference guide describes Apache Solr, the open source solution for search. You can download Apache Solr
from the Solr website at http://lucene.apache.org/solr/.
This Guide contains the following sections:
Getting Started: This section guides you through the Searching: This section presents an overview of the
installation and setup of Solr. search process in Solr. It describes the main components
used in searches, including request handlers, query
Using the Solr Administration User Interface: This
parsers, and response writers. It lists the query parameters
section introduces the Solr Web-based user interface.
that can be passed to Solr, and it describes features such
From your browser you can view configuration files,
as boosting and faceting, which can be used to fine-tune
submit queries, view logfile settings and Java
search results.
environment settings, and monitor and control distributed
configurations. The Well-Configured Solr Instance: This section
discusses performance tuning for Solr. It begins with an
Documents, Fields, and Schema Design: This section
overview of the solrconfig.xml file, then tells you how
describes how Solr organizes its data for indexing. It
to configure cores with solr.xml, how to configure the
explains how a Solr schema defines the fields and field
Lucene index writer, and more.
types which Solr uses to organize data within the
document files it indexes. Managing Solr: This section discusses important topics for
running and monitoring Solr. Other topics include how to
Understanding Analyzers, Tokenizers, and Filters:
back up a Solr instance, and how to run Solr with Java
This section explains how Solr prepares text for indexing
Management Extensions (JMX).
and searching. Analyzers parse text and produce a
stream of tokens, lexical units used for indexing and SolrCloud: This section describes the newest and most
searching. Tokenizers break field data down into tokens. exciting of Solr's new features, SolrCloud, which provides
Filters perform other transformational or selective work comprehensive distributed capabilities.
on token streams.
Legacy Scaling and Distribution: This section tells you
Indexing and Basic Data Operations: This section how to grow a Solr distribution by dividing a large index
describes the indexing process and basic index into sections called shards, which are then distributed
operations, such as commit, optimize, and rollback. across multiple servers, or by replicating a single index
across multiple services.
Client APIs: This section tells you how to access Solr
through various client APIs, including JavaScript, JSON,
and Ruby.
Apache Solr Reference Guide 5.0 2
About This Guide
This guide describes all of the important features and functions of Apache Solr. It is free to download from http://luce
ne.apache.org/solr/.
Designed to provide high-level documentation, this guide is intended to be more encyclopedic and less of a
cookbook. It is structured to address a broad spectrum of needs, ranging from new developers getting started to
well-experienced developers extending their application or troubleshooting. It will be of use at any point in the
application life cycle, for whenever you need authoritative information about Solr.
The material as presented assumes that you are familiar with some basic search concepts and that you can read
XML. It does not assume that you are a Java programmer, although knowledge of Java is helpful when working
directly with Lucene or when developing custom extensions to a Lucene/Solr installation.
Special Inline Notes
Special notes are included throughout these pages.
Note Type Look & Description
Information
Notes with a blue background are used for information that is important for you to know.
Notes
Yellow notes are further clarifications of important points to keep in mind while using Solr.
Tip
Notes with a green background are Helpful Tips.
Warning
Notes with a red background are warning messages.
Hosts and Port Examples
The default port when running Solr is 8983. The samples, URLs and screenshots in this guide may show different
ports, because the port number that Solr uses is configurable. If you have not customized your installation of Solr,
please make sure that you use port 8983 when following the examples, or configure your own installation to use the
port numbers shown in the examples. For information about configuring port numbers, see Managing Solr.
Similarly, URL examples use 'localhost' throughout; if you are accessing Solr from a location remote to the server
hosting Solr, replace 'localhost' with the proper domain or IP where Solr is running.
Paths
Path information is given relative to solr.home, which is the location under the main Solr installation where Solr's
collections and their conf and data directories are stored. When running the various examples mentioned through
out this tutorial (ie: bin/solr -e techproducts) the solr.home will be a sub directory of example/ created
for you automatically.
Apache Solr Reference Guide 5.0 3
Getting Started
Solr makes it easy for programmers to develop sophisticated, high-performance search applications with advanced
features such as faceting (arranging search results in columns with numerical counts of key terms). Solr builds on
another open source search technology: Lucene, a Java library that provides indexing and search technology, as
well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities. Both Solr and Lucene are
managed by the Apache Software Foundation (www.apache.org).
The Lucene search library currently ranks among the top 15 open source projects and is one of the top 5 Apache
projects, with installations at over 4,000 companies. Lucene/Solr downloads have grown nearly ten times over the
past three years, with a current run-rate of over 6,000 downloads a day. The Solr search server, which provides
application builders a ready-to-use search platform on top of the Lucene search library, is the fastest growing
Lucene sub-project. Apache Lucene/Solr offers an attractive alternative to the proprietary licensed search and
discovery software vendors.
This section helps you get Solr up and running quickly, and introduces you to the basic Solr architecture and
features. It covers the following topics:
Installing Solr: A walkthrough of the Solr installation process.
Running Solr: An introduction to running Solr. Includes information on starting up the servers, adding documents,
and running queries.
A Quick Overview: A high-level overview of how Solr works.
A Step Closer: An introduction to Solr's home directory and configuration options.
Solr Start Script Reference: a complete reference of all of the commands and options available with the bin/solr
script.

Installing Solr
This section describes how to install Solr. You can install Solr in any system where a suitable Java Runtime
Environment (JRE) is available, as detailed below. Currently this includes Linux, OS X, and Microsoft Windows. The
instructions in this section should work for any platform, with a few exceptions for Windows as noted.
Got Java?
You will need the Java Runtime Environment (JRE) version 1.7 or higher. At a command line, check your Java
version like this:
$ java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
The output will vary, but you need to make sure you have version 1.7 or higher. If you don't have the required
version, or if the java command is not found, download and install the latest version from Oracle at http://www.oracle
.com/technetwork/java/javase/downloads/index.html.
Installing Solr
Solr is available from the Solr website at http://lucene.apache.org/solr/.
Apache Solr Reference Guide 5.0 4
For Linux/Unix/OSX systems, download the .tgz file. For Microsoft Windows systems, download the .zip file.
When getting started, all you need to do is extract the Solr distribution archive to a directory of your choosing. When
you're ready to setup Solr for a production environment, please refer to the instructions provided on the Taking Solr
to Production page. To keep things simple for now, extract the Solr distribution archive to your local home
directory, for instance on Linux, do:
$ cd ~/
$ tar zxf solr-5.0.0.tgz
Once extracted, you are now ready to run Solr using the instructions provided in the Running Solr section.

Running Solr
This section describes how to run Solr with an example schema, how to add documents, and how to run queries.
Start the Server
If you didn't start Solr after installing it, you can start it by running bin/solr from the Solr directory.
$ bin/solr start
If you are running Windows, you can start Solr by running bin\solr.cmd instead.
bin\solr.cmd start
This will start Solr in the background, listening on port 8983.
When you start Solr in the background, the script will wait to make sure Solr starts correctly before returning to the
command line prompt.
The bin/solr and bin\solr.cmd scripts allow you to customize how you start Solr. Let's work through a few
examples of using the bin/solr script (if you're running Solr on Windows, the bin\solr.cmd works the same as
what is shown in the examples below):
Solr Script Options
The bin/solr script has several options.
Script Help
To see how to use the bin/solr script, execute:
$ bin/solr -help
For specific usage instructions for the start command, do:
$ bin/solr start -help
Start Solr in the Foreground
Since Solr is a server, it is more common to run it in the background, especially on Unix/Linux. However, to start Solr
Apache Solr Reference Guide 5.0 5
in the foreground, simply do:
$ bin/solr start -f
If you are running windows, you can run:
bin\solr.cmd start -f
Start Solr with a Different Port
To change the port Solr listens on, you can use the -p parameter when starting, such as:
$ bin/solr start -p 8984
Stop Solr
When running Solr in the foreground (using -f), then you can stop it using Ctrl-c. However, when running in the
background, you should use the stop command, such as:
$ bin/solr stop -p 8983
The stop command requires you to specify the port Solr is listening on or you can use the -all parameter to stop all
running Solr instances.
Start Solr with a Specific Example Configuration
Solr also provides a number of useful examples to help you learn about key features. You can launch the examples
using the -e flag. For instance, to launch the Tech products example, you would do:
$ bin/solr -e techproducts
Currently, the available examples you can run are: techproducts, dih, schemaless, and cloud.
Getting Started with SolrCloud
Running the cloud example starts Solr in SolrCloud mode. For more information on starting Solr in cloud
mode, see: Getting Started with SolrCloud.
Check if Solr is Running
If you're not sure if Solr is running locally, you can use the status command:
$ bin/solr status
This will search for running Solr instances on your computer and then gather basic information about them, such as
the version and memory usage.
That's it! Solr is running. If you need convincing, use a Web browser to see the Admin Console.
http://localhost:8983/solr/
Apache Solr Reference Guide 5.0 6
The Solr Admin interface.
If Solr is not running, your browser will complain that it cannot connect to the server. Check your port number and try
again.
Create a Core
If you did not start Solr with an example configuration, you would need to create a core in order to be able to index
and search. You can do so by running:
$ bin/solr create -c <name>
This will create a core that uses a data-driven schema which tries to guess the correct field type when you add
documents to the index.
To see all available options for creating a new core, execute:
$ bin/solr create -help
Add Documents
Solr is built to find documents that match queries. Solr's schema provides an idea of how content is structured (more
on the schema later), but without documents there is nothing to find. Solr needs input before it can do much.
You may want to add a few sample documents before trying to index your own content. The Solr installation comes
with different types of example documents located under the sub-directories of the example/ directory of your
installation.
In the bin/ directory is the post script, a command line tool which can be used to index different types of
documents. Do not worry too much about the details for now. The Indexing and Basic Data Operations section has
all the details on indexing.
To see some information about the usage of bin/post, use the -help option.
bin/post can post various types of content to Solr, including files in Solr's native XML and JSON formats, CSV
files, a directory tree of rich documents, or even a simple short web crawl. See the examples at the end of `bin/post
Apache Solr Reference Guide 5.0 7
-help` for various commands to easily get started posting your content into Solr.
Go ahead and add all the documents in some JSON files:
$ bin/post -c gettingstarted example/exampledocs/*.json
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/gettingstarted/update...
Entering auto mode. File endings considered are
xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,l
og
POSTing file books.json (application/json) to [base]
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update...
Time spent: 0:00:00.450
That's it! Solr has indexed the documents contained in those files.
Ask Questions
Now that you have indexed documents, you can perform queries. The simplest way is by building a URL that
includes the query parameters. This is exactly the same as building any other HTTP URL.
For example, the following query searches all document fields for "video":
http://localhost:8983/solr/gettingstarted/select?q=video
Notice how the URL includes the host name (localhost), the port number where the server is listening (8983), the
application name (solr), the request handler for queries (select), and finally, the query itself (q=video).
The results are contained in an XML document, which you can examine directly by clicking on the link above. The
document contains two parts. The first part is the responseHeader, which contains information about the response
itself. The main part of the reply is in the result tag, which contains one or more doc tags, each of which contains
fields from documents that match the query. You can use standard XML transformation techniques to mold Solr's
results into a form that is suitable for displaying to users. Alternatively, Solr can output the results in JSON, PHP,
Ruby and even user-defined formats.
Just in case you are not running Solr as you read, the following screen shot shows the result of a query (the next
example, actually) as viewed in Mozilla Firefox. The top-level response contains a lst named responseHeader a
nd a result named response. Inside result, you can see the three docs that represent the search results.
Apache Solr Reference Guide 5.0 8
An XML response to a query.
Once you have mastered the basic idea of a query, it is easy to add enhancements to explore the query syntax. This
one is the same as before but the results only contain the ID, name, and price for each returned document. If you
don't specify which fields you want, all of them are returned.
http://localhost:8983/solr/gettingstarted/select?q=video&fl=id,name,price
Here is another example which searches for "black" in the name field only. If you do not tell Solr which field to
search, it will search default fields, as specified in the schema.
http://localhost:8983/solr/gettingstarted/select?q=name:black
You can provide ranges for fields. The following query finds every document whose price is between $0 and $400.
http://localhost:8983/solr/gettingstarted/select?q=price:[0%20TO%20400]&fl=id,name,pr
ice
Faceted browsing is one of Solr's key features. It allows users to narrow search results in ways that are meaningful
Apache Solr Reference Guide 5.0 9
to your application. For example, a shopping site could provide facets to narrow search results by manufacturer or
price.
Faceting information is returned as a third part of Solr's query response. To get a taste of this power, take a look at
the following query. It adds facet=true and facet.field=cat.
http://localhost:8983/solr/gettingstarted/select?q=price:[0%20TO%20400]&fl=id,name,pr
ice&facet=true&facet.field=cat
In addition to the familiar responseHeader and response from Solr, a facet_counts element is also present.
Here is a view with the responseHeader and response collapsed so you can see the faceting information clearly.
An XML Response with faceting
<response>
<lst name="responseHeader">
...
</lst>
<result name="response" numFound="9" start="0">
<doc>
<str name="id">SOLR1000</str>
<str name="name">Solr, the Enterprise Search Server</str>
<float name="price">0.0</float></doc>
...
</result>
<lst name="facet_counts">
<lst name="facet_queries"/>
<lst name="facet_fields">
<lst name="cat">
<int name="electronics">6</int>
<int name="memory">3</int>
<int name="search">2</int>
<int name="software">2</int>
<int name="camera">1</int>
<int name="copier">1</int>
<int name="multifunction printer">1</int>
<int name="music">1</int>
<int name="printer">1</int>
<int name="scanner">1</int>
<int name="connector">0</int>
<int name="currency">0</int>
<int name="graphics card">0</int>
<int name="hard drive">0</int>
<int name="monitor">0</int>
</lst>
</lst>
<lst name="facet_dates"/>
<lst name="facet_ranges"/>
</lst>
</response>
The facet information shows how many of the query results have each possible value of the cat field. You could
easily use this information to provide users with a quick way to narrow their query results. You can filter results by
adding one or more filter queries to the Solr request. Here is a request further constraining the request to documents
with a category of "software".
Apache Solr Reference Guide 5.0 10