Setting up and playing with Apache Solr on Tomcat

A while back a had a little time to play with Solr, and was instantly blown away by the performance we could achieve on some of our bigger datasets.
Here is some of my initial setup and configuration learnings to maybe help someone get it up and running a little faster.
Starting with setting both up on windows.

Download and extract Apache Tomcat and Solr and copy into your working folders.

Tomcat Setup

If you want tomcat as a service install it using the following:
bin\service.bat install
Edit the tomcat users under conf.:

<role rolename="admin"/>
<role rolename="manager-gui"/>
<user username="tomcat" password="tomcat" roles="admin,manager-gui"/>

If you are going to query Solr using international characters (>127) using HTTP-GET, you must configure Tomcat to conform to the URI standard by accepting percent-encoded UTF-8. Add: URIEncoding=’UTF-8′

<connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443"
URIEncoding="UTF-8" />

to the conf/server.xml

Copy the contents of the example\solr your solr home directory D:\Java\apache-solr-3.6.0\home
create the code fragment on $CATALINA_HOME/conf/Catalina/localhost/solr.xml pointing to your solr home.

<?xml version="1.0" encoding="UTF-8"?>
<context docBase="D:\Java\apache-tomcat-7.0.27\webapps\solr.war" debug="0" crossContext="true" >
    <environment name="solr/home" type="java.lang.String" value="D:\Java\apache-solr-3.6.0\home" override="true" />
</Context>

Startup tomcat, login, deploy the solr.war.

Solr Setup

It should be available at http://localhost:8080/solr/admin/ To create a quick test using SolrJ the creates and reads data: Grab the following Maven Libs:

<dependency>
       <groupid>org.apache.solr</groupId>
       <artifactid>apache-solr-solrj</artifactId>
       <version>3.6.0</version>
       <type>jar</type>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>org.apache.httpcomponents</groupId>
       <artifactid>httpclient</artifactId>
       <version>4.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>org.apache.httpcomponents</groupId>
       <artifactid>httpcore</artifactId>
       <version>4.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>org.apache.james</groupId>
       <artifactid>apache-mime4j</artifactId>
       <version>0.6.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>org.apache.httpcomponents</groupId>
       <artifactid>httpmime</artifactId>
       <version>4.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>org.slf4j</groupId>
       <artifactid>slf4j-api</artifactId>
       <version>1.6.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>commons-logging</groupId>
       <artifactid>commons-logging</artifactId>
       <version>1.1.1</version>
       <scope>compile</scope>
   </dependency>
   <dependency>
       <groupid>junit</groupId>
       <artifactid>junit</artifactId>
       <version>4.9</version>
       <scope>test</scope>
   </dependency>

JUnit test:

package za.co.discovery.ecs.solr.test;
 
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.Collection;
 
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.HttpSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.SolrInputDocument;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
 
 
@RunWith(JUnit4.class)
public class TestSolr {
 
    private SolrServer server;
 
 
    /**
     * setup.
     */
    @Before
    public void setup() {
        server = new HttpSolrServer("http://localhost:8080/solr/");
        try {
            server.deleteByQuery("*:*");
        } catch (SolrServerException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    /**
     * Test Adding.
     *
     * @throws MalformedURLException error
     */
    @Test
    public void testAdding() throws MalformedURLException {
 
        try {
 
            final SolrInputDocument doc1 = new SolrInputDocument();
            doc1.addField("id", "id1", 1.0f);
            doc1.addField("name", "doc1", 1.0f);
            doc1.addField("price", 10);
 
            final SolrInputDocument doc2 = new SolrInputDocument();
            doc2.addField("id", "id2", 1.0f);
            doc2.addField("name", "doc2", 1.0f);
            doc2.addField("price", 20);
 
            final Collection<solrinputdocument> docs = new ArrayList<solrinputdocument>();
            docs.add(doc1);
            docs.add(doc2);
 
            server.add(docs);
            server.commit();
 
            final SolrQuery query = new SolrQuery();
            query.setQuery("*:*");
            query.addSortField("price", SolrQuery.ORDER.asc);
            final QueryResponse rsp = server.query(query);
 
            final SolrDocumentList solrDocumentList = rsp.getResults();
 
            for (final SolrDocument doc : solrDocumentList) {
                final String name = (String) doc.getFieldValue("name");
                final String id = (String) doc.getFieldValue("id"); //id is the uniqueKey field
                System.out.println("Name:" + name + " id:" + id);
            }
 
        } catch (SolrServerException e) {
            e.printStackTrace();
            Assert.fail(e.getMessage());
        } catch (IOException e) {
            e.printStackTrace();
            Assert.fail(e.getMessage());
        }
    }
 
}

Adding data directly from the DB

Firstly you need to add the relevant DB libs to the add classpath. Then create data-config.xml as below, if you require custom fields, those can be specified under the fieldstag in the schema.xml shown below the dataconfig.xml

<dataconfig>
 <datasource name="jdbc" driver="oracle.jdbc.driver.OracleDriver" url="jdbc:oracle:thin:@localhost:1525:DB" user="user" password="pass"/>
    <document name="products">
        <entity name="item" query="select * from demo">
            <field column="ID" name="id" />
            <field column="DEMO" name="demo" />
 
            <entity name="feature" query="select description from feature where item_id='${item.ID}'">
                <field name="features" column="description" />
            </entity>
            <entity name="item_category" query="select CATEGORY_ID from item_category where item_id='${item.ID}'">
                <entity name="category" query="select description from category where id = '${item_category.CATEGORY_ID}'">
                    <field column="description" name="cat" />
                </entity>
            </entity>
        </entity>
    </document> 
</dataConfig> 

A custom field in the schema.xml:

<fields>
  <field name="DEMO" type="string" indexed="true" stored="true" required="true" /> 
</fields

Add in the solrconfig.xml make sure to point the the data-config.xml, the handler has to be registered in the solrconfig.xml as follows

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
  <lst name="defaults">
    <str name="config">data-config.xml</str>
  </lst>
</requestHandler>

Once that is all setup a full import can be done with the following:

http://localhost:8080/solr/admin/dataimport?command=full-import

Then you should be good to go with some lightning fast data retrieval.

Reference: Setting up and playing with Apache Solr on Tomcat from our JCG partner Brian Du Preez at the Zen in the art of IT blog.

Related Whitepaper:

Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions

Get ready to program in a whole new way!

Functional Programming in Java will help you quickly get on top of the new, essential Java 8 language features and the functional style that will change and improve your code. This short, targeted book will help you make the paradigm shift from the old imperative way to a less error-prone, more elegant, and concise coding style that’s also a breeze to parallelize. You’ll explore the syntax and semantics of lambda expressions, method and constructor references, and functional interfaces. You’ll design and write applications better using the new standards in Java 8 and the JDK.

Get it Now!  

Leave a Reply


five + = 6



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

15,153 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books