2012年12月16日日曜日

SolrCloud try again!

SolrCloudをまた試してみた
I tried solrcloud again.
http://wiki.apache.org/solr/SolrCloud

Example C: Two shard cluster with shard replicas and zookeeper ensemble

Regarding Example C section, I got so many error. so describe my log.
ExampleCをやってたのだけどとてもエラーがでてうまくかなかったがなんとかうまくいった。それを書いておきます。

<結論 Conclusion>
Only one server setting is needed in example/solr/zoo.cfg. It goes well.
Describing 2 or more server goes error.
like this.
----------------------

tickTime=2000
initLimit=10
syncLimit=5
#I do not know this port num is properly.
server.1=localhost:9983:10983


----------------------
結論として上記のようにserver1を設定しておくとうまくいく。


If zoo.cfg has 2 or more servers. you will got like this error. it seems only 1 server is accepted.
zoo.cfgにサーバーを2つ以上指定してるとこんなエラー
どうもzookeeper用のアドレスは一つだけのようだ
server.2=(address):port:port     e.g) server.1=localhost:9983:10983

EVERE: null:java.lang.IllegalArgumentException: port out of range:-1
at java.net.InetSocketAddress.<init>(InetSocketAddress.java:101)
at org.apache.solr.cloud.SolrZkServerProps.setClientPort(SolrZkServer.java:309)
at org.apache.solr.cloud.SolrZkServerProps.getMySeverId(SolrZkServer.java:278)
at org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:453)
at org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:90)
at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:208)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:462)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:114)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:754)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:258)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1221)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:699)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:454)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:263)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
at org.eclipse.jetty.start.Main.start(Main.java:602)
at org.eclipse.jetty.start.Main.main(Main.java:82)

Then if zoo.cfg has no server in zoo.cfg and start the solr with 2 or more servers in DzkHost. like this error.
zoo.cfg になにもサーバ指定しないで、DzkHostに2つ以上指定してるとこんなエラー

java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -DnumShards=2 -jar start.jar

Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

Off course zoo.cfg has only one server like server.1=(address):port :port, you do not got error after starting solr with the comand above.
でももちろんzoo.cfgに1つだけserver.1=(address):port :portがあるとエラーにならない

And if server has only one port in zoo.cfg. you got like this error
zoo.cfgに
server.1=(address):port     //no second port    ポート1つだけにするとこんなエラー

2012/12/16 4:35:25 org.apache.solr.common.SolrException log
致命的: null:java.lang.IllegalArgumentException: Missing election port for server: 2
at org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:417)
at org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:90)
at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:208)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:462)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:114)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:754)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:258)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1221)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:699)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:454)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:263)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
at org.eclipse.jetty.start.Main.start(Main.java:602)
at org.eclipse.jetty.start.Main.main(Main.java:82)

Return back to solrcloud wiki, do comand below with only one server in zoo.cfg.
そこでzoo.cfgに1つだけサーバーを指定して
java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -DnumShards=2 -jar start.jar

cd example2
java -Djetty.port=7574 -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar

cd exampleB
java -Djetty.port=8900 -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar

until here it goes well but...next comand got error in example2B.
ここまではうまくいく。以下をすると
cd example2B
java -Djetty.port=7500 -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar

you got like this error.
こんな感じのエラーがどばーとでる
2012/12/16 5:38:53 org.apache.solr.cloud.ZkController getConfName
情報: Could not find collection configName - pausing for 3 seconds and trying again - try: 1
2012/12/16 5:38:56 org.apache.solr.cloud.ZkController getConfName
情報: Could not find collection configName - pausing for 3 seconds and trying again - try: 2
2012/12/16 5:38:59 org.apache.solr.cloud.ZkController getConfName
情報: Could not find collection configName - pausing for 3 seconds and trying again - try: 3
2012/12/16 5:39:02 org.apache.solr.cloud.ZkController getConfName
情報: Could not find collection configName - pausing for 3 seconds and trying again - try: 4
2012/12/16 5:39:05 org.apache.solr.cloud.ZkController getConfName
情報: Could not find collection configName - pausing for 3 seconds and trying again - try: 5
2012/12/16 5:39:08 org.apache.solr.cloud.ZkController getConfName
致命的: Could not find configName for collection collection1
2012/12/16 5:39:08 org.apache.solr.core.CoreContainer create
致命的: Unable to create core: collection1
org.apache.solr.common.cloud.ZooKeeperException: Could not find configName for collection collection1 found:null

So then added "-DzkRun".
そこでDzkRunを追加してうまくいった。

java -Djetty.port=7500 -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar

Success!

でSolr4.0についてるzkCli.shでのぞいてみようと思ったけどうまくいかない。どうも機能的にしぼられてるっぽいので、本家からDLして⬇のこまんど打ってうまく行きました。

To see config from zkCli.sh, I got error from using zkCli.sh embedded in Solr4.0.
So download zookeeper from original one,
http://zookeeper.apache.org/releases.html

do this command. you can see the config file.
bin/zkCli.sh -server localhost:2181


こんど別サーバにzookeeperを入れてやってみようかと思います。server2,3とか増やしてみます。

<additional>
sometimes I got like this error. but I could not reproduce it so i do not know the resolving way....


Dec 15, 2012 5:08:51 PM org.apache.solr.servlet.SolrDispatchFilter init
SEVERE: Could not start Solr. Check solr/home property and the logs
Dec 15, 2012 5:08:51 PM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.IllegalArgumentException: solr/zoo_data/myid file is missing
at org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:460)
at org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:90)
at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:208)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:462)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:114)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:754)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:258)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1221)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:699)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:454)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:263)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
at org.eclipse.jetty.start.Main.start(Main.java:602)
at org.eclipse.jetty.start.Main.main(Main.java:82)

そこで以下のように適当に作ってしまうとまたこんなエラー
then I made the directory and myid, I got the error again like this.....

mkdir solr/zoo_data/
touch solr/zoo_data/myid

Dec 15, 2012 5:28:12 PM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.IllegalArgumentException: serverid null is not a number
at org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:474)
at org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:90)
at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:208)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:462)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:114)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:754)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:258)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1221)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:699)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:454)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:36)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
at org.eclipse.jetty.server.Server.doStart(Server.java:263)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
at org.eclipse.jetty.start.Main.start(Main.java:602)
at org.eclipse.jetty.start.Main.main(Main.java:82)

2 件のコメント:

  1. Hi, I have hitted the same error and found this post while searching the answer. Zookeper docs helped me much. http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
    in yours solr/zoo_data/myid file you have to write a number 1 or 2 or etc.

    返信削除