Hibernate Books

All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 1 post ] 
Author Message
 Post subject: Jgroups backend slave locking exception
PostPosted: Wed Aug 02, 2017 2:27 pm 
Newbie

Joined: Wed Jun 07, 2017 5:44 pm
Posts: 6
I am new to hibernate search. We decided to user hibernate search for my application. We choose jgroups as a backend. Here is my configuration file
Code:
<?xml version="1.0" encoding="UTF-8"?>

<infinispan
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:infinispan:config:7.0 http://www.infinispan.org/schemas/infinispan-config-7.0.xsd
                        urn:infinispan:config:store:jdbc:7.0 http://www.infinispan.org/schemas/infinispan-cachestore-jdbc-config-7.0.xsd"
    xmlns="urn:infinispan:config:7.0"
    xmlns:jdbc="urn:infinispan:config:store:jdbc:7.0">

    <!-- *************************** -->
    <!-- System-wide global settings -->
    <!-- *************************** -->
    <jgroups>
        <!-- Note that the JGroups transport uses sensible defaults if no configuration
            property is defined. See the JGroupsTransport javadocs for more flags.
            jgroups-udp.xml is the default stack bundled in the Infinispan core jar: integration
            and tuning are tested by Infinispan. -->
      <stack-file name="default-jgroups-tcp" path="proform-jgroups.xml" />
    </jgroups>

    <cache-container name="HibernateSearch" default-cache="default" statistics="false" shutdown-hook="DONT_REGISTER">

        <transport stack="default-jgroups-tcp" cluster="venkatcluster"/>

        <!-- Duplicate domains are allowed so that multiple deployments with default configuration
            of Hibernate Search applications work - if possible it would be better to use JNDI to share
            the CacheManager across applications -->
        <jmx duplicate-domains="true" />

         <!-- *************************************** -->
         <!--  Cache to store Lucene's file metadata  -->
         <!-- *************************************** -->
         <replicated-cache name="LuceneIndexesMetadata" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <persistence passivation="false">
                <jdbc:string-keyed-jdbc-store preload="true" fetch-state="true" read-only="false" purge="false">
                    <property name="key2StringMapper">org.infinispan.lucene.LuceneKey2StringMapper</property>
                    <jdbc:connection-pool connection-url="jdbc:mysql://localhost:3306/entityindex" driver="com.mysql.jdbc.Driver" password="pf_user1!" username="pf_user"></jdbc:connection-pool>
                    <jdbc:string-keyed-table drop-on-exit="false" create-on-start="true" prefix="ISPN_STRING_TABLE">
                        <jdbc:id-column name="ID" type="VARCHAR(255)"/>
                        <jdbc:data-column name="DATA" type="BLOB"/>
                        <jdbc:timestamp-column name="TIMESTAMP" type="BIGINT"/>
                    </jdbc:string-keyed-table>
                </jdbc:string-keyed-jdbc-store>
            </persistence>
         </replicated-cache>

         <!-- **************************** -->
         <!--  Cache to store Lucene data  -->
         <!-- **************************** -->
         <distributed-cache name="LuceneIndexesData" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <persistence passivation="false">
                <jdbc:string-keyed-jdbc-store preload="true" fetch-state="true" read-only="false" purge="false">
                    <property name="key2StringMapper">org.infinispan.lucene.LuceneKey2StringMapper</property>
                    <jdbc:connection-pool connection-url="jdbc:mysql://localhost:3306/entityindex" driver="com.mysql.jdbc.Driver" password="pf_user1!" username="pf_user"></jdbc:connection-pool>
                    <jdbc:string-keyed-table drop-on-exit="false" create-on-start="true" prefix="ISPN_STRING_TABLE">
                        <jdbc:id-column name="ID" type="VARCHAR(255)"/>
                        <jdbc:data-column name="DATA" type="BLOB"/>
                        <jdbc:timestamp-column name="TIMESTAMP" type="BIGINT"/>
                    </jdbc:string-keyed-table>
                </jdbc:string-keyed-jdbc-store>
            </persistence>
         </distributed-cache>

         <!-- ***************************** -->
         <!--  Cache to store Lucene locks  -->
         <!-- ***************************** -->
        <replicated-cache name="LuceneIndexesLocking" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <persistence passivation="false">
                <jdbc:string-keyed-jdbc-store preload="true" fetch-state="true" read-only="false" purge="false">
                    <property name="key2StringMapper">org.infinispan.lucene.LuceneKey2StringMapper</property>
                    <jdbc:connection-pool connection-url="jdbc:mysql://localhost:3306/entityindex" driver="com.mysql.jdbc.Driver" password="pf_user1!" username="pf_user"></jdbc:connection-pool>
                    <jdbc:string-keyed-table drop-on-exit="false" create-on-start="true" prefix="ISPN_STRING_TABLE">
                        <jdbc:id-column name="ID" type="VARCHAR(255)"/>
                        <jdbc:data-column name="DATA" type="BLOB"/>
                        <jdbc:timestamp-column name="TIMESTAMP" type="BIGINT"/>
                    </jdbc:string-keyed-table>
                </jdbc:string-keyed-jdbc-store>
            </persistence>
        </replicated-cache>

    </cache-container>

</infinispan>

This is my jgroups-file:
Code:
<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.6.xsd">
   <TCP bind_addr="${jgroups.tcp.address:127.0.0.1}"
        bind_port="${jgroups.tcp.port:7801}"
        enable_diagnostics="false"
        thread_naming_pattern="pl"
        send_buf_size="640k"
        sock_conn_timeout="300"

        thread_pool.min_threads="${jgroups.thread_pool.min_threads:2}"
        thread_pool.max_threads="${jgroups.thread_pool.max_threads:30}"
        thread_pool.keep_alive_time="60000"
        thread_pool.queue_enabled="false"

        internal_thread_pool.min_threads="${jgroups.internal_thread_pool.min_threads:5}"
        internal_thread_pool.max_threads="${jgroups.internal_thread_pool.max_threads:20}"
        internal_thread_pool.keep_alive_time="60000"
        internal_thread_pool.queue_enabled="true"
        internal_thread_pool.queue_max_size="500"

        oob_thread_pool.min_threads="${jgroups.oob_thread_pool.min_threads:20}"
        oob_thread_pool.max_threads="${jgroups.oob_thread_pool.max_threads:200}"
        oob_thread_pool.keep_alive_time="60000"
        oob_thread_pool.queue_enabled="false"
   />
   <S3_PING access_key=""
            secret_access_key=""
            location="mybucket"

   />
   <MERGE3 min_interval="10000"
           max_interval="30000"
   />
   <FD_SOCK />
   <FD_ALL timeout="60000"
           interval="15000"
           timeout_check_interval="5000"
   />
   <VERIFY_SUSPECT timeout="5000" />
   <pbcast.NAKACK2 use_mcast_xmit="false"
                   xmit_interval="1000"
                   xmit_table_num_rows="50"
                   xmit_table_msgs_per_row="1024"
                   xmit_table_max_compaction_time="30000"
                   max_msg_batch_size="100"
                   resend_last_seqno="true"
   />
   <UNICAST3 xmit_interval="500"
             xmit_table_num_rows="50"
             xmit_table_msgs_per_row="1024"
             xmit_table_max_compaction_time="30000"
             max_msg_batch_size="100"
             conn_expiry_timeout="0"
   />
   <pbcast.STABLE stability_delay="500"
                  desired_avg_gossip="5000"
                  max_bytes="1M"
   />
   <pbcast.GMS print_local_addr="false"
               join_timeout="15000"
   />
   <MFC max_credits="2m"
        min_threshold="0.40"
   />
   <FRAG2 />
</config>


This is my flush-tcp file:-
Code:
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
    <TCP bind_port="7801"/>
   <S3_PING access_key=""
            secret_access_key=""
            location=""

   />
    <MERGE3/>
    <FD_SOCK/>
    <FD/>
    <VERIFY_SUSPECT/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE/>
    <pbcast.GMS/>
    <MFC/>
    <FRAG2/>
    <pbcast.STATE_TRANSFER/>
    <pbcast.FLUSH timeout="0"/>
</config>


These are hibernate settings:
Code:
        propertyMap.put("hibernate.search.default.directory_provider", "infinispan");
        propertyMap.put("hibernate.search.lucene_version", KeywordUtil.LUCENE_4_10_4);
        propertyMap.put("hibernate.search.infinispan.configuration_resourcename","hibernate-search-infinispan-config.xml");
        propertyMap.put("hibernate.search.default.​worker.execution","sync");
        propertyMap.put("hibernate.search.default.​worker.backend","jgroups");
        propertyMap.put("hibernate.search.services.jgroups.configurationFile","flush-tcp.xml");
        propertyMap.put("hibernate.search.default.exclusive_index_use","true");


Initially we start the cluster with one node with the above configuration. Depends on the load we will add nodes to the cluster. This is our architecture.
Assume that 10-00 AM we started the cluster. only node will become master node. and everthing is fine.
10-10 Am we added one more node to the cluster with slight config change. Here is the change
Code:
propertyMap.put("hibernate.search.default.exclusive_index_use","false");

I created an index through second node. Then the locking error comes up. here is the error.
Quote:
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@46578a74


Problem:- In theory second node should become slave and it should never acquire lock on the index. It should indicate the master node to create the index through jgroups channel. But its not happening. Can one of you please help me on this. Our production system is in problem. Please help me on this.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.