-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: Hibernate Index Problem!!
PostPosted: Mon Jul 13, 2009 5:27 am 
Newbie

Joined: Thu Jun 11, 2009 11:34 am
Posts: 7
I have more than 1200000 records on a table (MySQL)

The code i use for index all this records is this:
Code:
        // Extract Session from EntityManager
      Long numMsgBBDD = countNews();
      long numVueltas = numMsgBBDD / 5000L;
       Session lclSession = (Session)entityManager.getDelegate();
      System.out.println("Starting ReIndexing Routine");

       // Create a FullTextSession, so we can setup a scrollable cursor
       // to keep the memory footprint down while we iterate through the
       // list of objects.
       FullTextSession ftSession = Search.createFullTextSession(lclSession);

       ftSession.setFlushMode(FlushMode.MANUAL);
       int index = 0;
       ftSession.setCacheMode(CacheMode.IGNORE);

       Date start = new Date();
       ScrollableResults results = null;
       System.out.println("Getting Scrollable Cursor for Change Requests...");
       int i = 0;
       while(i<=numVueltas){
       //Scrollable results will avoid loading too many objects in memory
       results = ftSession.createCriteria( PolMensajes.class ).setFirstResult(i*5000+1).setMaxResults(5000).setFetchSize(125).scroll( ScrollMode.FORWARD_ONLY );
       System.out.println("Cursor retrieved... Start Indexing...Vuelta " +i + " de "+ numVueltas);
      
       while( results.next() ) {
           index++;
           ftSession.index( results.get(0) ); //index each element
           // The "25" matches the value established in the persistence.xml used to
           // control batch sizes for Hibernate Search.
           /**
                <property name="hibernate.search.worker.batch_size" value="25"/>
            *
            */
           if (index % 125 == 0) {
             // clear the session contents to keep memory footprint down.
             ftSession.clear();
             System.out.println("Records Indexed: " + index);
           }
       }
       i++;
       results.close();
       }


Only i can index over 30.000 - 50.000 records and i'm getting this error

Code:
11:15:23,287 INFO  [STDOUT] Records Indexed: 30000
11:15:24,066 INFO  [STDOUT] Cursor retrieved... Start Indexing...Vuelta 6 de 204
11:15:24,803 INFO  [STDOUT] Records Indexed: 30125
11:15:25,250 WARN  [arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_58] - Abort of action id 7f000101:85f1:4a5af7fb:32 invoked while multiple threads active within it.
11:15:25,250 WARN  [arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.CheckedAction_2] - CheckedAction::check - atomic action 7f000101:85f1:4a5af7fb:32 aborting with 1 threads active!
11:15:25,251 INFO  [ConnectionManager] forcing batcher resource cleanup on transaction completion; forgot to close ScrollableResults/Iterator?
11:15:25,360 WARN  [JDBCExceptionReporter] SQL Error: 0, SQLState: null
11:15:25,360 ERROR [JDBCExceptionReporter] Transaction TransactionImple < ac, BasicAction: 7f000101:85f1:4a5af7fb:32 status: ActionStatus.ABORTED > cannot proceed STATUS_ROLLEDBACK; - nested throwable: (javax.transaction.RollbackException: Transaction TransactionImple < ac, BasicAction: 7f000101:85f1:4a5af7fb:32 status: ActionStatus.ABORTED > cannot proceed STATUS_ROLLEDBACK)
11:15:25,480 ERROR [application] org.hibernate.exception.GenericJDBCException: could not initialize a collection: [org.poliedrico.awwmail.entity.PolMensajes.polCfgUsrMsg#120616]
javax.faces.el.EvaluationException: org.hibernate.exception.GenericJDBCException: could not initialize a collection: [org.poliedrico.awwmail.entity.PolMensajes.polCfgUsrMsg#120616]
   at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:102)
   at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
   at javax.faces.component.UICommand.broadcast(UICommand.java:387)
   at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:475)
   at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:755)
   at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:82)
   at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100)
   at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
   at com.icesoft.faces.webapp.http.core.JsfLifecycleExecutor.apply(JsfLifecycleExecutor.java:17)
   at com.icesoft.faces.webapp.http.core.ReceiveSendUpdates.renderCycle(ReceiveSendUpdates.java:94)
   at com.icesoft.faces.webapp.http.core.ReceiveSendUpdates.service(ReceiveSendUpdates.java:54)
   at com.icesoft.faces.webapp.http.core.RequestVerifier.service(RequestVerifier.java:26)
   at com.icesoft.faces.webapp.http.common.standard.PathDispatcherServer.service(PathDispatcherServer.java:24)
   at com.icesoft.faces.webapp.http.servlet.MainSessionBoundServlet.service(MainSessionBoundServlet.java:160)
   at com.icesoft.faces.webapp.http.servlet.SessionDispatcher$1.service(SessionDispatcher.java:38)
   at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:19)
   at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
   at com.icesoft.faces.webapp.http.servlet.SessionDispatcher.service(SessionDispatcher.java:55)
   at com.icesoft.faces.webapp.http.servlet.SessionVerifier.service(SessionVerifier.java:19)
   at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
   at com.icesoft.faces.webapp.http.servlet.MainServlet.service(MainServlet.java:91)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
   at com.icesoft.faces.webapp.xmlhttp.BlockingServlet.service(BlockingServlet.java:56)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
   at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:58)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
   at org.jboss.seam.web.HotDeployFilter.doFilter(HotDeployFilter.java:53)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
   at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
   at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
   at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
   at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
   at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
   at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)
   at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
   at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
   at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
   at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
   at java.lang.Thread.run(Thread.java:595)


As you can see in the code, i close always the scrollableResults!

How i can index all this records? Thanks in advance!!!


Juan.


Top
 Profile  
 
 Post subject: Re: Hibernate Index Problem!!
PostPosted: Mon Jul 13, 2009 1:48 pm 
Newbie

Joined: Thu Jun 11, 2009 11:34 am
Posts: 7
Ok, fixed.

The real problem ( i don't see anything about this at the stackTrace ....) is:

Transaction timeout exception.

Go jboss-service.xml and modify the default value '300'.

There is no the best solution but i have to do this by http request. For me is ok

;)


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.