-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 
Author Message
 Post subject: Hibernate 4.1.6 not working with Postgres Partition tables
PostPosted: Tue Sep 25, 2012 11:33 am 
Newbie

Joined: Wed Sep 12, 2012 6:03 pm
Posts: 9
In hibernate 3.3.2.GA, We previously utilized this java program to address ingesting data into our PostgreSQL partitioned tables.

package im.empl.core.service.ingest.postgres;

import org.hibernate.Interceptor;
import org.hibernate.jdbc.Batcher;
import org.hibernate.jdbc.BatchingBatcherFactory;
import org.hibernate.jdbc.ConnectionManager;

public class PostgresPartitionBatcherFactory extends
BatchingBatcherFactory
{
public PostgresPartitionBatcherFactory() {}

public Batcher createBatcher(
ConnectionManager connectionManager,
Interceptor interceptor ) {
return new PostgresPartitionBatcher(
connectionManager,
interceptor);
}
}

then in our hibernate.properties file, we had set this parameter at initialization:

# Enable when using Postgres with partitioned tables
hibernate.jdbc.factory_class=im.empl.core.service.ingest.postgres.PostgresPartitionBatcherFactory

With this configuration we were able to insert data into our Postgres Partitioned tables with no problem, now since we upgraded to Hibernate 4.1.6.Final we are getting errors
like this, since this property is no longer supported.

16:18:01,593 [Thread-23] DEBUG org.hibernate.SQL:104 - insert into EMPLOYEES (ID, DEPT_NUMBER, ADDRESS, GEO_LOCATION, TYPE, CURRENT_DATE)
values (?, ?, ?, ?, 'CONSULTANT', ?)
16:18:01,754 [Thread-23] WARN org.bushe.swing.event.EventService:? - Exception thrown by;EventService subscriber:im.empl.cores.service.ingest.EmplWriter$2@791a9134. Subscriber class:class im.empl.cores.service.ingest.EmplWriter$2
org.bushe.swing.exception.SwingException: Exception handling event topic event class=im.empl.cores.io.parse.impl.Corespec1467StreamReaderImpl$5, event=im.empl.cores.io.parse.impl.Corespec1467StreamReaderImpl$5@3f332b09, topic=null, eventObj=null
org.bushe.swing.exception.SwingException: Exception handling event topic event class=im.empl.cores.io.parse.impl.Corespec1467StreamReaderImpl$5, event=im.empl.cores.io.parse.impl.Corespec1467StreamReaderImpl$5@3f332b09, topic=null, eventObj=null
at org.bushe.swing.event.ThreadSafeEventService.handleException(ThreadSafeEventService.java:2021)
at org.bushe.swing.event.ThreadSafeEventService.handleException(ThreadSafeEventService.java:2009)
at org.bushe.swing.event.ThreadSafeEventService.publish(ThreadSafeEventService.java:975)
at org.bushe.swing.event.ThreadSafeEventService.publish(ThreadSafeEventService.java:904)
at blue.core.event.bushe.BusheApplicationEventService.publish(BusheApplicationEventService.java:28)
at im.empl.cores.service.ingest.DefaultcoresIngestManager.ingest(DefaultcoresIngestManager.java:383)
at im.empl.cores.service.ingest.AutocoresIngestMonitor.ingest(AutocoresIngestMonitor.java:142)
at im.empl.cores.service.ingest.AutocoresIngestMonitor$1.run(AutocoresIngestMonitor.java:104)
Caused by: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:81)
at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:73)
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:57)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2962)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3403)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88)
at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:275)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:326)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1214)
at im.empl.cores.service.ingest.BatchWriter.flush(BatchWriter.java:104)
at im.empl.cores.service.ingest.EmplWriter$1.create(EmplWriter.java:94)
at im.empl.cores.service.ingest.EmplWriter$1.create(EmplWriter.java:1)
at im.empl.cores.service.ingest.EmplWriter.handleEmplSegment(EmplWriter.java:339)
at im.empl.cores.service.ingest.EmplWriter$2.onEvent(EmplWriter.java:179)
at im.empl.cores.service.ingest.EmplWriter$2.onEvent(EmplWriter.java:1)
at org.bushe.swing.event.ThreadSafeEventService.publish(ThreadSafeEventService.java:971)

Based to some research, I found the best way to resolve this error condition, was to modify the code to use the SQLInsert annotation to suppress the row count check when
inserting the row into a partitioned table.

Questions:

Does anyone know why the previous .jdbc.batcher logic managed the partitioned inserts without any issues?
Are there any other alternative that will allow us to insert into a Postgres partition table without making massive code changes?
Would upgrading from 4.1.6.Final to Hibernate 4.1.7 in Linux, fix this problem?

thanks


Top
 Profile  
 
 Post subject: Re: Hibernate 4.1.6 not working with Postgres Partition tables
PostPosted: Tue Sep 25, 2012 5:36 pm 
Newbie

Joined: Wed Sep 12, 2012 6:03 pm
Posts: 9
I previously omitted the return call to the PostgresPartitionBatcher module component to the batcher process.

package im.empl.cores.service.ingest.postgres;

import java.lang.reflect.Field;
import java.sql.PreparedStatement;
import java.sql.SQLException;

import org.apache.log4j.Logger;
import org.hibernate.HibernateException;
import org.hibernate.engine.jdbc.batch.spi.Batch;
import org.hibernate.engine.jdbc.batch.spi.BatchKey;
import org.hibernate.engine.jdbc.batch.spi.BatchObserver;
import org.hibernate.engine.jdbc.spi.JdbcCoordinator;

/**
* See http://codingandmore.blogspot.com/2010/01/how-to-handle-400-billion-rows-in.html
* See http://stackoverflow.com/questions/83093/hibernate-insert-batch-with-postgresql
*
*
*/

public class PostgresPartitionBatcher implements Batch

{

private static Logger logger = Logger.getLogger("PostgresPartitionBatcher");


protected void doExecuteBatch() {

try {
/*
* use reflection to access the private field of the super class
* ugly but necessary
*/
Field field = this.getClass().getSuperclass().getDeclaredField(
"batchSize");
field.setAccessible(true);

int batchSize = field.getInt(this);
if (batchSize != 0) {
try {
// ps.executeBatch();
}
catch (RuntimeException re) {
throw re;
}
finally {
batchSize = 0;
field.setInt(
this,
batchSize);
}
}
}
catch (Exception e) {
logger.error("Batch execution failed.", e);
throw new HibernateException(
"Unable to execute batch.");
}
}


@Override
public void addObserver(BatchObserver arg0) {
// TODO Auto-generated method stub

}


@Override
public void addToBatch() {
// TODO Auto-generated method stub

}


@Override
public void execute() {
// TODO Auto-generated method stub

}


@Override
public PreparedStatement getBatchStatement(String arg0, boolean arg1) {
// TODO Auto-generated method stub
return null;
}


@Override
public BatchKey getKey() {
// TODO Auto-generated method stub
return null;
}


@Override
public void release() {
// TODO Auto-generated method stub

}
}


Top
 Profile  
 
 Post subject: Re: Hibernate 4.1.6 not working with Postgres Partition tables
PostPosted: Thu Feb 21, 2013 10:18 am 
Contributor
Contributor

Joined: Fri Apr 03, 2009 11:07 am
Posts: 11
Hi,
The solution is available in the slide deck for a JBoss World 2009 presentation.

https://www.redhat.com/f/pdf/jbw/jmlodgenski_940_scaling_hibernate.pdf

Regards,
Jeremy


Top
 Profile  
 
 Post subject: Re: Hibernate 4.1.6 not working with Postgres Partition tables
PostPosted: Thu Feb 21, 2013 2:17 pm 
Newbie

Joined: Wed Sep 12, 2012 6:03 pm
Posts: 9
Thanks for your feedback. I did look thru the JBoss 2009 presentation slides early in the troubleshooting process, but since we were not using Annotations, this wasn't a solution. This problem was resolved by adding the <sql-insert check="none">, <sql-update check="none"> clauses in our Hibernate mapping files for every partitioned table. We also had to tweak some of the flushing workflow as well, but it's working well now.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 4 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.