-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: Bulk Insert with JPA
PostPosted: Thu Apr 14, 2011 12:41 pm 
Newbie

Joined: Fri Aug 07, 2009 1:18 pm
Posts: 2
hi all,

I am developing a site with seam framework. This site has a task to be processed and then save 2 million records in a sql server database.
When running this task, just save the rows in the database takes about 2 hours.

there is any way to do a bulk insert with jpa?

this is my entity:

Code:
@Entity
@Table(name = "VALOR_REPOSICION_UNITARIO")
@SequenceGenerator(name = "SEQ_VALOR_REPOSICION_UNITARIO", sequenceName = "SEQ_VALOR_REPOSICION_UNITARIO")
public class ValorReposicionUnitario implements Serializable{
      
   private static final long serialVersionUID = 8987003299106089345L;

        private Long id;

        private String sku;
   private String curvatura;
   private Integer packingReal;
   private Integer materialPacking;

        ...

        @Id
   @Column(name = "VRU_ID", precision = 10, scale = 0)
   @GeneratedValue(strategy = GenerationType.AUTO, generator = "SEQ_VALOR_REPOSICION_UNITARIO")
   public Long getId() {
      return id;
   }
   
   public void setId(Long id) {
      this.id = id;
   }

        ...

   @Column(name = "SKU")
   public String getSku() {
      return sku;
   }

   public void setSku(String sku) {
      this.sku = sku;
   }
        ...


and to persist

Code:
   @PersistenceContext
        private EntityManager entityManager;

   private void prepareToSave(
         List<ValorReposicionUnitario> valoresReposicionUnitario) {

      List<ValorReposicionUnitario> valores = new ArrayList<ValorReposicionUnitario>();
      
      int counter = 0;
      for(ValorReposicionUnitario valor : valoresReposicionUnitario){
         
         valores.add(valor);
         counter++;
         
         valor.setValorReposicionUsuario(valor.getValorReposicionCurvatura());
         
         if(counter == 1000){
            
            save(valores);            
            valores = new ArrayList<ValorReposicionUnitario>();
            counter = 0;
         }            
      }
      
      if(valores.size() > 0)
         save(valores);   
      
   }

   @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
   private void save(List<ValorReposicionUnitario> valores) {

      int cantidad = 0;
      for(ValorReposicionUnitario valor : valores){         
         
         if(!Validaciones.isNull(valor.getValorReposicionCurvatura()))
            cantidad = valor.getValorReposicionCurvatura();
                  
         valor.setValorReposicionCurvatura(cantidad<0?0:cantidad);
         
         entityManager.persist(valor);   
      }
      
   }


thanks!


Top
 Profile  
 
 Post subject: Re: Bulk Insert with JPA
PostPosted: Sun Jan 11, 2015 4:34 pm 
Newbie

Joined: Sun Jan 11, 2015 4:28 pm
Posts: 5
Best way is to profile the application and check what is going on - is it eating too much memory? What SQL is generating?

You can also check this post, it explains various traps with batch inserts, how to solve them and also measured performance gains:
http://korhner.github.io/hibernate/hibernate-performance-traps-part-2/

Good luck!


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.