Hello everyone. I've been developing a tool to extract data from an ARSys system, convert it to data that I can use, and persist it on my own database.
Everything works, but the persisting part is going very slow :(
I've tried looking around on the internet before asking, but I couldn't fix it myself.
First I'll post some possibly useful code.
The entities:
Code:
@MappedSuperclass
public class Ticket implements java.io.Serializable {
@Id
@Column(name = "ID")
private long id;
@Temporal(TemporalType.TIMESTAMP)
@Column(name="INSERT_TIMESTAMP", length=7)
private Date insertTimestamp;
Code:
@Entity
@Table(name="TICKET_ALL", uniqueConstraints = @UniqueConstraint(columnNames="ID_TICKET_HDC"))
public class TicketReclami extends Ticket {
@Column(name="ID_TICKET_HDC", unique=true, length=4000)
private String idTicketHdc;
.......
@OneToMany(cascade=CascadeType.ALL, fetch=FetchType.LAZY, mappedBy="ticketAll")
private Set<TicketReclamiHistory> ticketHistories = new HashSet<TicketReclamiHistory>(0);
Code:
@Entity
@Table(name="TICKET_HISTORY")
public class TicketReclamiHistory implements java.io.Serializable {
@Id
@GeneratedValue(strategy= GenerationType.SEQUENCE, generator = "seq-gen")
@SequenceGenerator(name = "seq-gen", sequenceName = "HISTORY_SEQUENCE", allocationSize = 100)
@Column(name = "ID", unique=true, nullable=false, precision=16, scale=0)
private long id;
@ManyToOne(fetch=FetchType.LAZY)
@JoinColumn(name="ID_TICKET")
private TicketReclami ticketAll;
@Column(name="HISTORY", length=4000)
private String history;
@Temporal(TemporalType.TIMESTAMP)
@Column(name="TIMESTAMP", length=7)
private Date timestamp;
These are basically my entities. A converter extracts ARSYS objects, converts them to Tickets, and merges them calling:
Code:
private void merge(Ticket ticket){
EntityManager entityManager = null;
EntityTransaction tx = null;
try {
entityManager = EntityManagerFactoryUtil.GetEntityManagerFactory().createEntityManager();
tx = entityManager.getTransaction();
tx.begin();
entityManager.merge(ticket);
tx.commit();
} catch(Exception he) {
LOGGER.error("The Ticket that caused the exception is:"+ticket, he);
} finally {
if (tx != null && tx.isActive()) {
try {
tx.rollback();
} catch (Exception e1) {
LOGGER.error(this.getClass().getCanonicalName()+" could not rollback the current transaction",e1);
}
}
if(entityManager!=null && entityManager.isOpen()){
entityManager.close();
}
}
}
I have looked at possible ways to speed up the process. I tried to replace the SEQUENCE with UUIDs as I read that having the client query the sequence to get the ids would increase rountrips, but that didn't help, actually it slowed things down.
I've tried to modify my code to match this example that I found:
Code:
EntityManager em = getEntityManager();
int batchSize = 1000;
for (int i = 0; i < taloes.size(); i++) {
TalaoAIT talaoAIT = taloes.get(i);
em.persist(talaoAIT);
if(i % batchSize == 0) {
em.flush();
em.clear();
}
taloes.add(talaoAIT);
}
em.flush();
em.clear();
plus
Code:
<property name="hibernate.order_inserts" value="true"/>
<property name="hibernate.order_updates" value="true"/>
<property name="hibernate.jdbc.batch_size" value="1000"/>
but that didn't help either. Again it was possibly even slower than before...
I'm a bit out of ideas, can you guys help me speed things up?
Thank you so much,
Roberto
Login
FAQ
Search