I've written an application that will read a CSV file and take the contents and for each line, a row is inserted into the database. However I can't seem to get the application to batch insert the objects. It's taking 20 minutes to load a 1.3MB file and an hour to do 2MB file(about 50k rows). Is
I've made sure that there's a JDBC batch size set. I've tried using an Oracle sequence for the ID since I've read that GenerationType.Identity won't work. I even tried setting the ID by hand and it still won't batch insert.
I've read the Hibernate batch page, but that mentions a SessionFactory and I don't see how to retrieve that since I have an entityManager.
What can I do to get this to batch insert the objects?
Thanks.
Below is a portion of my code.
Main application code:
Code:
System.out.println("Loading data in attachment");
Reader inputStreamReader = new InputStreamReader(inputStream, "UTF-8");
CSVReader reader = new CSVReader(inputStreamReader);
ArrayList<String[]> csvFile = new ArrayList<String[]>();
String[] nextLine;
while ((nextLine = reader.readNext()) != null) {
csvFile.add(nextLine);
}
for (String[] fileLine : csvFile) {
BigDecimal evaluatee = new BigDecimal(fileLine[0]);
BigDecimal evaluator = new BigDecimal(fileLine[1]);
if (evaluatee != null && evaluator != null) { //If we don't have both an evaluator and evaluatee, skip the line.
nextRTAId = nextRTAId.add(BigDecimal.ONE);
RecordsToAssign rta = new RecordsToAssign();
rta.setId(nextRTAId);
rta.setEvaluatee(evaluatee);
rta.setEvaluator(evaluator);
rta.persist();
}
}
Object:
Code:
@Configurable
@Entity
@Table(name = "RECORDSTOASSIGN")
public class RecordsToAssign {
@PersistenceContext
transient EntityManager entityManager;
@Id
@Column(name = "ID")
private BigDecimal id;
@Column(name = "EVALUATEEDSID")
private BigDecimal evaluatee;
@Column(name = "EVALUATORDSID")
private BigDecimal evaluator;
@Transactional
public void persist() {
if (this.entityManager == null) this.entityManager = entityManager();
this.entityManager.persist(this);
}
}
Login
FAQ
Search