-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: Sloooow export program - how can I make it faster?
PostPosted: Mon Oct 03, 2005 2:23 pm 
Beginner
Beginner

Joined: Mon Aug 15, 2005 10:20 am
Posts: 32
Location: Brazil
I'm using Hibernate 3.0.5 to implement an "export to CSV" feature for our app. The data model is awful, with lots of composite keys. The first version we wrote tried to fetch the whole table at once, using:
Code:
Collection getAllAnswers(String examId) {
    Query q = session.createQuery("FROM answers WHERE examId = ? ORDER BY personId, questionId");
    q.setParameter(0, examId);
    return q.list();
}
. For some values of examId, it ran very fast, but for others, it raised an OutOfMemoryError.

So, we rewrote the export method to use:
Code:
static int PAGE_SIZE = 1000;
Query q;

Iterator getFirstPage(String examId) {
    q = session.createQuery("FROM answers WHERE examId = ? ORDER BY personId, questionId");
    q.setParameter(0, examId);
    q.setMaxResults(PAGE_SIZE);
    q.setFirstResult(-PAGE_SIZE);
    return getNextPage();
}

Iterator getNextPage() {
    nextResult += PAGE_SIZE;
    q.setFirstResult(nextResult);
    return q.iterate()
}

But this version runs very slowly, and eventually raises an OutOfMemoryError too. How can I make this run faster?


Table schema:
Code:
CREATE TABLE answers (
    questionId varchar(30) not null,
    value text,
    personId varchar(60) not null,
    examId varchar(30) not null,
    person_type int unsigned(2) default '0' not null)
    PRIMARY KEY (questionId, personId, examId, person_type)
);


Mapping file:
Code:
<hibernate-mapping>
    <class
        name="br.com.company.AnswerBean"
        table="resposta"
        dynamic-update="false"
        dynamic-insert="false"
    >

        <composite-id>
            <key-property name="questionId" column="questionId "/>
            <key-property name="personId" column="personId"/>
            <key-property name="personType" column="person_type"/>
            <key-property name="examId" column="examId "/>
        </composite-id>

        <property
            name="value"
            type="java.lang.String"
            update="true"
            insert="true"
            access="property"
            column="value"
        />

    </class>
</hibernate-mapping>

Code:
public class AnswerBean
    implements Serializable
{
    private String questionId;
    private String personId;
    private String value;
    private String examId;
    private String personType;
   
    public AnswerBean() {
        // no-args ctor for Hibernate
    }
   
    public AnswerBean(String id, String questionario, String personId, String value, String personType)
    {
        super();
        this.personId = personId;
        this.questionId = questionId;
        this.examId = examId;
        this.value = value;
        this.personType = personType;
    }
   
    public String getQuestionId()
    {
        return questionId;
    }
    public void setQuestionId(String questionId)
    {
        this.questionId = questionId;
    }

    public String getValue()
    {
        return value;
    }
    public void setValue(String value)
    {
        this.value = value;
    }

    public String getPersonId()
    {
        return personId;
    }
    public void setPersonId(String personId)
    {
        this.personId = personId;
       
    }

    public String getExamId()
    {
        return examId;
    }
    public void setExamId(String examId)
    {
        this.examId = examId;
    }

    public String getPersonType() {
        return personType;
    }

    public void setPersonType(String personType) {
        this.personType = personType;
    }

    public String toString() {
        return "AnswerBean (" +
            "personId=" + personId +
            ", personType=" + personType +
            ", examId=" + examId +
            ", questionId=" + questionId +
            ", value=" + value + ")";
    }

    public boolean equals(Object other) {
        if (! (other instanceof AnswerBean)) {
            return false;
        }
        AnswerBean outra = (AnswerBean) other;
        return questionId != null && questionId.equals(outra.getQuestionId())
            && personId != null && personId.equals(outra.getPersonId())
            && personType != null && personType.equals(outra.getPersonType())
            && examId != null && examId.equals(outra.getExamId());
    }

    public int hashCode() {
        if (questionId == null || personId == null || personType == null || questionId == null) {
            return 0;
        }
        return questionId.hashCode() + personId.hashCode() + personType.hashCode()
            + examId.hashCode();
    }

}


Thanks in advance,
Daniel Serodio


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 2:30 pm 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 6:10 am
Posts: 8615
Location: Neuchatel, Switzerland (Danish)
read and understand session.clear()/evict()

also read the docs about batch processeing

_________________
Max
Don't forget to rate


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 3:59 pm 
Beginner
Beginner

Joined: Thu Dec 09, 2004 7:04 pm
Posts: 26
Location: Denver, CO
max wrote:
read and understand session.clear()/evict()

also read the docs about batch processeing


The problem is that your objects are remaining in memory, attached to the session object. You need to break the link between the session and the created object. That's what max is talking about. The session clear() and event() function will break those links for you and the garbage collector will free up space.


Top
 Profile  
 
 Post subject: Problem solved
PostPosted: Mon Oct 03, 2005 4:08 pm 
Beginner
Beginner

Joined: Mon Aug 15, 2005 10:20 am
Posts: 32
Location: Brazil
max wrote:
read and understand session.clear()/evict()

also read the docs about batch processeing


Oh, that did it. Thanks for the pointer.


Top
 Profile  
 
 Post subject: Re: Problem solved
PostPosted: Thu Dec 08, 2005 6:07 am 
Newbie

Joined: Thu Sep 29, 2005 9:54 pm
Posts: 17
i met the same problem

who can tell me

how can do????


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
cron
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.