-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 
Author Message
 Post subject: YABT (yet-another-blob-thread)
PostPosted: Wed Oct 25, 2006 5:27 pm 
Newbie

Joined: Fri Jun 16, 2006 11:53 am
Posts: 12
Ok, here are the big lines. I want to store files in a database. When I work with small files (ex, 5 meg), I got no problem at all. I can save/load them perfectly. However, when I attempt the same thing with a large file (ex, 50 meg, 130 meg), while I can save the file fine, and I know the file entry is good and well as I can validate that with a mysql client, I can't load it properly. I get outofmemory errors.

Now, the problem might arise from my faulty understanding of how blob objects work. I was hoping for the blob object returned by the query NOT to return the full byte data, but rather some form of stream to it, that I could then manipulate using getBinaryStream(). What I notice happens instead is the entire byte array is getting loaded into the blob object.

Now, here's the question: is there a way to manipulate streams with blob objects? Is there some more intelligent way to retrieve large files from a database? Am I wrong in some of the assumptions I make and the problem is elsewhere? Could it be something related to the mysql driver not giving me a stream?

Any help appreciated :-)


Hibernate version:
3.2 everywhere

Code between sessionFactory.openSession() and session.close():
Code:
public void raw(int iID){
      Session s = HibernateUtilities.getSessionFactory().getCurrentSession();
      Transaction t = null ;
      boolean bSuccess = false;
      
      try{
         t = s.beginTransaction();
         
         Query q = s.createQuery("select fd.ID, fd.blobData from FileData fd where fd.ID = "+iID);
         List l = q.list();
         
         t.commit();
      }
      catch(Throwable th) {
         th.printStackTrace();
         if(t != null)
            t.rollback() ;
      }
      finally {
         if(s.isOpen())
            s.close();
      }
   }


Full stack trace of any exception that occurs:
java.lang.OutOfMemoryError: Java heap space

Name and version of the database you are using:
MySQL5, with v5 drivers

The generated SQL (show_sql=true):
Hibernate: select filedata0_.ID as col_0_0_, filedata0_.blobData as col_1_0_ from FileData filedata0_ where filedata0_.ID=2

The mapping:
The mapping is the following
Code:
        private Long lID;
   private Blob bBlobData;

        @Id
   @GeneratedValue
   public Long getID() {
      return lID;
   }

   public void setID(Long lID) {
      this.lID = lID;
   }

   

   @Lob
   @Column(columnDefinition="longblob")
   public Blob getBlobData() {
      return bBlobData;
   }

   public void setBlobData(Blob bBlobData) {
      this.bBlobData = bBlobData;
   }


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 9:37 am 
Newbie

Joined: Fri Jun 16, 2006 11:53 am
Posts: 12
Well, you can go and correct me if I'm wrong, but it appears my intuition was right:

There is a limitation in how MySQL retreives data from the database. Mainly, it can't 'chunk' the data, so the driver has to load everything into memory. Hence, the streaming part of the Blob simply won't work. So, to resume, MySQL is not good to store and retreive big files in java.

Here are some links I found that discuss this issue

http://bugs.mysql.com/bug.php?id=1605 (bit old)

http://www.nabble.com/Blob-streamability-t2090339.html (more recent, but source less trustworthy)


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 2 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.