-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 12 posts ] 
Author Message
 Post subject: Hibernate Performance
PostPosted: Fri Dec 12, 2003 3:07 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
Hi,

I am using an application which is very performance intensive.
In Hibernate when we load an Object through a session, I noticed that we read the XML file. Dowe need to do this everytime ????

Can we bypass this reading of XML file as it results in a lot of I/O operation which can considerably slow down my performance. Can I store these properties in memory instead of reading it from local disk everytime I do a load().

My tables are dynamic in nature, So i would be basically mapping my objects to different database tables of the same template at runtime. This will need me to change the XML files for an Object to point to different tables and also load the Object. So how intensive would this operation be. Is there any bench marking for the performance of hibernate????

What is the performance of Hibernate with relation to the above points. Will reading from XML files at runtime slow it down????

Thanks in Advance for any kind of help.

Amit


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 3:43 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Quote:
Will reading from XML files at runtime slow it down????


Absolutely!

By several orders of magnitude.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 4:07 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
Hibernate does parse the XML file when loading the Objects right ????
If so does it imply that for performance intensive application, Hibernate would be less suited???? Is there any way to optimize this ????


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 4:23 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Quote:
Hibernate does parse the XML file when loading the Objects right ????


Of course not.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 7:53 am 
Senior
Senior

Joined: Tue Nov 25, 2003 9:35 am
Posts: 194
Location: San Francisco
OK, the way you get Hibernate going is something like:

Code:
        //  This could be a static
   protected SessionFactory sessions;

         //  snip....

   Configuration cfg = new Configuration();

         // Load the cfg properties ...

   if (propFile != null) {
      Properties props = new Properties();
      props.load( new FileInputStream(propFile) );
      cfg.setProperties(props);
   }

        // From a list of file names, add .hbm.xml files

   if ( fileNames != null ) {
      Iterator it = fileNames.iterator();
      while ( it.hasNext() ) {
         String filename = (String) it.next();
         cfg.addFile(filename);
      }
   }
      
   sessions = cfg.buildSessionFactory();


This is where the reading of the XML files occurs. With your dynamic tables, you would have to go through this each time you add a new table (do you create them on the fly?) or want to change a mapping of a class to a different table.

Past that point, to load objects, you would be doing:

Code:
   Session aSession = null;
   try {
      aSession = sessions.openSession();
   
      // work with the session - load objects etc
         
      aSession.flush();
      aSession.connection().commit();
   } catch (Exception e) {
      e.printStackTrace();
      if (aSession != null)
         aSession.connection().rollback();
   } finally {
      if (aSession != null) {
         aSession.close();
                }   
   }


or something with Transactions.

This does not load the XML files again. You should not be creating the SessionFactory each time you want to do Hibernate work in your app.

The time it takes to create the SessionFactory varies depending on the complexity of the model you are dealing with: number of classes and relationships being mapped etc. Hibernate 2.1 is a lot faster at this than 2.0 - the change to cglib2 is the cause. When I start JBoss now under 2.1, the Hibernate initialization of my 25 classes and 8 collections is 1 sec.


Cheers,


Sherman


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 8:45 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
Thanks Sherman, I got your point.
Since the cfg file contains all the environment we do not need to read the XML file for every transaction.

Yes in my case tables need to be created on the Fly.

But there is another catch in my application.
I cannot have a single cfg file for all my tables, as there can be multiple tables of the same type but with different names. In that case, at runtime depending on a unique identifier for a table I need to map my Object to that table. For this, I will need to have a unique cfg file to connect to it.

I can keep a mapping of cfg files in hashtable against the unique table identifier. Then at runtime based on the identifer I load the corresponding cfg file and connect to the respective table.

So In this case Everytime I connect to a table I will need to do a buildConnectionFactory right since it will be a new cfg file being loaded ??????

Thanks always for your support,
Amit


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 9:50 am 
Senior
Senior

Joined: Tue Nov 25, 2003 9:35 am
Posts: 194
Location: San Francisco
In Hibernate 2.1, the Configuration class has a new method,

addDocument(org.w3c.dom.Document doc)

I have not used this myself, but with this, I would expect that you could get the base hbm.xml file for a given class, parse it to create a DOM, update the DOM with the table name you want and then pass that to the Configuration instance. This way you would only need to keep hbm.xml files for the base classes.

I don't know anything about your application, but the fact that you have multiple tables with the same structure sounds strange. If you can explain a little more about what you are trying to do (application architecture ie. Web based?) and why you are having multiple tables/one structure, maybe we can work something out that works well with Hibernate.


Cheers,


Sherman


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 11:07 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
Well this is exactly what I had in mind. I would have a base xml file which I would parse using DOM (or any other parser), then update the XML file with my specific table name and create a unique configuration file for each table and store it in a hashtable against the unique identifier.
Then when I need to connect to a table I extract the cfg file based on the identifer for that table and use it to get a session with that table.

Only thing I am worried about is whether doing a buildSessionFactory in every method call will be good in terms of performance.

I shall give you some background on my application:::

My application is a network application which stores cell phone users data on networks cards (with in built database)..Now since I can't store data for all users on 1 database, I need to do load balancing. For this I distribute my cell users into domains characterized by the first 5 digits of their cell number. So now if we have 8 database tables in each network card for a User type, then these tables need to be replicated based on the number of domains. So if I divide my subscribers into 5 domains I would be having basically 8*5 tables.

Now you see why I need to connect to my database dynamically. Since I would decide the domain based on the cell number, I would have to dynamically connect to a particular database for user info based on his cell number. The structure of the database tables would be same for all 5 domains but they would contain different data as they house different cell phone users.

So whenever I get any request, I need to find the table to connect to, then load the specific table and then work on it. So Do I have any choice other than doing a buildSessionFactory in every method call????????


Thanks so much for your help.
Amit


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 1:46 pm 
Senior
Senior

Joined: Tue Nov 25, 2003 9:35 am
Posts: 194
Location: San Francisco
Interesting. A few thoughts:

So you appear to have a large data volume problem you are trying to deal with.
  • What are the expected total sizes of the database tables - ignoring the partitioning?
  • For a customer base of 100 million (say), what's the real difference between partitioning into 5 domains with 20 million each, and just having 1 domain with everything in it? Application performance? Backup/recovery? If all the tables are in the same database, I don't see the point of the domains.
  • Are you making things too complicated for yourself in the application logic? Could you do something with the network, database or server configuration, so that you deal with the data volume problem in another way, apart from application logic?

If you are going to go down the domain splitting route, why don't you try caching the Hibernate SessionFactories for each domain? The approach would be on startup, to load a SessionFactory per domain, and then for each transaction, determine which SessionFactory to use.


Hey, I usually charge for this level of advice! Interesting problem, though. Let's see where we get to.



Sherman


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 12, 2003 9:47 pm 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Note that you should probably address this problem by using a custom persister, not by using many Configurations. Subclass EntityPersister.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Dec 13, 2003 2:34 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
Thanks for the suggestions.

To answer a few of your queries

I can't have all the users on a single table as the domains are based on phisical locations.
Take for example, we have around 10Million users on the East Coast of US, I would prefer to have all the users from that region on a network located in NY (or anywhere in the East Coast). It would not make sense for me to store all the Users in the US into a single database. It would be a great performance hit.

Another point here is that I am using an In Memory database for faster processing, So I cant increase my Users on a single card to a large extent else it will eat up all my RAM.

Also this is more of a business call. We can't do away with this.

Also as Gavin pointed out I shall explore using Custom Persistor class but any pointers towards this will always be helpful.

Regards,
Amit


Top
 Profile  
 
 Post subject:
PostPosted: Sat Dec 13, 2003 6:16 am 
Newbie

Joined: Wed Dec 10, 2003 11:29 am
Posts: 15
For now, I have implemented a SessionFactory for every domain.
Based on the unique domain ID, I would retrieve the Sessionfactory from cache and establish connection to the database table.
I would do the same for Configuration.

This seems to suit my needs but I would really like to explore the possibility of using EntityPersistor though I donot know how to go about it. I could not find enough documentation to implement it. If any pointer towards this it would be a great help.

Thanks Again,
Amit


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 12 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.