-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 
Author Message
 Post subject: Performance Problems With Insert
PostPosted: Sat Nov 18, 2006 6:40 pm 
Newbie

Joined: Mon Apr 10, 2006 11:59 am
Posts: 9
Hi,

I'm using NHibernate, in my application there is a time where I need
to perform an insert on the Database.

This insert carries two images, and some other information, the images are about 25KB each, they are stored as blobs on the Database.

I perform this operation at a rate of 3 per second, during a normal day of operation the table to where this information goes will reach something like 1.2GB.

During 4 hours we will need to maintain this rate, but it might go up to 6 of this operations per second. On the server we have other processes runing that are performing queries, and also deletes from the table. We have a lot of pressure on this table. This is the place on this application where the performance is very critical.

I saw that on average the Insert is taking something like 50ms, but sometimes it takes longer, much longer, sometimes around 100ms.
I need more control about this timming.

Any ideas about how can I improve this?
I know that on this part of my application I need to be very fast.
Thanks for any help.

Regards

Luis


Top
 Profile  
 
 Post subject:
PostPosted: Sun Nov 19, 2006 3:01 am 
Contributor
Contributor

Joined: Sat Sep 24, 2005 11:25 am
Posts: 198
50-100ms is not that much when you are talking about remote call with 50-100Kb to pass.
It is possible that this is simply the network cost.
What happen when you are using raw ADO.Net?


Top
 Profile  
 
 Post subject:
PostPosted: Mon Nov 20, 2006 10:40 am 
Senior
Senior

Joined: Sat Mar 25, 2006 9:16 am
Posts: 150
Its may be .NET garbage collection. Maybe you can call GC.Collect at certain points in your code where you can afford that extra 50ms? Otherwise I agree, this is a fixed time and would not cause scaling problems... not sure why this is a problem for you.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 22, 2006 7:35 pm 
Newbie

Joined: Mon Apr 10, 2006 11:59 am
Posts: 9
Hi.

This is a problem to me because I have some constraints about the timmings.

I have to perform an insert per each document. Each document have two images that are captured by a scanner.

This scanner works at a rate of 3.5 images per second, so I must the information to the database at this rate.
When I have the 100ms I might get in troubles.

Then I measured the time that the insert was taking, and it's not a fixed time, it's always around 50ms. 50 ms is the minimum, sometimes I get 100ms.

This is why I'm asking if this timming is too much for this information, meaning the two images.

The scanner works at a fixed rate and never slow's down, that's why the times are important to me, actually very important.

Thanks for the help.

Luis

p.s. I haven't measured yet with ADO.NET


Top
 Profile  
 
 Post subject:
PostPosted: Thu Nov 23, 2006 10:23 pm 
Senior
Senior

Joined: Sat Mar 25, 2006 9:16 am
Posts: 150
You need to re-design your solution. Even if I found something that was a consistent 50ms, I would be extremely nervous because you are so vulnerable to problems. What happens when say, Windows Update is running and your inserts start running 10x slower?

You need to architect a 2-stage processing system which uses a queueing system i.e. MSMQ or similiar, where the scanned images are accumulated in a queue and the back end pulls the items from the queue and inserts them in the database.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.