Hi, I am running a Web application in JBoss + JPA + Hibernate. I configured the application to automatically load a data file to MaraiDB by Hibernate: LOAD DATA LOCAL INFILE '/home/marlon/ubuntu/src/main/resources/Context.txt' INTO TABLE Context FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' (id,term_id,source,context,URL,publish_time); The plain text file is around 180MB, and I have been able to load it this way when the web application is started. But yesterday I suddenly received this memory error. I can load the data using the command at MariaDB terminal. The issue may be coming from Hibernate, JBoss or my application, but hope I can get some comments or insights here.
My Java heap size is set as 4GB which should be more than enough to handle this data. Please pay attention to this:
Caused by: java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:45) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:223) ------------------------------- java.lang.OutOfMemoryError: Java heap space Dumping heap to /home/marlon/ubuntu/dump/java_pid12245.hprof ... Heap dump file created [227481716 bytes in 1.047 secs] 15:42:00,971 WARN [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (ServerService Thread Pool – 31) IJ000305: Connection error occured: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@2f63260c[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@182f8ae2 connection handles=1 lastReturned=1458081711109 lastValidated=1458081711098 lastCheckedOut=1458081711252 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@32bb873d mcp=SemaphoreArrayListManagedConnectionPool@50935882[pool=BombwordDS] xaResource=LocalXAResourceImpl@34d31a8a[connectionListener=2f63260c connectionManager=504ffe68 warned=false currentXid=null productName=MySQL productVersion=10.1.12-MariaDB-1~trusty jndiName=java:/jboss/datasources/BombwordDS] txSync=null]: javax.resource.spi.ResourceAdapterInternalException: Unexpected error at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.broadcastConnectionError(BaseWrapperManagedConnection.java:688) at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.connectionError(BaseWrapperManagedConnection.java:654) at org.jboss.jca.adapters.jdbc.WrappedConnection.checkException(WrappedConnection.java:1934) at org.jboss.jca.adapters.jdbc.WrappedStatement.checkException(WrappedStatement.java:1437) at org.jboss.jca.adapters.jdbc.WrappedStatement.executeUpdate(WrappedStatement.java:418) at org.hibernate.tool.hbm2ddl.DatabaseExporter.export(DatabaseExporter.java:64) at org.hibernate.tool.hbm2ddl.SchemaExport.importScript(SchemaExport.java:447) at org.hibernate.tool.hbm2ddl.SchemaExport.execute(SchemaExport.java:379) at org.hibernate.tool.hbm2ddl.SchemaExport.create(SchemaExport.java:305) at org.hibernate.tool.hbm2ddl.SchemaExport.create(SchemaExport.java:294) at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:517) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1859) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850) at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849) at org.jboss.as.jpa.hibernate4.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:154) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:117) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:665) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:182) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at org.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:45) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:223) at org.mariadb.jdbc.internal.stream.PacketOutputStream.sendFile(PacketOutputStream.java:108) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:453) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.result(AbstractQueryProtocol.java:399) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:381) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:337) at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:280) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:474) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:490) at org.jboss.jca.adapters.jdbc.WrappedStatement.executeUpdate(WrappedStatement.java:414) at org.hibernate.tool.hbm2ddl.DatabaseExporter.export(DatabaseExporter.java:64) at org.hibernate.tool.hbm2ddl.SchemaExport.importScript(SchemaExport.java:447) at org.hibernate.tool.hbm2ddl.SchemaExport.execute(SchemaExport.java:379) at org.hibernate.tool.hbm2ddl.SchemaExport.create(SchemaExport.java:305) at org.hibernate.tool.hbm2ddl.SchemaExport.create(SchemaExport.java:294) at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:517) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1859) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850) at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849) at org.jboss.as.jpa.hibernate4.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:154) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:117) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:665) at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:182) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 15:42:00,975 ERROR [org.hibernate.tool.hbm2ddl.SchemaExport] (ServerService Thread Pool – 31) HHH000388: Unsuccessful: LOAD DATA LOCAL INFILE '/home/marlon/ubuntu/src/main/resources/Context.txt' INTO TABLE Context FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' (id,term_id,source,context,URL,publish_time) 15:42:00,975 ERROR [org.hibernate.tool.hbm2ddl.SchemaExport] (ServerService Thread Pool – 31) Error 15:42:00,976 INFO [org.hibernate.tool.hbm2ddl.SchemaExport] (ServerService Thread Pool – 31) HHH000230: Schema export complete
|