5.6.0 Alpha 3 integration tests to AWS Elasticsearch

clay_shooter · **Joined:** Mon Mar 28, 2016 7:14 am **Posts:** 1

Hello,

I tried to test Elasticsearch integration by running the integration tests against a (free tier) Elasticsearch running in AWS. Should the Integration Tests pass against an external instance? Most of the tests appear to pass but run very slowly. One test took 1.2M Ms.

Setup

Grab the latest Hibernate Search zip package
run "mvn" at the top and "mvn verify" in the elasticsearch project to verify the pre-aws state
Create an elasticsearch cluster in the AWS console. Enable access from my IP address. http and https are enabled by default.
Change the URL in ElasticsearchEnvironment.java to the URL in the console. (so I didn't have to understand the properties structure)
Disable elasticsearch-maven-plugin in the pom.xml by commenting it out. I left the pre-integration-test unpack pieces enabled to minimize the changes
run "mvn verify"

The AWS Elasticsearch console shows that the following indicies exist for this search domain for the first set of tests. It changes later when other tests are run.
.

actor
golfplayer
.kibana-4
org.hibernate.search.backend.elasticsearch.test.model.country
tweet
org.hibernate.search.backend.elasticsearch.test.golfcourse
org.hibernate.search.backend.elasticsearch.test.statecandidate

Results
It appears that some cleanup or delete operation failed in the service or that the Integration Tests expect some type of auto-cleanup. These tests succeed/fail the same way in each repeated run.

Code:

Running org.hibernate.search.backend.elasticsearch.test.CombiningLuceneAndElasticsearchIT
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.605 sec - in org.hibernate.search.backend.elasticsearch.test.CombiningLuceneAndElasticsearchIT
Running org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT
Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 10.353 sec <<< FAILURE! - in org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT
canDeleteByQuery(org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT)  Time elapsed: 10.341 sec  <<< FAILURE!
java.lang.AssertionError: unexpected element(s):<['Galore', 'Kidd']> in <['Brand', 'Hergesheimer', 'Galore', 'Kidd']>
        at org.fest.assertions.Fail.failure(Fail.java:228)
        at org.fest.assertions.Assert.failure(Assert.java:149)
        at org.fest.assertions.ItemGroupAssert.failureIfUnexpectedElementsFound(ItemGroupAssert.java:107)
        at org.fest.assertions.ItemGroupAssert.assertContainsOnly(ItemGroupAssert.java:79)
        at org.fest.assertions.ObjectGroupAssert.containsOnly(ObjectGroupAssert.java:65)
        at org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT.canDeleteByQuery(DeleteByQueryIT.java:104)

canDeleteByQuery(org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT)  Time elapsed: 10.345 sec  <<< ERROR!
java.lang.IllegalStateException: Previously opened Session wasn't closed!
        at org.hibernate.search.test.DefaultTestResourceManager.openSession(DefaultTestResourceManager.java:145)
        at org.hibernate.search.test.SearchTestBase.openSession(SearchTestBase.java:81)
        at org.hibernate.search.backend.elasticsearch.test.deletebyquery.DeleteByQueryIT.deleteTestData(DeleteByQueryIT.java:67)

Code:

Running org.hibernate.search.backend.elasticsearch.test.ElasticsearchAnalyzerIT
Tests run: 6, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 61.285 sec <<< FAILURE! - in org.hibernate.search.backend.elasticsearch.test.ElasticsearchAnalyzerIT
testCustomAnalyzer(org.hibernate.search.backend.elasticsearch.test.ElasticsearchAnalyzerIT)  Time elapsed: 10.157 sec  <<< ERROR!
org.hibernate.search.exception.SearchException: Could not create mapping for entity type org.hibernate.search.backend.elasticsearch.test.ElasticsearchAnalyzerIT$Tweet
        at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchIndexManager.createIndexMappings(ElasticsearchIndexManager.java:267)
        at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchIndexManager.initializeIndex(ElasticsearchIndexManager.java:155)
        at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchIndexManager.setSearchFactory(ElasticsearchIndexManager.java:143)
        at org.hibernate.search.indexes.impl.IndexManagerHolder.setActiveSearchIntegrator(IndexManagerHolder.java:188)
        at org.hibernate.search.engine.impl.MutableSearchFactoryState.setActiveSearchIntegrator(MutableSearchFactoryState.java:225)

Some tests are extremely slow. org.hibernate.search.backend.elasticsearch.test.ElasticsearchIT took long enough for me to type this message.

Code:

Running org.hibernate.search.backend.elasticsearch.test.ElasticsearchClassBridgeIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.288 sec - in org.hibernate.search.backend.elasticsearch.test.ElasticsearchClassBridgeIT
Running org.hibernate.search.backend.elasticsearch.test.ElasticsearchDSLIT
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.504 sec - in org.hibernate.search.backend.elasticsearch.test.ElasticsearchDSLIT
Running org.hibernate.search.backend.elasticsearch.test.ElasticsearchExceptionHandlingIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.167 sec - in org.hibernate.search.backend.elasticsearch.test.ElasticsearchExceptionHandlingIT
Running org.hibernate.search.backend.elasticsearch.test.ElasticsearchIndexMappingIT
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 103.32 sec - in org.hibernate.search.backend.elasticsearch.test.ElasticsearchIndexMappingIT

Gunnar · **Joined:** Sat Jan 24, 2009 12:46 pm **Posts:** 388

Hi,

Many thanks for giving the Alpha a try and even testing it with an AWS-hosted ES instance; It's very valuable to have this sort of feedback!

We haven't tried this very set-up, but my expectation is that it should work actually. Note that you need the delete-by-query plug-in installed in ES (that requirement is still missing from the docs, sorry about that; There is a JIRA issue for this) in order to perform the purgeAll() operation which is used when cleaning up test data. Also you need to have the Groovy support installed in order to run some tests around geo-spatial search. If you have these two things enabled within ES, I'd expect the test suite to pass.

That does not explain the severe slowness you experience, though. We have been focusing on the functional side of things so far and have not put very much effort into performance tuning, but that running one test takes more than a minute is not expected. If you gain any further insight e.g. by profiling, feedback from this will be appreciated very much.

We also need to test with a remote ES ourselves, just didn't find the time yet. What's the spec of the node running ES? Given it's free as you say, it might be rather limited. But I'm guessing here :)

--Gunnar