Juju Charms Collection
logstash-indexer package

Problems with Output to elasticsearch

Bug #1327231 reported by Nicolas Fernandez
24
This bug affects 3 people
Affects Status Importance Assigned to Milestone
logstash-indexer (Juju Charms Collection)
Fix Committed
High
Charles Butler

Bug Description

I've found a known problem with the Logstash version that is used in the Charm, it's actually is deployed with the version 1.2.1 and if its used with the elasticsearch Charm (cs:precise/elasticsearch-4) recommended by the Juju Team ( elasticsearch version 1.1.1) getting the following errors:

-------------------------------------
log4j, [2014-06-06T13:29:54.191] WARN: org.elasticsearch.transport.netty: [Grasshopper I&II] Message not fully read (response) for [69] handler org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$4@5a32a186, error [true], resetting
-------------------------------------

I've solved the issue changing the Output Plugin from elasticsearch to elasticsearch_http for example:

output { elasticsearch_http { host=>"xxxxxxxxxxxxxx" } }

The performance using elasticsearch_http not is the better but works.

Possible solutions, Upgrade the Logstash version to 1.4.1 or change the configuration for the Output.

Revision history for this message
Charles Butler (lazypower) wrote :

Nice work around Nicolas. Where does that configuration setting go?

Also, the upgrade from the 1.2.x series to 1.4.x looks larger than non-trivial as the packaging for Elastic Search has changed quite a bit from a flat jar to a tarball with bundled dependencies.

Have you perchance started any work on the upgrade process or are you making suit with the 1.2.x series that's bundled in the charm?

Revision history for this message
Nicolas Fernandez (nooky) wrote :

Hello Charles,

The configuration setting should be changed in /op/logstash/conf.d/output-elasticsearch.conf

In the Charm:

Line 22:
echo "output { elasticsearch { cluster=> \"${CLUSTER_NAME}\" host=>\"${ES_HOST}\" } }" > ${BASEPATH}/conf.d/output-elasticsearch.conf

Using "elasticsearch_http" plugin the option "cluster" not is used.

At this moment I just have changed the configuration in the 1.2.x to make it work properly

Charles Butler (lazypower)
Changed in logstash-indexer (Juju Charms Collection):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Charles Butler (lazypower)
Revision history for this message
Charles Butler (lazypower) wrote :

Just pushed a fix for this against /trunk of my personal branch.

After landing tests to validate the cluster deploys properly this will land as the trusty/logstash charm.

Changed in logstash-indexer (Juju Charms Collection):
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.