Juju Charms Collection
rabbitmq-server package

Massive memory usage in rabbit node with trusty-liberty cloud deployment

Bug #1656074 reported by Billy Olsen
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack RabbitMQ Server Charm
Fix Released
High
Shane Peters
OpenStack RabbitMQ Server Charm 17.02
rabbitmq-server (Juju Charms Collection)
Invalid
High
Shane Peters
Juju Charms Collection 17.01

Bug Description

RabbitMQ server on primary node in a rabbitmq-server cluster uses all available memory over time for the server. This causes client connection to become blocked due to low memory available on the rabbitmq-server.

The rabbitmq stats collection was consuming the majority of the database. This charm provides no option to tune stats collection and should.

This is documented in the OpenStack Ops Guide at http://docs.openstack.org/ops-guide/ops-maintenance-rabbitmq.html under the heading "Excessive database management memory consumption"

Tags: sts
Billy Olsen (billy-olsen)
tags: added: sts
Revision history for this message
David Ames (thedac) wrote :

"Edit the file:etc/rabbitmq/rabbitmq.config configuration file, and change the collect_statistics_interval parameter between 30000-60000 milliseconds. Alternatively you can turn off statistics collection by setting collect_statistics parameter to “none”."

Need to add configuration values for both collect_statistics_interval and collect_statistics so this is configurable.

rabbitmq.config will look something like this:
[
    {rabbit, [

        {tcp_listeners, [5672]},
        {collect_statistics_interval, 30000},

        {cluster_partition_handling, autoheal}

    ]}
].

Changed in rabbitmq-server (Juju Charms Collection):
status: New → Triaged
importance: Undecided → High
milestone: none → 17.01
Shane Peters (shaner)
Changed in rabbitmq-server (Juju Charms Collection):
assignee: nobody → Shane Peters (shaner)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-rabbitmq-server (master)

Fix proposed to branch: master
Review: https://review.openstack.org/431128

Changed in rabbitmq-server (Juju Charms Collection):
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-rabbitmq-server (master)

Reviewed: https://review.openstack.org/431128
Committed: https://git.openstack.org/cgit/openstack/charm-rabbitmq-server/commit/?id=4978fe304b5c843741be0d2dd69ff04ebc4fa611
Submitter: Jenkins
Branch: master

commit 4978fe304b5c843741be0d2dd69ff04ebc4fa611
Author: Shane Peters <email address hidden>
Date: Wed Feb 8 14:40:41 2017 -0500

    Change default collect_statistics_interval to 30s

    By default, the management plugin collects statistics every 5 seconds.
    This modifies it to 30s in order to alleviate memory pressure. This
    value is only considered when the management plugin is enabled.

    Change-Id: I7dbbcfc1cbc0a17c6a3c421914883e52cf74afb5
    Closes-Bug: 1656074

Changed in rabbitmq-server (Juju Charms Collection):
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-rabbitmq-server (stable/16.10)

Fix proposed to branch: stable/16.10
Review: https://review.openstack.org/432348

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-rabbitmq-server (stable/16.10)

Reviewed: https://review.openstack.org/432348
Committed: https://git.openstack.org/cgit/openstack/charm-rabbitmq-server/commit/?id=ddeed234ef6dee87f66bdc439d9616dd9b917d2f
Submitter: Jenkins
Branch: stable/16.10

commit ddeed234ef6dee87f66bdc439d9616dd9b917d2f
Author: Shane Peters <email address hidden>
Date: Wed Feb 8 14:40:41 2017 -0500

    Change default collect_statistics_interval to 30s

    By default, the management plugin collects statistics every 5 seconds.
    This modifies it to 30s in order to alleviate memory pressure. This
    value is only considered when the management plugin is enabled.

    Change-Id: I7dbbcfc1cbc0a17c6a3c421914883e52cf74afb5
    Closes-Bug: 1656074

James Page (james-page)
Changed in charm-rabbitmq-server:
assignee: nobody → Shane Peters (shaner)
importance: Undecided → High
status: New → Fix Committed
Changed in rabbitmq-server (Juju Charms Collection):
status: Fix Committed → Invalid
James Page (james-page)
Changed in charm-rabbitmq-server:
milestone: none → 17.02
James Page (james-page)
Changed in charm-rabbitmq-server:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.