Libertine

Use a key-value database such as Berkeley BD for ContainerConfig

  1. Series devel
  2. Bug #1662655
Bug #1662655 reported by Christopher Townsend
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Libertine
Status tracked in Devel
Devel
Fix Committed
Medium
Larry Price
Libertine 1.8
Trunk
Fix Committed
Medium
Larry Price
Libertine 1.8
libertine (Ubuntu)
Fix Released
Medium
Larry Price

Bug Description

There have been instances of corrupted or disappearing ContainersConfig.json files. We should investigate whether using a key-value database such as Berkeley DB or NoSQL may be a better solution.

Related branches

lp://staging/~larryprice/libertine/better-db-locking
Christopher Townsend (community): Approve
Libertine CI Bot: Approve (continuous-integration)
Diff: 119 lines (+52/-17)
1 file modified
python/libertine/ContainersConfig.py (+52/-17)
lp://staging/~ci-train-bot/libertine/libertine-ubuntu-zesty-2576.1 (Merged)
lp://staging/~larryprice/libertine/release-1.7.1
Christopher Townsend (community): Approve
Diff: 4695 lines (+1707/-1017)
62 files modified
CMakeLists.txt (+6/-0)
debian/changelog (+37/-0)
debian/control (+9/-11)
debian/libertine-tools.install (+1/-2)
debian/libertine-xmir-tools.install (+4/-0)
debian/libertined.preinst (+1/-1)
debian/libertined.prerm (+1/-1)
debian/python3-libertine.install (+1/-2)
liblibertine/libertined.cpp (+15/-10)
pasted/pasted.cpp (+97/-43)
pasted/pasted.h (+13/-5)
python/libertine/ChrootContainer.py (+2/-2)
python/libertine/ContainerControlClient.py (+14/-9)
python/libertine/ContainersConfig.py (+52/-17)
python/libertine/Libertine.py (+21/-15)
python/libertine/LxcContainer.py (+7/-9)
python/libertine/LxdContainer.py (+17/-16)
python/libertine/service/constants.py (+27/-0)
python/libertine/service/container.py (+38/-34)
python/libertine/service/container_control.py (+29/-58)
python/libertine/service/container_control_client.py (+75/-0)
python/libertine/service/download.py (+35/-40)
python/libertine/service/operations.py (+18/-62)
python/libertine/service/operations_monitor.py (+93/-0)
python/libertine/service/task_dispatcher.py (+8/-25)
python/libertine/service/tasks/__init__.py (+2/-1)
python/libertine/service/tasks/app_info_task.py (+5/-5)
python/libertine/service/tasks/base_task.py (+39/-14)
python/libertine/service/tasks/container_info_task.py (+4/-4)
python/libertine/service/tasks/create_task.py (+14/-11)
python/libertine/service/tasks/destroy_task.py (+10/-8)
python/libertine/service/tasks/install_task.py (+10/-8)
python/libertine/service/tasks/list_app_ids_task.py (+6/-6)
python/libertine/service/tasks/list_task.py (+4/-3)
python/libertine/service/tasks/remove_task.py (+9/-8)
python/libertine/service/tasks/search_task.py (+4/-4)
python/libertine/service/tasks/update_task.py (+11/-10)
snap/plugins/x-libertine-deps.py (+2/-1)
tests/integration/CMakeLists.txt (+21/-1)
tests/integration/data/libertine/ContainersConfig.json (+19/-0)
tests/integration/test_libertine_service.py (+41/-38)
tests/integration/test_liblibertine.cpp (+145/-0)
tests/unit/CMakeLists.txt (+2/-1)
tests/unit/ContainerConfigListTests.cpp (+1/-0)
tests/unit/service/CMakeLists.txt (+2/-1)
tests/unit/service/tasks/test_app_info_task.py (+24/-26)
tests/unit/service/tasks/test_container_info_task.py (+26/-27)
tests/unit/service/tasks/test_create_task.py (+170/-176)
tests/unit/service/tasks/test_destroy_task.py (+44/-46)
tests/unit/service/tasks/test_install_task.py (+38/-41)
tests/unit/service/tasks/test_list_app_ids_task.py (+23/-25)
tests/unit/service/tasks/test_list_task.py (+15/-16)
tests/unit/service/tasks/test_remove_task.py (+38/-41)
tests/unit/service/tasks/test_search_task.py (+9/-9)
tests/unit/service/tasks/test_update_task.py (+46/-46)
tests/unit/service/test_container.py (+32/-31)
tests/unit/service/test_operations_monitor.py (+146/-0)
tests/unit/service/test_task_dispatcher.py (+6/-5)
tools/libertine-container-manager (+41/-19)
tools/libertine-launch (+12/-2)
tools/libertine-lxd-setup (+37/-14)
tools/libertined (+28/-7)
Christopher Townsend (townsend)
Changed in libertine:
status: New → Triaged
importance: Undecided → Medium
milestone: none → 1.6
Christopher Townsend (townsend)
Changed in libertine:
milestone: 1.6 → 1.7
Christopher Townsend (townsend)
Changed in libertine:
milestone: 1.7 → 1.8
Revision history for this message
Larry Price (larryprice) wrote :

I think I've traced this down to be simpler than expected... We're currently using a file lock on ContainersConfig.json after opening it, so we can get into a situation where we:

1. Open file for writing (clears file)
2. Lock file
3. Open file for reading (so it's an empty file)
4. Wait for file to unlock
5. Write finishes - unlock file
6. Read unblocked, still has an empty file

The solution seems to be to attempt to reopen the file if we don't successfully get the lock.

Revision history for this message
Libertine CI Bot (libertine-ci-bot) wrote :

Fix committed into lp:libertine at revision 453, scheduled for release in libertine, milestone Unknown

Changed in libertine:
status: Triaged → Fix Committed
Christopher Townsend (townsend)
Changed in libertine (Ubuntu):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Larry Price (larryprice)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libertine - 1.7.1+17.04.20170331-0ubuntu1

---------------
libertine (1.7.1+17.04.20170331-0ubuntu1) zesty; urgency=medium

  [ Chris Townsend ]
  * When starting pasted, ensure DISPLAY is set and valid before continuing to
    run. (LP: #1666472)
  * Fix pasted to work with rootless Xmir. (LP: #1671257)
  * Add a '-x' option to pkill so it only kills the libertined process(es)
    and nothing else. (LP: #1676005)
  * Remove hard-coded dependency on libraries.

  [ Larry Price ]
  * Bump version to 1.7.1
  * Gracefully handle creating a LibertineContainer object when the container
    backend is unavailable.
  * Update libertine xmir components to not depend on container backends.
    (LP: #1671938)
  * Catch all errors and gracefully shutdown libertined. (LP: #1671009)
  * Fix method call from ContainerControl d-bus to interfaces.
  * Update signal handlers in test_libertine_service to reflect new API.
  * Inject client for accessing ContainerControl within containers.
  * Rearchitect libertine service python backend for simpler access to running
    tasks. (LP: #1669091)
  * Ignore completions from dependencies during snapcraft build.
  * LXD needs to forward host environment to container when running arbitrary
    commands.
  * Add client object to list_app_ids_task to avoid making reentrant service
    calls.
  * Take advantage of new network subcommand during lxd init on newer
    installs.
  * Prevent installing empty package names and appropriately update status for
    unstopped containers.
  * Modifications to make test_libertine_service more stable.
  * Reopen the database file after every failure to grab the lock.
    (LP: #1662655)

 -- Larry Price <email address hidden> Fri, 31 Mar 2017 20:09:39 +0000

Changed in libertine (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.