Flatten containers configuration changes tracking under a single host path

Bug #1814927 reported by Bogdan Dobrelya
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Incomplete
Medium
Unassigned

Bug Description

I know users as of Pike keep asking it on mail lists,
like which paths to use in templates for bind-mounts? And we have blog posts and docs, but none of that really helps.

We should ditch the puppet-generated directories that had span through all of the t-h-t places around, for the simplicity and code maintenance reasons. There is a sort of extremely hard to catch/debug issues, like that is fixed here https://review.openstack.org/#/c/634649/2..3/docker/services/database/mysql.yaml. So a subtle and easy to make, highly error prone, typo may end up affecting a service badly. Really, let's just flatten all config files for a service foo into its /var/lib/config-data/foo and drop that confusing /var/lib/config-data/puppet-generated/foo. If one wants to debug things, make a copy of it and issue a diff command against it and the altered location.

Related bug 1788138

Changed in tripleo:
status: New → Triaged
importance: Undecided → Medium
milestone: none → stein-3
tags: added: containers tech-debt tripleo-heat-templates
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

As jistr explained on IRC, "the reason was that in the config-data/foo there's a lot of files which we don't want copied into the containers. So we only look at files which changed in the duration of the puppet run, copy them into config-data/puppet-generated/foo and that then gets used for the containers."

But I hope we can *still* simplify that (and make the config dirs structure less confusing for users creating new templates of containerized services). For example, we could instead maintain full configuration of a service foo without the puppet-generated subset of it, under two paths named /var/lib/config-data/foo and /var/lib/config-data/foo.bak or *.1, or *.shadow. That would address the main issue: confusing paths to use for templates (https://review.openstack.org/#/c/634649/2..3/docker/services/database/mysql.yaml, right...). Assuming no one will use those .bak/.1/.shadow paths normally in templates or elsewhere, cuz that's internal needs only!

And then we could identify changed config files to be copied into containers by kolla extended start hook by simply running эdiff -nuaR /var/lib/config-data/foo /var/lib/config-data/foo.1 (or .bak/.shadow) or the like. And then calculating hash/crc for it just like we do currently by evaluating the "origin of time" mark, but using the diff outputs instead. I hope that could work as well.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

whatever works for me, if I can put a "never use puppet-generated in tht files" into the YAML/pep checker in the end

description: updated
description: updated
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

some background from irc

mandre: the only cases where we're normally using /var/lib/config-data/<service> vs /var/lib/config-data/puppet-generated/<service> is when the container isn't started by kolla_start

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

so the current logic is: if the container is started by kolla_start, it must use files from puppet_generated.

My proposition is let's make all containers using kolla_start with a custom hook that somehow reworks the static puppet_generated paths into a dynamic paths created for a one-time startup source for containers. And the content that dynamic path gets is evaluated via 'diff -nuaR foo foo.shadow' executed against the *full* paths stored at the host's /var/lib/config-data path.

Changed in tripleo:
milestone: stein-3 → train-1
Changed in tripleo:
milestone: train-1 → train-2
Changed in tripleo:
milestone: train-2 → train-3
Changed in tripleo:
milestone: train-3 → ussuri-1
Changed in tripleo:
milestone: ussuri-1 → ussuri-2
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-2 → ussuri-3
wes hayutin (weshayutin)
Changed in tripleo:
status: Triaged → Incomplete
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-3 → ussuri-rc3
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-rc3 → victoria-1
Changed in tripleo:
milestone: victoria-1 → victoria-3
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.