salt-ami-cloud-builder #233863 Major overhaul [deprecated]

Current State

currently, SACB uses a masterless configuration:

  • a minion is installed on the builder machine and create the vanilla image inside a chroot
  • a minion is installed in the chroot (variation) and does a highstate on the variation repo, this chroot is then uploaded as the new image

this has multiple issues:

  • we need to apply a sequence of states, which would require a runner (overstate for example). Runners are only available on a master. the current solution does not stop in case of errors
  • we use only grains in the configuration, as pillar are normally obtained from the server
  • the masterless mode should not be used for anything else than tests
  • the user data passed as a python config file are parsed and used as grains by a dedicated script
  • all additionnal grain information must be passed at boot
  • the grains are not passed in the variation -> the variation states must be autonomous and not require pillar data to work

Proposed Changes

these issues should disappear if we use the following architecture:

  • one master is available in the ami-builder virtual machine
  • a minion is installed in the same machine
  • salt is localhost in /etc/hosts
  • the creation process uses an overstate (we may switch to another more specialized runner later)
  • the user-data are directly copied as pillar data and used in both the variation and the builder
  • most pillar data can be put directly in the variation repository
  • only the variation repo address must be passed as user data if everything else is available somewhere else (in yaml format)
  • the variations and builder machines use different environments (file_roots and pillar_roots with different priorities to avoid collisions / conflicts)

the rationale:

  • this uses salt in a "normal" client-server way
  • both environment can exchange data (you can add pillar data at boot for the variation)
  • the overstate method makes it easier to access multiple states in a sequence
  • the overstate has an integrated mechanism to check the validation of step n before launching step n+1
  • salt enables to concatenate different repositories for files and pillar with different priorities depending on environments, which are necessary as the environment for the builder should not be the same as the environment for the variation

the issues which have arisen so far:

  • the salt-minion daemon must be killed before unmounting the variation
  • as configuration files for the master are modified, the master must be restarted at some point
  • higher complexity in the paths to have something functionnal and with enough parameters
appeared in<not specified>
done in<not specified>
load left0.000
closed by<not specified>