StealthOS/subiquity
StealthOS
/
subiquity
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #1956 from Chris-Peterson444/top-level-autoinstall 

Allow top-level autoinstall in all delivery methods
This commit is contained in:
Chris Peterson 2024-04-04 14:32:37 -07:00 committed by GitHub
parent d77bfbe1e0 26761b4f1a
commit 16bcfcf7ae
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 268 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/.custom_wordlist.txt
View File

@ -73,6 +73,7 @@ pw
realname realname
rootfs rootfs
rsyslog rsyslog
runtime
subvolume subvolume
subvolumes subvolumes
superset superset

2
doc/.wordlist.txt
View File

@ -8,6 +8,8 @@ EBS
EKS EKS
Grafana Grafana
IAM IAM
ISO
ISOs
JSON JSON
Jira Jira
Juju Juju

12
doc/intro-to-autoinstall.rst
View File

@ -94,18 +94,6 @@ path is relative to the rootfs of the installation system. For example:
* :code:`subiquity.autoinstallpath=path/to/autoinstall.yaml` * :code:`subiquity.autoinstallpath=path/to/autoinstall.yaml`
.. note::
Directly specifying autoinstall as a :code:`autoinstall.yaml` file does not
require a :code:`#cloud-config` header, and does not use a top level
``autoinstall:`` key. The autoinstall directives are placed at the top
level. For example:
.. code-block:: yaml
version: 1
....
Order precedence of the autoinstall locations Order precedence of the autoinstall locations
============================================= =============================================

294
doc/reference/autoinstall-reference.rst
View File

@ -3,10 +3,40 @@
Autoinstall configuration reference manual Autoinstall configuration reference manual
========================================== ==========================================
The autoinstall file uses the YAML format. At the top level, it must be a The autoinstall file uses the YAML format. At the top level is a
mapping containing the keys described in this document. Unrecognised keys single key ``autoinstall`` which contains a mapping of the keys described in
are ignored in version 1, but will cause a fatal validation error in future this document. Unrecognised keys are ignored in version 1, but will cause a
versions. fatal validation error in future versions.
Here is an example of a minimal autoinstall configuration:
.. code-block:: yaml
autoinstall:
version: 1
identity:
...
At the top level is the ``autoinstall`` keyword, which contains a version section
and an (incomplete) identity section which are explained in more detail below.
Any other key at the level of ``autoinstall``, will result in an autoinstall
validation error at runtime.
.. warning::
This behaviour was first introduced during 24.04 (Noble). On any ISOs built
before this, you will need to refresh the installer to see this behaviour.
Please the note below about the old format.
.. note::
Technically, in all but one case the top level ``autoinstall`` keyword is
strictly unnecessary. This keyword is only necessary when serving autoinstall
via cloud-config. For backwards compatibility this format is still supported
for non-cloud-config based delivery methods; however, it is
**highly recommended** to use the format with a top-level ``autoinstall``
keyword as mistakes in this formatting are a common source of confusion.
.. _ai-schema: .. _ai-schema:
@ -29,6 +59,10 @@ Several configuration keys are lists of commands to be executed. Each command ca
Top-level keys Top-level keys
-------------- --------------
The following keys can be used to configure various aspects of the installation.
If the global ``autoinstall`` key is provided, then all "top-level keys" must
be provided underneath it and "top-level" refers to this sub-level. The
examples below demonstrate this structure.
.. warning:: .. warning::
In version 1, Subiquity will emit warnings when encountering unrecognised In version 1, Subiquity will emit warnings when encountering unrecognised
@ -57,12 +91,13 @@ A list of configuration keys to still show in the user interface (UI). For examp
.. code-block:: yaml .. code-block:: yaml
version: 1 autoinstall:
interactive-sections: version: 1
- network interactive-sections:
identity: - network
username: ubuntu identity:
password: $crypted_pass username: ubuntu
password: $crypted_pass
This example stops on the network screen and allows the user to change the defaults. If a value is provided for an interactive section, it is used as the default. This example stops on the network screen and allows the user to change the defaults. If a value is provided for an interactive section, it is used as the default.
@ -221,23 +256,25 @@ For example, to run DHCP version 6 on a specific network interface:
.. code-block:: yaml .. code-block:: yaml
network: autoinstall:
version: 2
ethernets:
enp0s31f6:
dhcp6: true
Note that in the 20.04 GA release of Subiquity, the behaviour is slightly different and requires you to write this with an extra ``network:`` key:
.. code-block:: yaml
network:
network: network:
version: 2 version: 2
ethernets: ethernets:
enp0s31f6: enp0s31f6:
dhcp6: true dhcp6: true
Note that in the 20.04 GA release of Subiquity, the behaviour is slightly different and requires you to write this with an extra ``network:`` key:
.. code-block:: yaml
autoinstall:
network:
network:
version: 2
ethernets:
enp0s31f6:
dhcp6: true
Versions later than 20.04 support this syntax, too (for compatibility). When using a newer version, use the regular syntax. Versions later than 20.04 support this syntax, too (for compatibility). When using a newer version, use the regular syntax.
.. _ai-proxy: .. _ai-proxy:
@ -274,17 +311,18 @@ The default is:
.. code-block:: yaml .. code-block:: yaml
apt: autoinstall:
preserve_sources_list: false apt:
mirror-selection: preserve_sources_list: false
primary: mirror-selection:
- country-mirror primary:
- arches: [i386, amd64] - country-mirror
uri: "http://archive.ubuntu.com/ubuntu" - arches: [i386, amd64]
- arches: [s390x, arm64, armhf, powerpc, ppc64el, riscv64] uri: "http://archive.ubuntu.com/ubuntu"
uri: "http://ports.ubuntu.com/ubuntu-ports" - arches: [s390x, arm64, armhf, powerpc, ppc64el, riscv64]
fallback: abort uri: "http://ports.ubuntu.com/ubuntu-ports"
geoip: true fallback: abort
geoip: true
mirror-selection mirror-selection
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
@ -330,21 +368,23 @@ To specify a mirror, use a configuration like this:
.. code-block:: yaml .. code-block:: yaml
apt: autoinstall:
mirror-selection: apt:
primary: mirror-selection:
- uri: YOUR_MIRROR_GOES_HERE primary:
- country-mirror - uri: YOUR_MIRROR_GOES_HERE
- uri: http://archive.ubuntu.com/ubuntu - country-mirror
- uri: http://archive.ubuntu.com/ubuntu
To add a PPA: To add a PPA:
.. code-block:: yaml .. code-block:: yaml
apt: autoinstall:
sources: apt:
curtin-ppa: sources:
source: ppa:curtin-dev/test-archive curtin-ppa:
source: ppa:curtin-dev/test-archive
.. _ai-storage: .. _ai-storage:
@ -364,31 +404,33 @@ The three supported layouts at the time of writing are ``lvm``, ``direct`` and `
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: lvm layout:
storage: name: lvm
layout: storage:
name: direct layout:
storage: name: direct
layout: storage:
name: zfs layout:
name: zfs
By default, these layouts install to the largest disk in a system, but you can supply a match spec (see below) to indicate which disk to use: By default, these layouts install to the largest disk in a system, but you can supply a match spec (see below) to indicate which disk to use:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: lvm layout:
match: name: lvm
serial: CT* match:
storage: serial: CT*
layout: storage:
name: direct layout:
match: name: direct
ssd: true match:
ssd: true
.. note:: Match spec -- using ``match: {}`` matches an arbitrary disk. .. note:: Match spec -- using ``match: {}`` matches an arbitrary disk.
@ -396,10 +438,11 @@ When using the ``lvm`` layout, LUKS encryption can be enabled by supplying a pas
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: lvm layout:
password: LUKS_PASSPHRASE name: lvm
password: LUKS_PASSPHRASE
The default is to use the ``lvm`` layout. The default is to use the ``lvm`` layout.
@ -427,11 +470,12 @@ Example with no size scaling and a passphrase:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: lvm layout:
sizing-policy: all name: lvm
password: LUKS_PASSPHRASE sizing-policy: all
password: LUKS_PASSPHRASE
Reset Partition Reset Partition
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
@ -444,29 +488,32 @@ An example to enable Reset Partition:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: direct layout:
reset-partition: true name: direct
reset-partition: true
The size of the reset partition can also be fixed to a specified size. This is an example to fix Reset Partition to 12 GiB: The size of the reset partition can also be fixed to a specified size. This is an example to fix Reset Partition to 12 GiB:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: direct layout:
reset-partition: 12G name: direct
reset-partition: 12G
The installer can also install Reset Partition without installing the system. To do this, set ``reset-partition-only`` to ``true``: The installer can also install Reset Partition without installing the system. To do this, set ``reset-partition-only`` to ``true``:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
layout: storage:
name: direct layout:
reset-partition: true name: direct
reset-partition-only: true reset-partition: true
reset-partition-only: true
Action-based configuration Action-based configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -482,15 +529,16 @@ An example storage section:
.. code-block:: yaml .. code-block:: yaml
storage: autoinstall:
swap: storage:
size: 0 swap:
config: size: 0
- type: disk config:
id: disk0 - type: disk
serial: ADATA_SX8200PNP_XXXXXXXXXXX id: disk0
- type: partition serial: ADATA_SX8200PNP_XXXXXXXXXXX
... - type: partition
...
The extensions to the curtin syntax allow for disk selection and partition or logical-volume sizing. The extensions to the curtin syntax allow for disk selection and partition or logical-volume sizing.
@ -613,11 +661,12 @@ Example:
.. code-block:: yaml .. code-block:: yaml
identity: autoinstall:
realname: 'Ubuntu User' identity:
username: ubuntu realname: 'Ubuntu User'
password: '$6$wdAcoXrU039hKYPd$508Qvbe7ObUnxoj15DRCkzC3qO7edjH0VV7BPNRDYK4QR8ofJaEEF2heacn0QgD.f8pO8SNp83XNdWG6tocBM1' username: ubuntu
hostname: ubuntu password: '$6$wdAcoXrU039hKYPd$508Qvbe7ObUnxoj15DRCkzC3qO7edjH0VV7BPNRDYK4QR8ofJaEEF2heacn0QgD.f8pO8SNp83XNdWG6tocBM1'
hostname: ubuntu
.. _ai-active-directory: .. _ai-active-directory:
@ -758,10 +807,11 @@ A list of snaps to install. Each snap is represented as a mapping with a require
.. code-block:: yaml .. code-block:: yaml
snaps: autoinstall:
- name: etcd snaps:
channel: edge - name: etcd
classic: false channel: edge
classic: false
.. _ai-debconf-selections: .. _ai-debconf-selections:
@ -898,41 +948,45 @@ The default configuration is:
.. code-block:: yaml .. code-block:: yaml
reporting: autoinstall:
builtin: reporting:
type: print builtin:
type: print
Report to rsyslog: Report to rsyslog:
.. code-block:: yaml .. code-block:: yaml
reporting: autoinstall:
central: reporting:
type: rsyslog central:
destination: "@192.168.0.1" type: rsyslog
destination: "@192.168.0.1"
Suppress the default output: Suppress the default output:
.. code-block:: yaml .. code-block:: yaml
reporting: autoinstall:
builtin: reporting:
type: none builtin:
type: none
Report to a curtin-style webhook: Report to a curtin-style webhook:
.. code-block:: yaml .. code-block:: yaml
reporting: autoinstall:
hook: reporting:
type: webhook hook:
endpoint: http://example.com/endpoint/path type: webhook
consumer_key: "ck_value" endpoint: http://example.com/endpoint/path
consumer_secret: "cs_value" consumer_key: "ck_value"
token_key: "tk_value" consumer_secret: "cs_value"
token_secret: "tk_secret" token_key: "tk_value"
level: INFO token_secret: "tk_secret"
level: INFO
.. _ai-user-data: .. _ai-user-data:

8
scripts/validate-autoinstall-user-data.py
View File

@ -60,7 +60,13 @@ def main() -> None:
assert user_data.readline() == "#cloud-config\n" assert user_data.readline() == "#cloud-config\n"
def get_autoinstall_data(data): return data["autoinstall"] def get_autoinstall_data(data): return data["autoinstall"]
else: else:
def get_autoinstall_data(data): return data def get_autoinstall_data(data):
try:
cfg = data["autoinstall"]
except KeyError:
cfg = data
return cfg
# Verify autoinstall doc link is in the file # Verify autoinstall doc link is in the file

38
subiquity/server/server.py
View File

@ -668,7 +668,38 @@ class SubiquityServer(Application):
context=ctx, context=ctx,
) )
def load_autoinstall_config(self, *, only_early): @with_context(name="read_config")
def _read_config(self, *, cfg_path: str, context: Context) -> dict[str, Any]:
with open(cfg_path) as fp:
config: dict[str, Any] = yaml.safe_load(fp)
autoinstall_config: dict[str, Any]
# Support "autoinstall" as a top-level key
if "autoinstall" in config:
autoinstall_config = config.pop("autoinstall")
# but the only top level key
if len(config) != 0:
self.interactive = bool(autoinstall_config.get("interactive-sections"))
msg: str = (
"autoinstall.yaml is not a valid cloud config datasource.\n"
"No other keys may be present alongside 'autoinstall' at "
"the top level."
)
context.error(msg)
raise AutoinstallValidationError(
owner="top-level keys",
details="autoinstall.yaml is not a valid cloud config datasource",
)
else:
autoinstall_config = config
return autoinstall_config
@with_context()
def load_autoinstall_config(self, *, only_early, context):
log.debug( log.debug(
"load_autoinstall_config only_early %s file %s", "load_autoinstall_config only_early %s file %s",
only_early, only_early,
@ -689,8 +720,9 @@ class SubiquityServer(Application):
self.interactive = True self.interactive = True
return return
with open(self.autoinstall) as fp: self.autoinstall_config = self._read_config(
self.autoinstall_config = yaml.safe_load(fp) cfg_path=self.autoinstall, context=context
)
# Check every time # Check every time
self.interactive = bool(self.autoinstall_config.get("interactive-sections")) self.interactive = bool(self.autoinstall_config.get("interactive-sections"))

49
subiquity/server/tests/test_server.py
View File

@ -20,6 +20,7 @@ from typing import Any
from unittest.mock import AsyncMock, Mock, patch from unittest.mock import AsyncMock, Mock, patch
import jsonschema import jsonschema
import yaml
from jsonschema.validators import validator_for from jsonschema.validators import validator_for
from subiquity.cloudinit import CloudInitSchemaValidationError from subiquity.cloudinit import CloudInitSchemaValidationError
@ -156,6 +157,7 @@ early-commands: ["{cmd}"]
class TestAutoinstallValidation(SubiTestCase): class TestAutoinstallValidation(SubiTestCase):
async def asyncSetUp(self): async def asyncSetUp(self):
self.tempdir = self.tmp_dir()
opts = Mock() opts = Mock()
opts.dry_run = True opts.dry_run = True
opts.output_base = self.tmp_dir() opts.output_base = self.tmp_dir()
@ -171,6 +173,15 @@ class TestAutoinstallValidation(SubiTestCase):
} }
self.server.make_apport_report = Mock() self.server.make_apport_report = Mock()
def path(self, relative_path):
return self.tmp_path(relative_path, dir=self.tempdir)
def create(self, path, contents):
path = self.path(path)
with open(path, "w") as fp:
fp.write(contents)
return path
# Pseudo Load Controllers to avoid patching the loading logic for each # Pseudo Load Controllers to avoid patching the loading logic for each
# controller when we still want access to class attributes # controller when we still want access to class attributes
def pseudo_load_controllers(self): def pseudo_load_controllers(self):
@ -445,6 +456,44 @@ class TestAutoinstallValidation(SubiTestCase):
self.assertEqual(cfg, expected) self.assertEqual(cfg, expected)
async def test_autoinstall_validation__top_level_autoinstall(self):
"""Test allow autoinstall as top-level key"""
new_style = {
"autoinstall": {
"version": 1,
"interactive-sections": ["identity"],
"apt": "...",
}
}
old_style = new_style["autoinstall"]
# Read new style correctly
path = self.create("autoinstall.yaml", yaml.dump(new_style))
self.assertEqual(self.server._read_config(cfg_path=path), old_style)
# No changes to old style
path = self.create("autoinstall.yaml", yaml.dump(old_style))
self.assertEqual(self.server._read_config(cfg_path=path), old_style)
async def test_autoinstall_validation__not_cloudinit_datasource(self):
"""Test no cloud init datasources in new style autoinstall"""
new_style = {
"autoinstall": {
"version": 1,
"interactive-sections": ["identity"],
"apt": "...",
},
"cloudinit-data": "I am data",
}
with self.assertRaises(AutoinstallValidationError) as ctx:
path = self.create("autoinstall.yaml", yaml.dump(new_style))
self.server._read_config(cfg_path=path)
self.assertEqual("top-level keys", ctx.exception.owner)
class TestMetaController(SubiTestCase): class TestMetaController(SubiTestCase):
async def test_interactive_sections_not_present(self): async def test_interactive_sections_not_present(self):