This requires some tweaks to make the test machinery accept an
autoinstall that doesn't do most of the things we usually expect. Also
set the .target attribute to None when reset-partition-only is true to
catch more issues in the dry-run environment.
When running subiquity in dry-run mode with SUBIQUITY_DEBUG=run-drivers,
we now support using a YAML file describing the hardware in a
umockdev-compatible way. This allows to give some control on what
ubuntu-drivers list will reply.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
The script can be invoked in different ways:
Run mypy in the current working directory and display the output:
$ scripts/run-mypy.py
Run mypy in a clean copy of the HEAD revision:
$ scripts/run-mypy.py --checkout-head
Run mypy in the current working directory and compare the result with
another revision (here main):
$ scripts/run-mypy.py --diff-against main
Run mypy in a clean copy of the HEAD revision and compare with another
revision (here main):
$ scripts/run-mypy.py --diff-against main --checkout-head
The produced result might be slightly different from what the CI does
because it also clones checks out curtin and probert (at the right
revision). This is something we might want to do in the CI as well.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
We got multiple bug reports in the past stating that the installer is
not honoring some part of the apt settings; either supplied by means of
autoinstall (e.g., pinning) or via the subiquity UI (e.g., proxy).
What happens under the hood is that curtin overwrites the APT settings
as part of the curthooks stage ; effectively discarding earlier settings
applied in the apt-config stage.
Curtin does so because we pass debconf_selections directives to
curthooks. In the past, curtin used to handle debconf_selections
separately but nowadays it considers that they are part of the APT
config. As a result, it decides to run apt-config again (but with a
close to empty configuration) as part of the curthooks stage.
We now pass debconf_selections as part of the apt-config stage. This
should hint curtin not to run apt-config again as part of curthooks.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
Oftentimes, we want to simulate a specific behavior of the application
when running in dry-run mode. To do so, we use either command line
parameters or environment variables.
This patch introduces a configuration object for dry-run executions
only. The object can be automatically loaded from a JSON file specified
via the --dry-run-config CLI argument.
Such a configuration object should help us cover way more test cases.
Going forward, I would like to use this object for things like:
* drivers - to instruct Subiquity what third-party drivers it should
suggest ; or if Subiquity should run ubuntu-drivers on the host
instead.
* ubuntu-pro - to specify the ua-contracts test environment URL - or
predefined automatic replies for the server
* to assume that /var/lib/snapd/seed/systems directory exists on the
source (or not).
* to specify the Ubuntu release that is returned by lsb_release ; can
be used to test behavior on LTS vs non LTS releases.
*
* ...
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
The second `trap ... EXIT` registration meant that the first one, the
one repsonsible for providing a clear PASS / FAIL at the end, was not
being shown.
The --cloud-config option now replaces the old --autoinstall-file
option.
The use of --autoinstall-file always made kvm-test pass the autoinstall
argument to the kernel command line. With the --cloud-config option, we
now determine if the autoinstall argument is needed by reading the cloud
config and checking if an autoinstall section is present.
The two new options --force-autoinstall and --force-no-autoinstall can
be used to override this behaviour.
The --autoinstall option is also dropped. If we want to use the default,
hardcoded cloud-config, one can use --cloud-config-default.
This can be useful to ease debugging of the VM using a config that looks
like:
#cloud-config
users:
- default
- name: root
ssh_import_id: lp:ogayot
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
We now invoke curtin install in a step-by-step mode. Before, we would
perform a single invocation of curtin install for the following stages:
* early (but no early_commands configured)
* partitioning
* extract
* curthooks
* hook
* late (but no late_commands configured)
We now run multiple invocations of curtin install, in 5 different steps:
* initial: does not run any stage per se but prepares curtin
for the next invocations (this invocation is not strictly necessary
and could be merged with the next step but having it separate makes
the flow easier to understand, I think)
* partitioning: runs the partitioning stage
* extract: runs the extract stage
* curthooks: runs the curthooks stage
The early and late stages were dropped since they would act as no-ops.
We also dropped the hook stage since it does nothing in Ubuntu.
The configuration files for each step ends up in
/var/log/installer/curtin-install/subiquity-{step_name}.conf
The log files for each step end up in
/var/log/installer/curtin-install/{step_name}.log
If errors occur, a tarball of the necessary logs end up in
/var/log/installer/curtin-install/{step_name}-error.tar
All files (i.e. configuration files, log files, error-files) are bundled
in the apport report in case of crash.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
The source autoinstall section now supports the "id" field where the
user can supply the ID of a source, e.g., "ubuntu-server" or
"ubuntu-server-minimal".
If the field is not supplied, the installation will use the source
declared default: true (if any) in the source catalog. Otherwise, it the
first source declared will be used.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
In the past, curtin log lines had the syslog identifier matching
"curtin_log.xxx". Nowadays, curtin commands are started via
systemd-run so they end up inheriting from the default
"subiquity_log.xxx" syslog identifier.
When updating the examples/curtin-*.json files, one must make sure to
filter out the entries that have the "subiquity_log.xxx" but are not
related to curtin invocations.
In the future, maybe we need to consider overriding the syslog
identifier when invoking curtin commands.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
We now use argparse on replay-curtin-log, which provides the --help
option along with named parameters for what used to be positional
parameters only.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
timeout 60 might be too short for someone with lots of network
interfaces.
Since we grab the PID and run the server in the background, the timeout
is not critically necessary.
The only way we had to pass -nic options to QEMU was to specify the
number of networks using the --nets option. Depending on its value, the
option would instruct QEMU to:
* instantiate *n* user host networks if --nets >= 1
* instantiate no network if --nets is 0
* instantiate a "dead" network if --nets < 0
To simulate more advanced networks (such as networks where HTTP proxy is
needed), we want to add the possibility to use TAP network interfaces.
This patch adds the ability to pass custom -nic options to QEMU.
Three new options are available:
* The --nic option passes the argument as-is to QEMU -nic option
* The --nic-user option passes a user host network -nic option to QEMU
- with SSH forwarding enabled. This is just like we do when using the
--nets >= 1 option)
* The --net-tap takes the name of an existing tap interface and
generates the following -nic argument:
tap,id={ifname},ifname={ifname},script=no,downscript=no,model=e1000
In the example below:
* the first network uses the tap0 interface.
* the second network uses the tap1 interface (it is syntactically
equivalent to the first one)
* the third network uses a user host network (with SSH forwarding)
$ kvm-test.py \
--nic tap,id=tap0,ifname=tap0,script=no,downscript=no,model=e1000 \
--nic-tap tap1 \
--nic-user
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
Before, we would not run answers for the source controller if only one
source was specified. The controller would automatically get marked
configured.
Since we're adding a new "search drivers" option in this screen, we need
to make it interactive in integration tests now.
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
In order we place:
* the arguments that are for client & server
* the arguments for the client only
* the arguments for the server only
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
The trap on ERR doesn't trigger in all the cases that we want.
The EXIT trap seems to be more robust, albeit with the requirement to
actually check the result code.
Move from using .subiquity as output-base to a tmpdir. This solves
several practical problems, like accidentally running the tests while
dryrun is open in another terminal or any other case where the server
process that is connected to is not the one that is expected.
Also makes cleanup nicer.
On failure, it will show which temporary directory was used to allow for
easy log examination.
Michael Hudson-Doyle