Commit Graph

217 Commits

Author SHA1 Message Date
Olivier Gayot 813448683c Enable tracemalloc in integration tests with at most 3 frames
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-03-08 12:12:07 +01:00
Olivier Gayot a1df5ea409 Make integration tests fail if anything shows up in server-stderr
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-03-08 12:12:07 +01:00
Dan Bungert c3a8285300 aptdeps: move to external file
This info is redundant, and contained depends in one spot that were not
in the other.  Centralize.
2022-03-04 10:49:23 -07:00
Dan Bungert dbbf43e945 aptdeps: +cloud-init 2022-03-04 10:49:23 -07:00
J-P Nurmi cbde0a4762 Allow calling installdeps.sh from anywhere
It's nice for the desktop installer contributors to be able to run
"./path/to/subiquity/scripts/installdeps.sh" without changing dir.
2022-03-04 18:35:37 +01:00
Dan Bungert a6268011f5 test: fix installdeps 2022-02-11 16:08:34 -07:00
Dan Bungert 3803ae0e80
Merge pull request #1172 from ogayot/validate-output-autoinstall-data
Validate output autoinstall data in integration tests
2022-01-26 10:44:46 -07:00
Dan Bungert 72fbbc2868 api test: parallelize
The auto setting will choose based on the number of CPUs, including
virtual cores.  For me this is a 7x speedup.
2022-01-26 10:39:23 -07:00
Dan Bungert a6901ddc25 kvm-test: allow deadnet simulation with --nets -1 2022-01-24 17:03:00 -07:00
Olivier Gayot e2216c1d53 Run script to validate autoinstall-user-data in integration tests
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-21 13:48:43 +01:00
Olivier Gayot 535a92dad4 Add script to validate autoinstall data against JSON schema
The script can be used to validate autoinstall user data against the
schema. By default, it expects a #cloud-config header and the user-data
to be under the autoinstall: key.

By passing the --no-expect-cloudconfig, it validates the data directly.
We can use this option to validate the YAML files under
examples/autoinstall-*.yaml

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-21 13:48:43 +01:00
Olivier Gayot 81bc59019c Remove --use-fuse switch and make it the default
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-20 12:01:44 +01:00
Olivier Gayot f1119b02a6 Add --use-fuse switch to kvm-test --install so it can run as non-root
Passing --use-fuse to scripts/kvm-test.py allows to run as non-root.

It requires installation of the package fuseiso so the switch is
disabled by default.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-19 10:10:33 +01:00
Olivier Gayot 82aeeecbc6 Fix kernel command line from kvm-test.sh
Since 06ac3f92, we invoke kvm directly through subprocess.run.
Therefore, we must not add extra quotes around the -append options,
otherwise they persist and are passed in the kernel command line:

  $ cat /proc/cmdline
  "autoinstall subiquity-channel=stable" initrd=initrd

Later on, we fail to parse "autoinstall" and "subiquity-channel=stable"
as two distinct options.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-13 12:33:50 -07:00
Olivier Gayot ed0e6861aa Fix -drive option from kvm-test.sh
The function drive() used to return a string in the following format:

  "-drive file=/path/to/iso,..."

However, qemu/kvm expects "-drive" to be an argument and
"file=/path/to/iso,..." to be another argument.

The command was constructed as below since the beginning:

  kvm = [
     "kvm",
     "-cdrom", "custom.iso",          # <- OK
     "-drive file=/path/to/iso,...",  # <- NOK
  ]

Before 06ac3f92, we would join all the arguments using spaces before
executing the kvm command. Therefore we would luckily end up with a
correct command:

  " ".join(kvm)  ->  "kvm -cdrom custom.iso -drive file=/path/to/iso,..."

However, now that we supply the command to subprocess.run directly, the
problem shows up.

Fixed by returning a tuple("-drive", "file=/path/to/iso,...") from
the drive() function.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-13 12:33:47 -07:00
Dan Bungert 79b0c41b65
Merge pull request #1159 from dbungert/kvm-test-ctd
kvm-test: proxy, livefs_edit, debug
2022-01-13 12:29:58 -07:00
Dan Bungert 3a696ca459
Update scripts/kvm-test.py
Co-authored-by: Olivier Gayot <duskcoder@gmail.com>
2022-01-13 12:25:12 -07:00
Dan Bungert 98bf2943aa kvm-test: pep8 fixes 2022-01-12 14:56:58 -07:00
Dan Bungert e5e5d69a2d kvm-test: proxy, livefs_edit, debug
* add http_proxy recognition.  Picks up from DEBOOTSTRAP_PROXY.
* fix livefs_edit invocation
* include --debug in snap construction
2022-01-12 14:56:58 -07:00
Olivier Gayot 9e7bc5053b Add support for APT preferences through autoinstall in Subiquity
The autoinstall schema for "apt" now supports the "preferences"
configuration. Each preference element should contain the properties
described below, that 1:1 map with the keywords from apt_preferences(5):

  * package      <-> Package:
  * pin          <-> Pin:
  * pin-priority <-> Pin-Priority:

These preferences are forwarded to curtin through
subiquity-curtin-apt.conf. Support for these rules must be added to
curtin as well.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-11 18:42:40 +01:00
Dan Bungert 96a53eff14
Merge pull request #1154 from dbungert/autoinstall-disable-components
apt: fix disable_components w/ autoinstall
2022-01-10 09:45:49 -07:00
Dan Bungert 3d10a71d3c
Merge pull request #1153 from ogayot/ua-with-autoinstall
Add support for ubuntu-advantage with autoinstall
2022-01-07 15:27:41 -07:00
Olivier Gayot 1a079ecddd Fix extraction of default locale when running integration tests
When running integration tests, the LANG variable in
.subiquity/etc/default/locale is unquoted. The pattern that we use
expects quotes so it does not match.

Fixed by making the test work with or without quotes.

In case quotes are not present, the final part of the pipeline will fail
to find a delimiter so it will print the line unchanged

excerpt from cut(1)

   -f, --fields=LIST
          select  only these fields;  also print any line that contains
          no delimiter character, unless the -s option is specified

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-07 10:42:53 +01:00
Olivier Gayot 27ac4b43b1 Fix language pack detection when running integration tests
Looking at the logs, we can observe that the below command does
not do what it is intended to do:

  if [ -z "$( ls .subiquity/var/cache/apt/archives/) | grep $lang" ] ; then

  ++ ls .subiquity/var/cache/apt/archives/
  + '[' -z 'language-pack-en:amd64
  wamerican:amd64
  wbritish:amd64 | grep ' ']'

The "| grep $lang" part does not execute because it is outside the $()
construct. Therefore, the -z check is always false.

We can fix it by moving the "| grep $lang" part inside the subshell
construct but I took the opportunity to drop the use of the subshell.

Also added --fixed-strings and --quiet options to grep.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-06 12:05:03 +01:00
Olivier Gayot 81c67780ff Add basics checks for token validity in the autoinstall JSON schema
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-06 10:54:08 +01:00
Dan Bungert 8ad8feec31
Merge pull request #1148 from CarlosNihelton/fix-wsl-shutdown
[SystemSetup] Moving pieces of configuration from Subiquity to WSL Launcher
2022-01-05 13:31:22 -07:00
Dan Bungert 5a2a7f5ad6 apt: fix disable_components w/ autoinstall
* The mixture of storage for disable_components in and out of the config
  dict was confusing and caused bugs, depending on the flow
* disable_components now solely lives in the dict, like other items
2022-01-05 10:35:05 -07:00
Dan Bungert 68cb0d826f
Merge pull request #1152 from ogayot/fix-test-this-branch
Make sure test-this-branch.sh exits if distro-info is not installed
2022-01-05 09:58:22 -07:00
Dan Bungert 52ede053ac kvm-test: review feedback and fixes
* fix several syntax errors
* missing 'this' item was renamed to 'iso'
* change the simulated 'bash -x' output to stderr
* use shlex functions for join/split
2022-01-04 10:58:09 -07:00
Olivier Gayot d345cf999a Use autoinstall for ubuntu-advantage in integration tests
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-04 17:58:46 +01:00
Olivier Gayot 6d88618b9e Make sure test-this-branch.sh exits if distro-info is not installed
Although the script is running with -e, having two distinct invocations
of a subshell in the same instruction masks failures in the first
subshell invocation. It is similar in essence to what the pipefail
option controls.

As a consequence, the following instruction does not fail if distro-info
is not installed:

  isoname=$(distro-info -d)-live-server-$(dpkg --print-architecture).iso

And therefore, we end up with something like:

  isoname=-live-server-amd64.iso

Fixed by first assigning the value of $(distro-info -d) to a variable.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-04 17:38:43 +01:00
Dan Bungert 06ac3f92b5 kvm-test: cleanups
* better command execution
* various build arguments imply --build, so one can do just '-q' where
  one would have to previously '--build -q'
2021-12-16 13:31:13 -07:00
Carlos Nihelton bad945f5d7 Remove username from wsl.conf
- This is a cleanup for SystemSetup per DEENG-134 and DEENG-139
- Default user should be set by the WSL launcher.
- Shutdown/reboot actions also.
- Added a structured form of communication between OOBE and launcher.
- /run/launcher-command
2021-12-14 12:04:38 -03:00
Carlos Nihelton 37758047f2 Improved control over lang packs on integration
- Dryrun should never return empty packages list unless under failure.
- runtests check if the mimic-installed files match the proper language.
2021-11-22 12:16:01 -03:00
Carlos Nihelton 1e6c4472a5 Complementary to 1127 on integration side 2021-11-17 13:26:41 -03:00
Didier Roche dde0a6d53c
Merge pull request #1116 from CarlosNihelton/ft-wsl-locale-deeng-31
System locale manipulation wired into Subiquity.
2021-11-16 08:43:51 +01:00
Michael Hudson-Doyle fe9caa4715
Merge pull request #1121 from CarlosNihelton/ft-test-lxd-local
Enables preserving existing container instances.
2021-11-10 12:46:04 +13:00
Carlos Nihelton d944d48d5a
Less likely to conflict with existing containers.
By renaming the test images with a `subiquity-` prefix.
2021-11-09 18:20:41 -03:00
Carlos Nihelton 7eee081405 Updated runtests to look for locale files 2021-11-09 18:16:10 -03:00
Carlos Nihelton e88eb7334f Avoiding grep 2021-11-09 09:42:57 -03:00
Dan Bungert e31a3c31ac kvm-test: fix slimy invocation
We can't delete the snap file created by quick-test because we don't
have the path to it.  The with statement makes no sense.
2021-11-08 16:44:35 -07:00
Carlos Nihelton 8f4a6927bb Enables preserving existing container instances.
- Useful for develop env.
- Mimics the CI while avoiding downloading images all the time.
- Instance names change from 'tester' to the Ubuntu release adjective.
- lxc will only download if there is no local instance with that name.
- Useful for preserving matrix build instances.
2021-11-08 19:20:44 -03:00
Carlos Nihelton 6be4aea138 l-s-c as dep on Makefile, avoid raising exceptions
- And clear .subiquity/var/cache
2021-11-05 18:09:15 -03:00
Carlos Nihelton 5f9f2122ce System locale manipulation wired into Subiquity.
- We don't have cloud-init in WSL.
    - Subiquity must perform any actions to support the choosen locale.
    - l-s-c package added as dependency.
2021-11-05 17:29:33 -03:00
Michael Hudson-Doyle ce94726a43 kvm-test: do not boot via -kernel/-initrd unless needed 2021-11-05 16:05:22 +13:00
Michael Hudson-Doyle b6e4181fe7 kvm-test: fix a typo 2021-11-05 16:05:00 +13:00
Michael Hudson-Doyle c3727ac506 kvm-test: fix assumption around snap name quick-test-this-branch.sh creates 2021-11-05 16:04:51 +13:00
Michael Hudson-Doyle 8b7662b0a3 kvm-test: add a --snap argument to inject a specific snap 2021-11-05 16:04:18 +13:00
Michael Hudson-Doyle 277e1fe173 kvm-test: add a --basenap argument
this is like --quick but updates a passed snap rather than the one
already in the iso
2021-11-05 16:03:41 +13:00
Dan Bungert a67b9d18dc scripts: add kvm-test
Updated version of the kvm-test script shown previously.
2021-11-04 17:32:52 -06:00