Commit Graph

234 Commits

Author SHA1 Message Date
Carlos Nihelton 37758047f2 Improved control over lang packs on integration
- Dryrun should never return empty packages list unless under failure.
- runtests check if the mimic-installed files match the proper language.
2021-11-22 12:16:01 -03:00
Carlos Nihelton 1e6c4472a5 Complementary to 1127 on integration side 2021-11-17 13:26:41 -03:00
Didier Roche dde0a6d53c
Merge pull request #1116 from CarlosNihelton/ft-wsl-locale-deeng-31
System locale manipulation wired into Subiquity.
2021-11-16 08:43:51 +01:00
Michael Hudson-Doyle fe9caa4715
Merge pull request #1121 from CarlosNihelton/ft-test-lxd-local
Enables preserving existing container instances.
2021-11-10 12:46:04 +13:00
Carlos Nihelton d944d48d5a
Less likely to conflict with existing containers.
By renaming the test images with a `subiquity-` prefix.
2021-11-09 18:20:41 -03:00
Carlos Nihelton 7eee081405 Updated runtests to look for locale files 2021-11-09 18:16:10 -03:00
Carlos Nihelton e88eb7334f Avoiding grep 2021-11-09 09:42:57 -03:00
Dan Bungert e31a3c31ac kvm-test: fix slimy invocation
We can't delete the snap file created by quick-test because we don't
have the path to it.  The with statement makes no sense.
2021-11-08 16:44:35 -07:00
Carlos Nihelton 8f4a6927bb Enables preserving existing container instances.
- Useful for develop env.
- Mimics the CI while avoiding downloading images all the time.
- Instance names change from 'tester' to the Ubuntu release adjective.
- lxc will only download if there is no local instance with that name.
- Useful for preserving matrix build instances.
2021-11-08 19:20:44 -03:00
Carlos Nihelton 6be4aea138 l-s-c as dep on Makefile, avoid raising exceptions
- And clear .subiquity/var/cache
2021-11-05 18:09:15 -03:00
Carlos Nihelton 5f9f2122ce System locale manipulation wired into Subiquity.
- We don't have cloud-init in WSL.
    - Subiquity must perform any actions to support the choosen locale.
    - l-s-c package added as dependency.
2021-11-05 17:29:33 -03:00
Michael Hudson-Doyle ce94726a43 kvm-test: do not boot via -kernel/-initrd unless needed 2021-11-05 16:05:22 +13:00
Michael Hudson-Doyle b6e4181fe7 kvm-test: fix a typo 2021-11-05 16:05:00 +13:00
Michael Hudson-Doyle c3727ac506 kvm-test: fix assumption around snap name quick-test-this-branch.sh creates 2021-11-05 16:04:51 +13:00
Michael Hudson-Doyle 8b7662b0a3 kvm-test: add a --snap argument to inject a specific snap 2021-11-05 16:04:18 +13:00
Michael Hudson-Doyle 277e1fe173 kvm-test: add a --basenap argument
this is like --quick but updates a passed snap rather than the one
already in the iso
2021-11-05 16:03:41 +13:00
Dan Bungert a67b9d18dc scripts: add kvm-test
Updated version of the kvm-test script shown previously.
2021-11-04 17:32:52 -06:00
Michael Hudson-Doyle 4fa0e22b58 create a helper class for invoking curtin 2021-11-03 15:28:26 +13:00
Didier Roche 8b996f9ba5
Merge pull request #1100 from canonical/wsl_oobe_adduser_tests
WSL OOBE: add `useradd` tests
2021-10-20 09:33:46 +02:00
Patrick Wu a3fce9f9c2
system_setup: add useradd tests 2021-10-20 00:27:10 +08:00
Dan Bungert 8da3df2cd2 schema: remove timezones, and tz schema check 2021-10-19 10:06:01 -06:00
Patrick Wu ec04a6833a
system_setup: shutdown/reboot tests 2021-10-18 17:15:45 +08:00
Dan Bungert 39b3394047 runtests: fail on detected crash
* exit if a crash is detected
* intentionally not cleanup on exit, to make it easier to analyze
2021-10-12 17:17:05 -06:00
Didier Roche deab1b0859
Merge pull request #1091 from canonical/wsl_oobe_autoinstall
WSL OOBE - properly setup autoinstall
2021-10-11 09:34:59 +02:00
Patrick Wu 7ae8e108a6
system_setup: include full autoinstall system 2021-10-11 10:20:18 +08:00
Michael Hudson-Doyle 206b7ae678
Merge pull request #1092 from dbungert/source-catalog-tests
sources: add sample source catalog
2021-10-11 12:09:25 +13:00
Dan Bungert a9bf5e004b sources: add sample source catalog
* Add sample source catalog from impish iso
* Show in dryrun
* Use in one of the runtests invocations (but not really tested yet)
2021-10-08 13:02:55 -06:00
Patrick Wu 07b0407815
system_setup: refactor the testing system
1. updating testing system with the proper autoinstall schema
2. reconf mode testing included
2021-10-08 20:49:06 +08:00
Dan Bungert 3b6b89c5d9 storage/v2: create api test 2021-10-06 18:44:22 -06:00
Dan Bungert 92500bf8c9 storage/v2: start using -1 for 'rest of disk' 2021-10-06 18:44:22 -06:00
Dan Bungert 69873d8ab2 storage/v2: test cleanup 2021-10-06 18:44:22 -06:00
Dan Bungert 7f22e91cb3 storage/v2: add api test 2021-10-06 18:44:22 -06:00
Dan Bungert e14febd709
Merge pull request #1082 from dbungert/pytest-3
unittest: use pytest-3
2021-10-05 17:42:22 -06:00
Dan Bungert b757b0f972 unittest: use pytest-3
pytest-3 lets us set exclude directories, which is helpful with the
desired API testing to seperate them from 'unittest discover' so that we
can keep unit test runtime short.
2021-10-05 17:36:54 -06:00
Dan Bungert 8092b3405b slimy: python 3.8 2021-10-05 12:27:47 -06:00
Jean-Baptiste Lallement 4a2f2af268 Run WSL tests on 20.04+
The behaviour of python regarding asynchronous packages changed between
18.04 and 20.04+. OOBE is only supported on 20.04 and higher and there
is no point in making the WSL specific code compatible with ealier
releases than 20.04

Co-authored-by: Didier Roche <didrocks@ubuntu.com>
2021-09-28 09:32:39 +02:00
Jean-Baptiste Lallement 4966173d6c Test system setup autoinstall
It also adds an argument --ignore-tz to the schema comparison test to
not fail on system where the schema has no TZ like WSL.

Co-authored-by: Didier Roche <didrocks@ubuntu.com>
2021-09-28 09:32:39 +02:00
fheimes f6154d9129
Update zdev-generate.py
adjusted whitespaces
2021-09-25 20:09:54 +02:00
fheimes 913436c77a
Update zdev-generate.py
Adding "--quiet" as additional option to "lszdev" - here more for the reason of completeness and for consistency reasons.
The relevant change for solving LP#1944516 is the change in subiquity/server/controllers/zdev.py
2021-09-24 20:01:31 +02:00
Dan Bungert 525060be16 runtests: misc cleanup
* remove a stray space
* remove an extra call to check-yaml-fields.py, which can accept a list
  of directives
* remove an entire redundant autoinstall run that I added as part of
  bitlocker work but isn't actually verified in any way other than not
  timing out
2021-09-20 16:46:23 -06:00
Jean-Baptiste Lallement b0299f255f Added integration test for WSL
This tests covers the configuration of WSL on first boot.
2021-08-31 15:52:03 +02:00
Michael Hudson-Doyle d1cc1cf593 make the snaps produced by slimy-update-snap.sh pass review
I mean we /probably/ shouldn't be using the snaps this makes for
anything serious but it can be useful for testing emergency fixes...
2021-08-24 11:43:24 +12:00
Michael Hudson-Doyle a29d14a3f1 unset GIT_DIR in update-part.py
Otherwise very confusing things happen (it turns out GIT_DIR is set when
running an exec command during rebase).
2021-08-19 17:12:16 +12:00
Dan Bungert ee38a4506f deps: +build-essential 2021-08-10 13:14:08 -06:00
Dan Bungert ffc4bbde0a runtests support - first API test, has_bitlocker 2021-08-09 10:41:54 -06:00
Michael Hudson-Doyle 88384f0a98 have slimy-update-snap.sh update curtin in the snap too
Also tweak scripts/update-part.py to work better when switching
repositories.
2021-08-05 10:46:39 +12:00
Michael Hudson-Doyle 9e0f11c02e update quick-test-this-branch.sh too 2021-07-21 10:42:25 +12:00
Michael Hudson-Doyle 9a4871f416 use livefs-editor to make customized isos
a while ago I rewrote inject-subiquity-snap in python, generalized it
and put it at https://github.com/mwhudson/livefs-editor.  TBH, it's
always been better than the shell version but now there's a reason to
switch to it: the impish live server ISO use layers, which the current
shell scripts do not support and livefs-editor now does.
2021-07-19 14:44:27 +12:00
Dan Bungert c79aa602dc Proper fix for setting timezone inappropriately in dryrun
Reenable integration test for set of timezone.
Don't set it while in dryrun.
2021-07-15 10:49:32 -06:00
Michael Hudson-Doyle 545d77eeab
Merge pull request #1006 from mwhudson/lp-1936248
fix "make check" on uefi systems
2021-07-15 13:18:46 +12:00
Dan Bungert 4ed4e51a2e Workaround dryrun test actually setting timezone
Until a proper fix can be finished, remove the set of timezone from the
integration tests.
2021-07-14 19:02:42 -06:00
Michael Hudson-Doyle 18c651343d fix "make check" on uefi systems
an ESP can be on a RAID now. Also always run the integration tests in UEFI mode.
2021-07-15 08:49:10 +10:00
Dan Bungert 9f1478cd96
Add new schema comparison, robust to timezone changes (#1002)
* Add new schema comparison, robust to timezone changes

Improve the schema comparison - mostly I'm worried about the
non-timezone items, spot check the timezone list for a few that should
be there.

* Incorporate feedback - pop
2021-07-13 18:13:46 -06:00
Dan Bungert 502c1193fd
Temp disable schema check on Impish (#1000)
Impish is adding more timezones, which is cool, but breaks my schema
check, so disable this there for now and come back later with a better
answer.

Also schema check seems to need curtin.
2021-07-13 11:17:55 -06:00
Dan Bungert 40945f1823
TimeZone: autoinstall and API (FR-1184) (#986)
* TimeZone: autoinstall and API

Add support for Get/Set timezone methods.  Get means that we inquire
with GeoIP as to which timezone is suggested.  Non-availability of
GeoIP, or a previous explicit Set, means that we return the system
timezone.  Set of timezone by Post results in set of the live system
timzeone, and queuing a set of the target system by way of cloud-init.

* Add clarifying comment about _request.
2021-07-13 06:25:03 -06:00
Michael Hudson-Doyle cf931f673f update script to update exsting checkout 2021-07-05 09:23:39 +12:00
Michael Hudson-Doyle f9410e4e86 install probert and curtin via Makefile for testing
rather than pip or apt respectively
2021-07-02 13:49:40 +12:00
Michael Hudson-Doyle 50ce450f2a add a gitdeps target to Makefile that checks out probert and curtin
This uses a silly python script to get the exact version that the
snapcraft.yaml specifies.
2021-07-02 13:48:49 +12:00
Dan Bungert 90b1bb9eed
Refactor - drop external_temp_file (#985)
Move unattended-upgrades to part of postinstall, so it can be done
before the restore_apt_config umount, which lets us get rid of the
redundant external_temp_file logic.
2021-06-28 17:11:49 -06:00
Michael Hudson-Doyle 6ab29c5868 install packages with -o APT::Get::Always-Include-Phased-Updates=true
to work around bug #1925745 in apt until it is SRUed.
2021-06-17 10:37:35 +12:00
Dan Bungert 21058464fd
make_cloudconfig - start moving cloud config generation to models (#975)
Add make_cloudconfig as a function that models should offer that allows
for generation of the cloud-config snippet.
Move snaplist and locale to this mechanism.
Add a test for snaplist output.
2021-06-08 16:54:29 -06:00
Michael Hudson-Doyle e812202869 wait for cloud-init in test-in-lxd.sh 2021-06-09 10:07:32 +12:00
Dan Bungert 1d42f897a1
test-in-lxd: remove privileged container usage (#974)
* test-in-lxd: remove privileged container usage

* Don't assume certain home directory name

* Try enabling groovy
2021-06-07 14:36:37 -06:00
Dan Bungert 4f9de12556
Split unit / integration tests a bit (#956)
Unit tests should be quick and run frequently.
Integration tests can be longer and run less often.
Split them up.
2021-05-17 07:31:29 -06:00
Michael Hudson-Doyle 1da1ead19c add a integation test for non-rich mode 2021-05-04 12:44:41 +12:00
Michael Hudson-Doyle 3b60666495
Merge pull request #925 from mwhudson/lp-1885644-again
add all documented fields to filesystem model classes
2021-04-12 11:24:23 +12:00
Michael Hudson-Doyle 5dbd872919 add test of mount options being propagated 2021-04-06 09:16:55 +12:00
Dan Bungert a0e2e244bd
add setting policy for post-install updates via autoinstall (#920)
* Add element updates (non-UI)

This can be controlled with autoinstall
updates: security or all

Also an API for controlling this:
curl --silent --unix-socket .subiquity/socket a/updates ->
"security"
curl -d '"all"' --unix-socket .subiquity/socket a/updates

* Automated tests - log grep for default/none/all states

* Enforce possible values on Updates controller

Route all the various get/set thru 2 common functions.
Validate incoming data.
2021-03-31 13:57:53 -06:00
Michael Hudson-Doyle fb649bf7d5 pre-process keyboard auto detection steps into API friendly format
Continuing the theme of previous work, this branch parses pc105.tree
into API-friendly attr-based classes at snap build time (which also lets
us check some constraints then and not at run time).
2021-03-16 16:02:23 +13:00
Michael Hudson-Doyle 5d93eb824a process keyboard data into api friendly format when building snap
This is a bit sideways from the real thing I'm working on which is
moving most of the logic of keyboard handling to the server side but
anyway. This also lets me check some assumptions while processing the
data rather than in the view code.
2021-03-16 16:02:23 +13:00
Michael Hudson-Doyle b25d899f7c use rsync instead of git archive to get uncommitted changes 2021-03-10 09:05:15 +13:00
Michael Hudson-Doyle d645ca1642 add a couple of super hacky scripts for quickly testing subiquity changes
The first of these scripts mashes the current branch's code over that of
a pre-existing subiquity snap.

The second uses the first to mash the current branch's code over the
subiquity snap from an ISO and then creates a new ISO with this new
snap.

It's all a bit terrifying but being able to get your changes into a
bootable ISO in about 30s is quite a large improvement on building the
snap from scratch.
2021-03-09 17:34:59 +13:00
Dan Bungert 2c6474f3cb Lock in current schema
Accidental schema updates are unintended, so enforce that.  Schema
updates that don't break API are fine, we can handle that by manually
regenerating the schema at such time.
2021-03-03 11:31:04 -07:00
Paride Legovini 8bfa0d705c inject-subiquity-snap: log xorriso errors
In command substitution stderr is not grabbed, so it's safe to drop the
/dev/null redirect to log it.
2021-02-23 13:11:54 +01:00
Michael Hudson-Doyle 2bdddd7858 update zfs service override 2021-02-19 09:36:17 +13:00
Michael Hudson-Doyle 7ce8bdfa81 fix github action, apparently we are running on 20.04 now
this worked yesterday, I promise!
2021-02-19 09:24:20 +13:00
Michael Hudson-Doyle 4f241d830b
Merge pull request #892 from mwhudson/lp-1914587
fail if -report_el_torito fails
2021-02-19 09:14:27 +13:00
Michael Hudson-Doyle d1afc54389 switch to github actions 2021-02-18 13:01:45 +13:00
Michael Hudson-Doyle 047a903353 fail if -report_el_torito fails 2021-02-12 15:42:31 +13:00
Michael Hudson-Doyle d39c52e4c4 put a full locale (language/location/codeset) in languagelist
the location is the default location for a language in
localechooser-data.
2021-02-12 15:03:10 +13:00
Michael Hudson-Doyle 565a570328 tidy code slightly and add a comment 2021-02-10 14:28:51 +13:00
Michael Hudson-Doyle f6b70096df restore special case for kab, as iso-codes in bionic is too old 2021-02-10 14:28:04 +13:00
Dimitri John Ledkov ff4a578308
scripts: fix lookup of languages without 639-3 translations
Fixes https://github.com/CanonicalLtd/subiquity/pull/886
2021-02-04 14:53:16 +00:00
Dimitri John Ledkov bb98d57fa0
scripts: fix make-language-lists for non-languagelist languages
Fixes #885
2021-02-04 14:39:54 +00:00
Michael Hudson-Doyle ef78e7d01c re-run snap-preseed in inject-subiquity-snap.sh 2021-01-06 14:04:50 +13:00
Michael Hudson-Doyle 7381df63d5
Merge pull request #859 from mwhudson/lp-1896497
use -report_el_torito as_mkisofs option of xorriso to find command to repack ISO
2020-11-04 15:30:26 +13:00
Michael Hudson-Doyle 449ad839aa update integration tests 2020-11-04 10:57:22 +13:00
Michael Hudson-Doyle d7a070962b fix shell quoting 2020-11-04 09:30:52 +13:00
Michael Hudson-Doyle 8987ab792c use -report_el_torito as_mkisofs option of xorriso to find command to repack ISO 2020-10-28 16:32:24 +13:00
Lucas Kanashiro 8cfa8c64d4 scripts/test-this-branch.sh: install needed dependencies 2020-08-22 14:11:52 -03:00
Foster "Forst" Snowhill dca298a815
inject-subiquity-snap: recalculate MD5 for installer.squashfs
The inject-subiquity-snap.sh script rebuilds the installer.squashfs filesystem, but does not recalculate its MD5 hash.
This leads to an error message during bootup process, complaining about one of the files being damaged, that file
being installer.squashfs.

This commit introduces an extra step to recalculate the MD5 hash of the installer.squashfs and write it into
md5sum.txt, as well as to remove the GPG signature of the installer filesystem, since it's invalid anyway.
2020-08-02 18:05:34 +02:00
Michael Hudson-Doyle d9cea264f1 clean up the xorriso arguments in inject-subiquity-snap a bit 2020-07-22 15:06:29 +12:00
Michael Hudson-Doyle bbfcdd7b25 sort installer cloud config after 90_dpkg.cfg 2020-06-16 13:55:30 +12:00
Michael Hudson-Doyle 2ddde49a43 use the "None" datasource for on-first-boot configuration
Using the NoCloud source meant that a filesystem label of "cidata"
(probably containing autoinstall config) could override the cloud-init
nocloud seed subiquity wrote and then users would not get created. So
instead write cloud config directly that hardcodes using the
DataSourceNone source and config for it.

https://bugs.launchpad.net/subiquity/+bug/1879103
2020-06-15 15:07:03 +12:00
Michael Hudson-Doyle 99ad097306 snapcraft.yaml fixery 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle a6d98f29e6 change languagelist format a bit, add some special cases 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle 5cb37acefc make some language lists at snap build time 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle 3c98f68fcf use a real locale in examples/autoinstall.yaml 2020-05-21 10:49:42 +12:00
Dimitri John Ledkov 2e85a2ecd6 i18n: add scripts to help with translation updates.
Signed-off-by: Dimitri John Ledkov <xnox@ubuntu.com>
2020-05-06 13:22:13 +01:00
Michael Hudson-Doyle ba106bbbed tests 2020-05-05 23:05:59 +12:00