Commit Graph

211 Commits

Author SHA1 Message Date
Didier Roche deab1b0859
Merge pull request #1091 from canonical/wsl_oobe_autoinstall
WSL OOBE - properly setup autoinstall
2021-10-11 09:34:59 +02:00
Patrick Wu 7ae8e108a6
system_setup: include full autoinstall system 2021-10-11 10:20:18 +08:00
Michael Hudson-Doyle 206b7ae678
Merge pull request #1092 from dbungert/source-catalog-tests
sources: add sample source catalog
2021-10-11 12:09:25 +13:00
Dan Bungert a9bf5e004b sources: add sample source catalog
* Add sample source catalog from impish iso
* Show in dryrun
* Use in one of the runtests invocations (but not really tested yet)
2021-10-08 13:02:55 -06:00
Patrick Wu 07b0407815
system_setup: refactor the testing system
1. updating testing system with the proper autoinstall schema
2. reconf mode testing included
2021-10-08 20:49:06 +08:00
Dan Bungert 3b6b89c5d9 storage/v2: create api test 2021-10-06 18:44:22 -06:00
Dan Bungert 92500bf8c9 storage/v2: start using -1 for 'rest of disk' 2021-10-06 18:44:22 -06:00
Dan Bungert 69873d8ab2 storage/v2: test cleanup 2021-10-06 18:44:22 -06:00
Dan Bungert 7f22e91cb3 storage/v2: add api test 2021-10-06 18:44:22 -06:00
Dan Bungert e14febd709
Merge pull request #1082 from dbungert/pytest-3
unittest: use pytest-3
2021-10-05 17:42:22 -06:00
Dan Bungert b757b0f972 unittest: use pytest-3
pytest-3 lets us set exclude directories, which is helpful with the
desired API testing to seperate them from 'unittest discover' so that we
can keep unit test runtime short.
2021-10-05 17:36:54 -06:00
Dan Bungert 8092b3405b slimy: python 3.8 2021-10-05 12:27:47 -06:00
Jean-Baptiste Lallement 4a2f2af268 Run WSL tests on 20.04+
The behaviour of python regarding asynchronous packages changed between
18.04 and 20.04+. OOBE is only supported on 20.04 and higher and there
is no point in making the WSL specific code compatible with ealier
releases than 20.04

Co-authored-by: Didier Roche <didrocks@ubuntu.com>
2021-09-28 09:32:39 +02:00
Jean-Baptiste Lallement 4966173d6c Test system setup autoinstall
It also adds an argument --ignore-tz to the schema comparison test to
not fail on system where the schema has no TZ like WSL.

Co-authored-by: Didier Roche <didrocks@ubuntu.com>
2021-09-28 09:32:39 +02:00
fheimes f6154d9129
Update zdev-generate.py
adjusted whitespaces
2021-09-25 20:09:54 +02:00
fheimes 913436c77a
Update zdev-generate.py
Adding "--quiet" as additional option to "lszdev" - here more for the reason of completeness and for consistency reasons.
The relevant change for solving LP#1944516 is the change in subiquity/server/controllers/zdev.py
2021-09-24 20:01:31 +02:00
Dan Bungert 525060be16 runtests: misc cleanup
* remove a stray space
* remove an extra call to check-yaml-fields.py, which can accept a list
  of directives
* remove an entire redundant autoinstall run that I added as part of
  bitlocker work but isn't actually verified in any way other than not
  timing out
2021-09-20 16:46:23 -06:00
Jean-Baptiste Lallement b0299f255f Added integration test for WSL
This tests covers the configuration of WSL on first boot.
2021-08-31 15:52:03 +02:00
Michael Hudson-Doyle d1cc1cf593 make the snaps produced by slimy-update-snap.sh pass review
I mean we /probably/ shouldn't be using the snaps this makes for
anything serious but it can be useful for testing emergency fixes...
2021-08-24 11:43:24 +12:00
Michael Hudson-Doyle a29d14a3f1 unset GIT_DIR in update-part.py
Otherwise very confusing things happen (it turns out GIT_DIR is set when
running an exec command during rebase).
2021-08-19 17:12:16 +12:00
Dan Bungert ee38a4506f deps: +build-essential 2021-08-10 13:14:08 -06:00
Dan Bungert ffc4bbde0a runtests support - first API test, has_bitlocker 2021-08-09 10:41:54 -06:00
Michael Hudson-Doyle 88384f0a98 have slimy-update-snap.sh update curtin in the snap too
Also tweak scripts/update-part.py to work better when switching
repositories.
2021-08-05 10:46:39 +12:00
Michael Hudson-Doyle 9e0f11c02e update quick-test-this-branch.sh too 2021-07-21 10:42:25 +12:00
Michael Hudson-Doyle 9a4871f416 use livefs-editor to make customized isos
a while ago I rewrote inject-subiquity-snap in python, generalized it
and put it at https://github.com/mwhudson/livefs-editor.  TBH, it's
always been better than the shell version but now there's a reason to
switch to it: the impish live server ISO use layers, which the current
shell scripts do not support and livefs-editor now does.
2021-07-19 14:44:27 +12:00
Dan Bungert c79aa602dc Proper fix for setting timezone inappropriately in dryrun
Reenable integration test for set of timezone.
Don't set it while in dryrun.
2021-07-15 10:49:32 -06:00
Michael Hudson-Doyle 545d77eeab
Merge pull request #1006 from mwhudson/lp-1936248
fix "make check" on uefi systems
2021-07-15 13:18:46 +12:00
Dan Bungert 4ed4e51a2e Workaround dryrun test actually setting timezone
Until a proper fix can be finished, remove the set of timezone from the
integration tests.
2021-07-14 19:02:42 -06:00
Michael Hudson-Doyle 18c651343d fix "make check" on uefi systems
an ESP can be on a RAID now. Also always run the integration tests in UEFI mode.
2021-07-15 08:49:10 +10:00
Dan Bungert 9f1478cd96
Add new schema comparison, robust to timezone changes (#1002)
* Add new schema comparison, robust to timezone changes

Improve the schema comparison - mostly I'm worried about the
non-timezone items, spot check the timezone list for a few that should
be there.

* Incorporate feedback - pop
2021-07-13 18:13:46 -06:00
Dan Bungert 502c1193fd
Temp disable schema check on Impish (#1000)
Impish is adding more timezones, which is cool, but breaks my schema
check, so disable this there for now and come back later with a better
answer.

Also schema check seems to need curtin.
2021-07-13 11:17:55 -06:00
Dan Bungert 40945f1823
TimeZone: autoinstall and API (FR-1184) (#986)
* TimeZone: autoinstall and API

Add support for Get/Set timezone methods.  Get means that we inquire
with GeoIP as to which timezone is suggested.  Non-availability of
GeoIP, or a previous explicit Set, means that we return the system
timezone.  Set of timezone by Post results in set of the live system
timzeone, and queuing a set of the target system by way of cloud-init.

* Add clarifying comment about _request.
2021-07-13 06:25:03 -06:00
Michael Hudson-Doyle cf931f673f update script to update exsting checkout 2021-07-05 09:23:39 +12:00
Michael Hudson-Doyle f9410e4e86 install probert and curtin via Makefile for testing
rather than pip or apt respectively
2021-07-02 13:49:40 +12:00
Michael Hudson-Doyle 50ce450f2a add a gitdeps target to Makefile that checks out probert and curtin
This uses a silly python script to get the exact version that the
snapcraft.yaml specifies.
2021-07-02 13:48:49 +12:00
Dan Bungert 90b1bb9eed
Refactor - drop external_temp_file (#985)
Move unattended-upgrades to part of postinstall, so it can be done
before the restore_apt_config umount, which lets us get rid of the
redundant external_temp_file logic.
2021-06-28 17:11:49 -06:00
Michael Hudson-Doyle 6ab29c5868 install packages with -o APT::Get::Always-Include-Phased-Updates=true
to work around bug #1925745 in apt until it is SRUed.
2021-06-17 10:37:35 +12:00
Dan Bungert 21058464fd
make_cloudconfig - start moving cloud config generation to models (#975)
Add make_cloudconfig as a function that models should offer that allows
for generation of the cloud-config snippet.
Move snaplist and locale to this mechanism.
Add a test for snaplist output.
2021-06-08 16:54:29 -06:00
Michael Hudson-Doyle e812202869 wait for cloud-init in test-in-lxd.sh 2021-06-09 10:07:32 +12:00
Dan Bungert 1d42f897a1
test-in-lxd: remove privileged container usage (#974)
* test-in-lxd: remove privileged container usage

* Don't assume certain home directory name

* Try enabling groovy
2021-06-07 14:36:37 -06:00
Dan Bungert 4f9de12556
Split unit / integration tests a bit (#956)
Unit tests should be quick and run frequently.
Integration tests can be longer and run less often.
Split them up.
2021-05-17 07:31:29 -06:00
Michael Hudson-Doyle 1da1ead19c add a integation test for non-rich mode 2021-05-04 12:44:41 +12:00
Michael Hudson-Doyle 3b60666495
Merge pull request #925 from mwhudson/lp-1885644-again
add all documented fields to filesystem model classes
2021-04-12 11:24:23 +12:00
Michael Hudson-Doyle 5dbd872919 add test of mount options being propagated 2021-04-06 09:16:55 +12:00
Dan Bungert a0e2e244bd
add setting policy for post-install updates via autoinstall (#920)
* Add element updates (non-UI)

This can be controlled with autoinstall
updates: security or all

Also an API for controlling this:
curl --silent --unix-socket .subiquity/socket a/updates ->
"security"
curl -d '"all"' --unix-socket .subiquity/socket a/updates

* Automated tests - log grep for default/none/all states

* Enforce possible values on Updates controller

Route all the various get/set thru 2 common functions.
Validate incoming data.
2021-03-31 13:57:53 -06:00
Michael Hudson-Doyle fb649bf7d5 pre-process keyboard auto detection steps into API friendly format
Continuing the theme of previous work, this branch parses pc105.tree
into API-friendly attr-based classes at snap build time (which also lets
us check some constraints then and not at run time).
2021-03-16 16:02:23 +13:00
Michael Hudson-Doyle 5d93eb824a process keyboard data into api friendly format when building snap
This is a bit sideways from the real thing I'm working on which is
moving most of the logic of keyboard handling to the server side but
anyway. This also lets me check some assumptions while processing the
data rather than in the view code.
2021-03-16 16:02:23 +13:00
Michael Hudson-Doyle b25d899f7c use rsync instead of git archive to get uncommitted changes 2021-03-10 09:05:15 +13:00
Michael Hudson-Doyle d645ca1642 add a couple of super hacky scripts for quickly testing subiquity changes
The first of these scripts mashes the current branch's code over that of
a pre-existing subiquity snap.

The second uses the first to mash the current branch's code over the
subiquity snap from an ISO and then creates a new ISO with this new
snap.

It's all a bit terrifying but being able to get your changes into a
bootable ISO in about 30s is quite a large improvement on building the
snap from scratch.
2021-03-09 17:34:59 +13:00
Dan Bungert 2c6474f3cb Lock in current schema
Accidental schema updates are unintended, so enforce that.  Schema
updates that don't break API are fine, we can handle that by manually
regenerating the schema at such time.
2021-03-03 11:31:04 -07:00
Paride Legovini 8bfa0d705c inject-subiquity-snap: log xorriso errors
In command substitution stderr is not grabbed, so it's safe to drop the
/dev/null redirect to log it.
2021-02-23 13:11:54 +01:00
Michael Hudson-Doyle 2bdddd7858 update zfs service override 2021-02-19 09:36:17 +13:00
Michael Hudson-Doyle 7ce8bdfa81 fix github action, apparently we are running on 20.04 now
this worked yesterday, I promise!
2021-02-19 09:24:20 +13:00
Michael Hudson-Doyle 4f241d830b
Merge pull request #892 from mwhudson/lp-1914587
fail if -report_el_torito fails
2021-02-19 09:14:27 +13:00
Michael Hudson-Doyle d1afc54389 switch to github actions 2021-02-18 13:01:45 +13:00
Michael Hudson-Doyle 047a903353 fail if -report_el_torito fails 2021-02-12 15:42:31 +13:00
Michael Hudson-Doyle d39c52e4c4 put a full locale (language/location/codeset) in languagelist
the location is the default location for a language in
localechooser-data.
2021-02-12 15:03:10 +13:00
Michael Hudson-Doyle 565a570328 tidy code slightly and add a comment 2021-02-10 14:28:51 +13:00
Michael Hudson-Doyle f6b70096df restore special case for kab, as iso-codes in bionic is too old 2021-02-10 14:28:04 +13:00
Dimitri John Ledkov ff4a578308
scripts: fix lookup of languages without 639-3 translations
Fixes https://github.com/CanonicalLtd/subiquity/pull/886
2021-02-04 14:53:16 +00:00
Dimitri John Ledkov bb98d57fa0
scripts: fix make-language-lists for non-languagelist languages
Fixes #885
2021-02-04 14:39:54 +00:00
Michael Hudson-Doyle ef78e7d01c re-run snap-preseed in inject-subiquity-snap.sh 2021-01-06 14:04:50 +13:00
Michael Hudson-Doyle 7381df63d5
Merge pull request #859 from mwhudson/lp-1896497
use -report_el_torito as_mkisofs option of xorriso to find command to repack ISO
2020-11-04 15:30:26 +13:00
Michael Hudson-Doyle 449ad839aa update integration tests 2020-11-04 10:57:22 +13:00
Michael Hudson-Doyle d7a070962b fix shell quoting 2020-11-04 09:30:52 +13:00
Michael Hudson-Doyle 8987ab792c use -report_el_torito as_mkisofs option of xorriso to find command to repack ISO 2020-10-28 16:32:24 +13:00
Lucas Kanashiro 8cfa8c64d4 scripts/test-this-branch.sh: install needed dependencies 2020-08-22 14:11:52 -03:00
Foster "Forst" Snowhill dca298a815
inject-subiquity-snap: recalculate MD5 for installer.squashfs
The inject-subiquity-snap.sh script rebuilds the installer.squashfs filesystem, but does not recalculate its MD5 hash.
This leads to an error message during bootup process, complaining about one of the files being damaged, that file
being installer.squashfs.

This commit introduces an extra step to recalculate the MD5 hash of the installer.squashfs and write it into
md5sum.txt, as well as to remove the GPG signature of the installer filesystem, since it's invalid anyway.
2020-08-02 18:05:34 +02:00
Michael Hudson-Doyle d9cea264f1 clean up the xorriso arguments in inject-subiquity-snap a bit 2020-07-22 15:06:29 +12:00
Michael Hudson-Doyle bbfcdd7b25 sort installer cloud config after 90_dpkg.cfg 2020-06-16 13:55:30 +12:00
Michael Hudson-Doyle 2ddde49a43 use the "None" datasource for on-first-boot configuration
Using the NoCloud source meant that a filesystem label of "cidata"
(probably containing autoinstall config) could override the cloud-init
nocloud seed subiquity wrote and then users would not get created. So
instead write cloud config directly that hardcodes using the
DataSourceNone source and config for it.

https://bugs.launchpad.net/subiquity/+bug/1879103
2020-06-15 15:07:03 +12:00
Michael Hudson-Doyle 99ad097306 snapcraft.yaml fixery 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle a6d98f29e6 change languagelist format a bit, add some special cases 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle 5cb37acefc make some language lists at snap build time 2020-05-21 11:54:49 +12:00
Michael Hudson-Doyle 3c98f68fcf use a real locale in examples/autoinstall.yaml 2020-05-21 10:49:42 +12:00
Dimitri John Ledkov 2e85a2ecd6 i18n: add scripts to help with translation updates.
Signed-off-by: Dimitri John Ledkov <xnox@ubuntu.com>
2020-05-06 13:22:13 +01:00
Michael Hudson-Doyle ba106bbbed tests 2020-05-05 23:05:59 +12:00
Michael Hudson-Doyle f814d9514f use more general context machinery to feed install events to progress view 2020-05-04 19:05:42 +12:00
Michael Hudson-Doyle 4564871ad2 fix tests 2020-05-04 15:28:42 +12:00
Michael Hudson-Doyle 8a84e470c5 make sure the size of a partition gets set to int 2020-05-04 14:16:03 +12:00
Michael Hudson-Doyle f8eef1d16a fix Subiquity.restart dry-run path now we actually hit that in runtests.sh 2020-04-29 19:46:17 +12:00
Michael Hudson-Doyle 05ca96d7ba run netplan generate --root .subiquity after each integration test 2020-04-29 19:39:10 +12:00
Michael Hudson-Doyle 670c9c838a add test for locale: in autoinstall file 2020-04-20 20:34:08 +12:00
Michael Hudson-Doyle 18e4d977e9 fix autoinstall refresh code to (a) match docs (b) work 2020-04-19 22:48:27 +12:00
Michael Hudson-Doyle e91f8bbf89 fix for omitting identity section when passing user-data 2020-04-19 21:44:09 +12:00
Michael Hudson-Doyle e3ec05a6c1 always include partition number in curtin config 2020-04-14 10:21:29 +12:00
Michael Hudson-Doyle 96a6dfd010 add integration-style test for reusing an existing partitions 2020-04-14 09:50:56 +12:00
Michael Hudson-Doyle be146972da fix travis 2020-04-07 21:37:42 +12:00
Michael Hudson-Doyle b0302484d2 fix tty check for autoinstalls 2020-04-07 20:34:38 +12:00
Michael Hudson-Doyle 467257ff9a hack for testing in lxd 2020-04-02 11:11:25 +13:00
Michael Hudson-Doyle 9a68ce91c1 fix not supplying snaps in autoinstall, oops 2020-04-02 10:47:47 +13:00
Michael Hudson-Doyle c1c33dee03 add a simple integration test for autoinstall 2020-04-02 10:22:08 +13:00
Dimitri John Ledkov 897fe56213 Try overriding zfs-mount.service 2020-03-05 16:53:35 +00:00
Dimitri John Ledkov cbaab430a3 Switch script to use python3, as python is no longer available
I guess good luck to the Jenkins CI?
2020-01-29 18:00:54 +00:00
Michael Hudson-Doyle 683d1932ac have replayed curtin runs create a log file too 2019-11-21 14:55:23 +13:00
Michael Hudson-Doyle ae033101e6 code for uploading to daisy 2019-11-21 14:54:35 +13:00
Michael Hudson-Doyle e8a9cdf4f8 have dry-run install failures replay events from a real failure 2019-11-21 11:25:43 +13:00
Michael Hudson-Doyle 0141defa01 changes suggested in review 2019-11-08 09:47:34 +13:00
Michael Hudson-Doyle 65a961be1a do not run the 100T raid tests, mdadm fails with SIGPIPE 2019-11-08 09:31:51 +13:00
Michael Hudson-Doyle 5d8c983448 make raid-size-tests.py a tiny bit friendlier 2019-11-08 09:31:51 +13:00