Commit Graph

6474 Commits

Author SHA1 Message Date
Olivier Gayot a34bce470f workflows: differentiate CI and Snap workflows
The workflows defined respectively in build.yaml and snap.yaml were
both called "CI". On the Github web interface, it resulted in two menus
called "CI" with no easy way to know which is which.

To make things clearer, we now:
 * rename build.yaml -> ci.yaml
 * call "Snap" the workflow defined by snap.yaml

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-10-02 14:38:04 +02:00
Olivier Gayot fff2f6591f source: do not fire a configured event again if nothing changed
When handling a POST request to /source, Subiquity sends a 'source
configured' event. This signals other controllers / models that they
need to restart their tasks that depend on the source being used.

However, if the user of the installer goes back all the way to the
source page and submits it again without changing the settings, there
should be no reason to restart the machinery.

If a call to source ends up doing no modification to the model (i.e.,
not changing the source used or the search_drivers setting), we now
avoid emitting the 'source configured' event ; except if the model has
not been configured yet.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-10-02 14:12:54 +02:00
Olivier Gayot abef05178c shutdown: do not try to unmount /target if install was not started
If we ask for reboot before the installation has started (i.e., if
curtin install was not invoked at least once), the following call fails
and prevents the system from rebooting.

 $ umount --recursive /target

Make sure we check that /target exists and is mounted before calling
umount.

Another approach would be to check the return value of umount but the
values are not documented.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-10-02 13:29:53 +02:00
Olivier Gayot bd52c483de
Merge pull request #1812 from ogayot/install-package-skip-offline
codecs: skip installation when running an offline install
2023-10-02 09:13:58 +02:00
Olivier Gayot 01ec1da86f codecs: skip installation when running an offline install
ubuntu-restricted-addons is a multiverse package and is not included in
the pool. Therefore, trying to get it installed when offline leads to an
obvious error.

Instead of making the whole Ubuntu installation fail, we now warn and
skip installation of the package when performing an offline install.
In a perfect world, we should not have offered to install the package in
the first place, but in practice, we can run an offline installation as
the result of failed mirror testing (bad network for instance).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-29 18:20:19 +02:00
Dan Bungert 9bf0a50a24
Merge pull request #1806 from dbungert/lp-2009141-udev-events
filesystem: revamp udev handling
2023-09-27 14:12:08 -06:00
Dan Bungert b11726d398 filesystem: revamp udev handling
In LP: #2009141, we are hitting kernel limits and pyudev buffer limits.
We don't care about specific events, so much as getting one event,
waiting for things to calm down, then reprobing.

Outright disable the event monitor, and re-enable later.  If there is a
storm of events, testing has shown that stopping the listener is not
enough.
2023-09-27 13:43:14 -06:00
Olivier Gayot 9d30dc29be
Merge pull request #1809 from ogayot/oem-crash-early-confirmation
oem: make sure storage is configured before using is_core_boot_classic
2023-09-27 21:28:47 +02:00
Olivier Gayot 59849f7f45 oem: make sure storage is configured before using is_core_boot_classic
Before using fs_controller.is_core_boot_classic(), we wait for the call
to /meta/confirmation?tty=xxx. That said, in semi-automated desktop
installs, sometimes the call to /meta/confirmation happens before
marking storage configured. This leads to the following error:

  File "subiquity/server/controllers/oem.py", line 209, in apply_autoinstall_config
    await self.load_metapkgs_task
  File "subiquity/server/controllers/oem.py", line 81, in list_and_mark_configured
    await self.load_metapackages_list()
  File "subiquitycore/context.py", line 149, in decorated_async
    return await meth(self, **kw)
  File "subiquity/server/controllers/oem.py", line 136, in load_metapackages_list
    if fs_controller.is_core_boot_classic():
  File "subiquity/server/controllers/filesystem.py", line 284, in is_core_boot_classic
    return self._info.is_core_boot_classic()
AttributeError: 'NoneType' object has no attribute 'is_core_boot_classic'

Receiving the confirmation before getting the storage configured is
arguably wrong - but let's be prepared for it just in case.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-27 11:18:58 +02:00
Olivier Gayot f951146d6a
Merge pull request #1808 from ogayot/early-v2-orig-config
filesystem: don't crash if v2/orig_config is called early
2023-09-27 09:05:58 +02:00
Olivier Gayot 7de6f0538b filesystem: don't crash if v2/orig_config is called early
When v2/orig_config is called too early, the load_probe_data function
will fail because probe_data is None:

  Traceback (most recent call last):
    File "subiquity/common/api/server.py", line 164, in handler
      result = await implementation(**args)
    File "subiquity/server/controllers/filesystem.py", line 1029, in v2_orig_config_GET
      model = self.model.get_orig_model()
    File "subiquity/models/filesystem.py", line 1428, in get_orig_model
      orig_model.load_probe_data(self._probe_data)
    File "subiquity/models/filesystem.py", line 1894, in load_probe_data
      for devname, devdata in probe_data["blockdev"].items():
  TypeError: 'NoneType' object is not subscriptable

Make sure we don't dereference model._probe_data if it is None.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-26 16:00:34 +02:00
Chris Peterson 926bfeb5e1
Merge pull request #1807 from Chris-Peterson444/lp-2037133-locale-interactive
locale is not implicitly interactive with other interactive sessions
2023-09-25 17:12:22 -07:00
Chris Peterson 4ba59a503b docs: locale must be specified to be interactive 2023-09-25 15:59:07 -07:00
Olivier Gayot c9f3e252af
Merge pull request #1802 from ogayot/rich-mode-s390x
ui: have a distinct state file for rich mode over serial
2023-09-25 09:06:58 +02:00
Dan Bungert 333e4d96c8
Merge pull request #1804 from dbungert/lp-2034715-just-wait-longer
snapd api: wait longer
2023-09-22 16:09:15 -06:00
Dan Bungert 5a573f2cef snapd api: wait longer
While these changes are not supposed to take nearly this long,
per LP: #2034715 we know that they are, and that some systems will
correctly perform the finish_install() step if just given more time.
2023-09-22 15:05:01 -06:00
Olivier Gayot d7f5ef1a99
Merge pull request #1805 from ogayot/zdev-key-encryption
Bump curtin rev for recovery key on systems using zkey
2023-09-22 16:44:21 +02:00
Olivier Gayot 22b6d1258b Bump curtin rev for recovery key on systems using zkey
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-22 09:34:22 +02:00
Olivier Gayot c95261e0de ui: have a distinct state file for rich mode over serial
We recently made sure that after doing a snap refresh, the rich mode
(i.e., either rich or basic) is preserved. This was implemented by
storing the rich mode in a state file. When the client starts, it loads
the rich mode from said state file if it exists.

Unfortunately, on s390x, it causes installs to default to basic mode.
This happens because on this architecture, a subiquity install consists
of:

 * a first client (over serial) showing the SSH password
 * a second client (logging over SSH) actually going through the
   installation UI.

Since the first client uses a serial connection, the state file is
created with rich-mode set to basic. Upon connecting using SSH, the
state file is read and the rich-mode is set to basic as well.

Fixed by storing the rich-mode in two separate files, one for clients
over serial and one for other clients.

LP: #2036096

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-22 09:27:13 +02:00
Chris Peterson 4010f0ad5f
Merge pull request #1803 from Chris-Peterson444/kernel-autoinstall-file
Allow for specifying the autoinstall file over the kernel command line
2023-09-21 13:45:29 -07:00
Chris Peterson 28dd55f9dd docs: reflect autoinstall options and precedence 2023-09-21 13:23:04 -07:00
Chris Peterson 76b520afa9 autoinstall: allow for specifying autoinstall path on kernel command line 2023-09-21 11:57:46 -07:00
Dan Bungert aaaf87355c
Merge pull request #1800 from dbungert/curtin-2023-09-19
snapcraft: rev curtin for deb822, large sectors
2023-09-19 18:31:02 -06:00
Dan Bungert 21e6b1ebfc
Merge pull request #1799 from ogayot/chreipl-earlier
s390x: ensure chreipl is called before unmounting /target
2023-09-19 18:28:24 -06:00
Dan Bungert ea7b683d8e snapcraft: rev curtin for deb822, large sectors
This curtin rev adds the following:
Dan Bungert (3):
      extract: log source information
      tests/data: 4k sector disk
      storage_config: handle partitions on 4k disk

Nick Rosbrook (1):
      apt: disable default deb822 migration
2023-09-19 18:22:58 -06:00
Michael Hudson-Doyle 30cbd28152
Merge pull request #1796 from mwhudson/reusing-raid
include api-only data when sending storage objects from client to server
2023-09-20 09:27:49 +12:00
Olivier Gayot cf828eeb8d s390x: ensure chreipl is called before unmounting /target
For ZFS, we recently introduced a call to $(umount --recursive /target)
slighly before shutting down or rebooting. Unfortunately, on s390x, we
also had a very late call to chreipl to make the firmware boot from the
installed system.

The call to chreipl reads data from /target/boot, and it fails if the
filesystem is no longer mounted.

Fixed by calling chreipl earlier in the installation, during the
postinst phase rather than after the user clicks "reboot".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-19 15:31:43 +02:00
Dan Bungert a4fe424df6
Merge pull request #1797 from Chris-Peterson444/faster-unattended-upgrades
Make unattended upgrades run faster by disabling minimal steps
2023-09-15 13:30:12 -06:00
Chris Peterson cd7d3dd2cb add Unattended-Upgrade::MinimalSteps "faslse" to uu_apt conf
Co-authored-by: Dan Bungert <danielbungert@gmail.com>
2023-09-15 12:20:08 -07:00
Michael Hudson-Doyle 4d24865a63 include api-only data when sending storage objects from client to server
Making an install that used an existing RAID failed because of an
attempt to log the size of the RAID when rendering the curtin config.

This turns out to be because when the client sends the storage objects
back to the server it loses all the "api only" data including the udev
data that is needed to display the size.

In some sense this is a bit silly, we could just drop the log statement
and it would be fine but I think it's probably better to always have the
full storage objects in the server (until we can get away from this
hackish API anyway).
2023-09-15 13:28:20 +12:00
Dan Bungert 3540ad077f
Merge pull request #1791 from dbungert/lp-2034939-guided-zfs-align
storage: guided_zfs must align the swap part size
2023-09-11 11:55:19 -06:00
Dan Bungert e0cd93881b
Merge pull request #1793 from dbungert/workflow-tidy
workflows: to checkout@v4
2023-09-11 10:24:25 -06:00
Dan Bungert baa475aa15 storage: guided_zfs must align the swap part size 2023-09-11 10:13:05 -06:00
Dan Bungert 705c752320
Merge pull request #1792 from dbungert/snap-builds
Fix snap build, and CI enforce it
2023-09-11 10:12:19 -06:00
Dan Bungert a0c776008b workflows: standardize on checkout@v4 2023-09-11 10:02:28 -06:00
Dan Bungert 7ef514f79a workflows: build a snap 2023-09-11 09:53:27 -06:00
Dan Bungert d84c445612 network: fix import during snap build
Adding this import means a dependency on probert, which also means
anybody importing subiquity.common.types also has that requirement.

The make-kbd-info script imports types, and that steps was causing
snapcraft build failures due to not finding probert.
2023-09-11 09:45:28 -06:00
Dan Bungert f7c5d8c665
Merge pull request #1790 from ogayot/security-archive
Ensure the security archive is set to the proper URL
2023-09-08 15:45:21 -06:00
Olivier Gayot 75ab969d8b snapcraft: update curtin - deb822 source fix when all series disabled
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 22:59:34 +02:00
Dan Bungert afc0f3faa3
Merge pull request #1763 from s-makin/doc-move-pages
[docs] Add content and change to reST
2023-09-08 06:33:07 -06:00
Olivier Gayot 5556313652 mirror: do not let curtin decide the URL of the security archive
When the URL of the security archive is unset, curtin will set it to the
URL of the primary archive.

This is not the behavior we want for Ubuntu installations. On amd64 (and
i386), the URL of the security archive should be set to
http://security.ubuntu.com/ubuntu

On other architectures, it should be set to
http://ports.ubuntu.com/ubuntu-ports

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 10:25:02 +02:00
Olivier Gayot 79f2c4c432 mirror: for mirror-testing, disable the security suite
Mirror testing should focus on testing the primary mirror, not the
security archive - therefore we disable the -security suite.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 09:59:54 +02:00
Michael Hudson-Doyle 1b1e89b464
Merge pull request #1784 from mwhudson/disallow-core_boot-if-third-party-drivers
Disallow core boot if searching for third party drivers
2023-09-08 13:48:05 +12:00
Michael Hudson-Doyle 0a1e572fd2 update description of source.search_drivers in autoinstall.md 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 3672524761 defer deciding search_drivers default with autoinstall
in particular, defer until we know if we are installing core boot classic
2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 2306703918 disallow core boot options if searching for drivers 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle c03f0ba4f4 better server UI behaviour when all core boot options disallowed 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 8e9cf1f3c3 refactor to use new machinery a little more 2023-09-08 10:41:07 +12:00
Michael Hudson-Doyle a594b09f9f refactor to return a reason why core boot options disabled on !uefi 2023-09-08 10:39:32 +12:00
Olivier Gayot bb50ee9fe5
Merge pull request #1785 from ogayot/wifi-package-not-needed
network: fix Wi-Fi interfaces not listed in dry-run
2023-09-07 14:06:31 +02:00