Commit Graph

6458 Commits

Author SHA1 Message Date
Olivier Gayot d7f5ef1a99
Merge pull request #1805 from ogayot/zdev-key-encryption
Bump curtin rev for recovery key on systems using zkey
2023-09-22 16:44:21 +02:00
Olivier Gayot 22b6d1258b Bump curtin rev for recovery key on systems using zkey
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-22 09:34:22 +02:00
Olivier Gayot c95261e0de ui: have a distinct state file for rich mode over serial
We recently made sure that after doing a snap refresh, the rich mode
(i.e., either rich or basic) is preserved. This was implemented by
storing the rich mode in a state file. When the client starts, it loads
the rich mode from said state file if it exists.

Unfortunately, on s390x, it causes installs to default to basic mode.
This happens because on this architecture, a subiquity install consists
of:

 * a first client (over serial) showing the SSH password
 * a second client (logging over SSH) actually going through the
   installation UI.

Since the first client uses a serial connection, the state file is
created with rich-mode set to basic. Upon connecting using SSH, the
state file is read and the rich-mode is set to basic as well.

Fixed by storing the rich-mode in two separate files, one for clients
over serial and one for other clients.

LP: #2036096

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-22 09:27:13 +02:00
Chris Peterson 4010f0ad5f
Merge pull request #1803 from Chris-Peterson444/kernel-autoinstall-file
Allow for specifying the autoinstall file over the kernel command line
2023-09-21 13:45:29 -07:00
Chris Peterson 28dd55f9dd docs: reflect autoinstall options and precedence 2023-09-21 13:23:04 -07:00
Chris Peterson 76b520afa9 autoinstall: allow for specifying autoinstall path on kernel command line 2023-09-21 11:57:46 -07:00
Dan Bungert aaaf87355c
Merge pull request #1800 from dbungert/curtin-2023-09-19
snapcraft: rev curtin for deb822, large sectors
2023-09-19 18:31:02 -06:00
Dan Bungert 21e6b1ebfc
Merge pull request #1799 from ogayot/chreipl-earlier
s390x: ensure chreipl is called before unmounting /target
2023-09-19 18:28:24 -06:00
Dan Bungert ea7b683d8e snapcraft: rev curtin for deb822, large sectors
This curtin rev adds the following:
Dan Bungert (3):
      extract: log source information
      tests/data: 4k sector disk
      storage_config: handle partitions on 4k disk

Nick Rosbrook (1):
      apt: disable default deb822 migration
2023-09-19 18:22:58 -06:00
Michael Hudson-Doyle 30cbd28152
Merge pull request #1796 from mwhudson/reusing-raid
include api-only data when sending storage objects from client to server
2023-09-20 09:27:49 +12:00
Olivier Gayot cf828eeb8d s390x: ensure chreipl is called before unmounting /target
For ZFS, we recently introduced a call to $(umount --recursive /target)
slighly before shutting down or rebooting. Unfortunately, on s390x, we
also had a very late call to chreipl to make the firmware boot from the
installed system.

The call to chreipl reads data from /target/boot, and it fails if the
filesystem is no longer mounted.

Fixed by calling chreipl earlier in the installation, during the
postinst phase rather than after the user clicks "reboot".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-19 15:31:43 +02:00
Dan Bungert a4fe424df6
Merge pull request #1797 from Chris-Peterson444/faster-unattended-upgrades
Make unattended upgrades run faster by disabling minimal steps
2023-09-15 13:30:12 -06:00
Chris Peterson cd7d3dd2cb add Unattended-Upgrade::MinimalSteps "faslse" to uu_apt conf
Co-authored-by: Dan Bungert <danielbungert@gmail.com>
2023-09-15 12:20:08 -07:00
Michael Hudson-Doyle 4d24865a63 include api-only data when sending storage objects from client to server
Making an install that used an existing RAID failed because of an
attempt to log the size of the RAID when rendering the curtin config.

This turns out to be because when the client sends the storage objects
back to the server it loses all the "api only" data including the udev
data that is needed to display the size.

In some sense this is a bit silly, we could just drop the log statement
and it would be fine but I think it's probably better to always have the
full storage objects in the server (until we can get away from this
hackish API anyway).
2023-09-15 13:28:20 +12:00
Dan Bungert 3540ad077f
Merge pull request #1791 from dbungert/lp-2034939-guided-zfs-align
storage: guided_zfs must align the swap part size
2023-09-11 11:55:19 -06:00
Dan Bungert e0cd93881b
Merge pull request #1793 from dbungert/workflow-tidy
workflows: to checkout@v4
2023-09-11 10:24:25 -06:00
Dan Bungert baa475aa15 storage: guided_zfs must align the swap part size 2023-09-11 10:13:05 -06:00
Dan Bungert 705c752320
Merge pull request #1792 from dbungert/snap-builds
Fix snap build, and CI enforce it
2023-09-11 10:12:19 -06:00
Dan Bungert a0c776008b workflows: standardize on checkout@v4 2023-09-11 10:02:28 -06:00
Dan Bungert 7ef514f79a workflows: build a snap 2023-09-11 09:53:27 -06:00
Dan Bungert d84c445612 network: fix import during snap build
Adding this import means a dependency on probert, which also means
anybody importing subiquity.common.types also has that requirement.

The make-kbd-info script imports types, and that steps was causing
snapcraft build failures due to not finding probert.
2023-09-11 09:45:28 -06:00
Dan Bungert f7c5d8c665
Merge pull request #1790 from ogayot/security-archive
Ensure the security archive is set to the proper URL
2023-09-08 15:45:21 -06:00
Olivier Gayot 75ab969d8b snapcraft: update curtin - deb822 source fix when all series disabled
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 22:59:34 +02:00
Dan Bungert afc0f3faa3
Merge pull request #1763 from s-makin/doc-move-pages
[docs] Add content and change to reST
2023-09-08 06:33:07 -06:00
Olivier Gayot 5556313652 mirror: do not let curtin decide the URL of the security archive
When the URL of the security archive is unset, curtin will set it to the
URL of the primary archive.

This is not the behavior we want for Ubuntu installations. On amd64 (and
i386), the URL of the security archive should be set to
http://security.ubuntu.com/ubuntu

On other architectures, it should be set to
http://ports.ubuntu.com/ubuntu-ports

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 10:25:02 +02:00
Olivier Gayot 79f2c4c432 mirror: for mirror-testing, disable the security suite
Mirror testing should focus on testing the primary mirror, not the
security archive - therefore we disable the -security suite.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 09:59:54 +02:00
Michael Hudson-Doyle 1b1e89b464
Merge pull request #1784 from mwhudson/disallow-core_boot-if-third-party-drivers
Disallow core boot if searching for third party drivers
2023-09-08 13:48:05 +12:00
Michael Hudson-Doyle 0a1e572fd2 update description of source.search_drivers in autoinstall.md 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 3672524761 defer deciding search_drivers default with autoinstall
in particular, defer until we know if we are installing core boot classic
2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 2306703918 disallow core boot options if searching for drivers 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle c03f0ba4f4 better server UI behaviour when all core boot options disallowed 2023-09-08 10:41:08 +12:00
Michael Hudson-Doyle 8e9cf1f3c3 refactor to use new machinery a little more 2023-09-08 10:41:07 +12:00
Michael Hudson-Doyle a594b09f9f refactor to return a reason why core boot options disabled on !uefi 2023-09-08 10:39:32 +12:00
Olivier Gayot bb50ee9fe5
Merge pull request #1785 from ogayot/wifi-package-not-needed
network: fix Wi-Fi interfaces not listed in dry-run
2023-09-07 14:06:31 +02:00
Olivier Gayot 16688c56c4
Merge pull request #1789 from ogayot/netdev-deleted
network: fix crash when Wi-Fi or eth interface gets removed from the system
2023-09-07 13:47:30 +02:00
Olivier Gayot e10343b7e5 network: fix crash when Wi-Fi or eth interface gets removed from the system
When a network interface is disconnected from the system (e.g.,
physically removed if it's a USB adapter), probert asynchronously calls
the del_link() method.

Upon receiving this notification, Subiquity server wants to send an
update to the Subiquity clients. The update contains information about
the interface that disappeared - which is obtained through a call to
netdev_info.

Unfortunately, for Wi-Fi and Ethernet interfaces, netdev_info
dereferences the NetworkDev.info variable. Interfaces that no longer
exist on the system (and also interfaces that do not yet exist), have
their "info" variable set to None - so an exception is raised when
dereferencing it.

Wi-Fi interface:

    File "subiquitycore/models/network.py", line 227, in netdev_info
      scan_state=self.info.wlan['scan_state'],
  AttributeError: 'NoneType' object has no attribute 'wlan'

Ethernet interface:

    File "subiquitycore/models/network.py", line 201, in netdev_info
      is_connected = bool(self.info.is_connected)
  AttributeError: 'NoneType' object has no attribute 'is_connected'

Fixed by making sure netdev_info does not raise if the dev.info variable
is None. This is a valid use-case.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-07 09:56:55 +02:00
Olivier Gayot 5cc6f5df4a
Merge pull request #1786 from ogayot/virt-netdev-help-crash
network: fix crash upon accessing Help menu after creating a bond
2023-09-07 09:01:35 +02:00
Dan Bungert b175c3d1da
Merge pull request #1788 from dbungert/po-refresh-2023-09-06
translations: refresh po files
2023-09-06 20:59:46 -06:00
Dan Bungert 2de6b725f1 translations: refresh po files 2023-09-06 15:15:59 -06:00
Olivier Gayot 221466aa70 network: document and add type hint for NetworkDev.info
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-06 14:59:51 +02:00
Olivier Gayot dc26a1d252 network: fix crash upon accessing Help after creating a bond
When accessing the Help menu, Subiquity looks up the IP addresses
currently configured - so it knows whether to show the "Help on SSH
access" option.

Unfortunately, it also looks for IP addresses on devices that were
"configured" through the network screen but that still do not exist in
the system. When such a device exist (e.g., a bond), the Subiquity
client crashes with the following exception:

 Traceback (most recent call last):
   File "subiquity/common/api/server.py", line 164, in handler
     result = await implementation(**args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "subiquity/server/server.py", line 117, in ssh_info_GET
     ips.extend(map(str, dev.actual_global_ip_addresses))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "subiquitycore/models/network.py", line 394, in actual_global_ip_addresses
     for _, addr in sorted(self.info.addresses.items())
                           ^^^^^^^^^^^^^^^^^^^
 AttributeError: 'NoneType' object has no attribute 'addresses'

A similar crash is observed when calling /network/global_addresses after
creating the bond.

Fixed by only checking the IP addresses of devices that have a
probert.network.Link instance (i.e., they exist in the system).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-06 14:58:51 +02:00
Dan Bungert 4d6fb69b73
Merge pull request #1787 from dbungert/lp-2034270-guided-small-disk-crash
storage: fix crash when guided and a small disk
2023-09-05 15:41:45 -06:00
Dan Bungert 20848f44b7 storage: fix crash when guided and a small disk
In LP: #2034270, a 1MiB disk is present.  This is triggering a crash
while attempting to decide if we can do a guided install.
2023-09-05 12:11:24 -06:00
Olivier Gayot 1670d0711f pkghelper: have alternative dry-run implementation
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-05 13:23:43 +02:00
Olivier Gayot c09a32c5bb network: fix Wi-Fi interfaces not listed in dry-run
When a Wi-Fi interface is present in the machine configuration (e.g.,
mwhudson.json), the GUI seemingly ignores it. This happens because there
is a filter on the server side which only returns Wi-Fi interfaces if
the wlan_support_install_state() function returns
PackageInstallState.DONE.

However, calling the /network endpoint shows that the state is set to
the wrong value:

 {"wlan_support_install_state": "NOT_NEEDED"}

This turns out to be inconsistent because:
 * we lean on a PackageInstaller instance to tell if wpasupplicant is
installed (this is what the wlan_support_install_state() function
reflects) ; but
 * in dry-run mode, we pretend to install wpasupplicant without
actually relying on the PackageInstaller instance.

Fixed by using the PackageInstaller instance to install the
wpasupplicant package - with a special implementation that only pretends
to install it. This is enough to make the PackageInstaller instance
think the package is installed.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-04 13:45:49 +02:00
Sally Makin 0c20d8b917 Applying suggestions 2023-09-04 11:34:19 +01:00
Olivier Gayot 8e03050dbb
Merge pull request #1782 from ogayot/fix-x-error-msg-cr-lf
api: encode x-error-msg as JSON - so it does not contain <CR> or <LF>
2023-09-04 09:35:43 +02:00
Olivier Gayot b866bd2a56 api: encode x-error-msg as JSON - so it does not contain <CR> or <LF>
When the server raises an exception in a HTTP request handler context,
more often than not, the exception is sent back to the client in the
body.

Additionally, the message of the exception (if any), is also copied as
is in a x-error-msg HTTP header.

That said, HTTP headers must obey strict rules. The "\r\n" sequence
indicate the end of the current HTTP header. When using aiohttp, the
library rejects any header that has a "\r" or "\n" in its value:

  ValueError: Newline or carriage return character detected in HTTP status message or header. This is a potential security issue.

As an example, any curtin.util.ProcessExecutionError exception will
contain "\n" characters when converted into a string.

We now encode the error message as JSON before copying it in the HTTP
header.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-04 09:10:29 +02:00
Dan Bungert 674727595e
Merge pull request #1783 from dbungert/curtin-rev-apt
snapcraft: update curtin for apt distro fix
2023-09-01 17:07:41 -06:00
Dan Bungert 5aba05d267 snapcraft: update curtin for apt distro fix 2023-09-01 14:26:39 -06:00