Commit Graph

6344 Commits

Author SHA1 Message Date
Michael Hudson-Doyle 01fd8d22a9
Merge pull request #1835 from dbungert/target-log-dir-mantic
shutdown: fix perms on /target/var/log/installer
2023-10-11 11:09:50 +13:00
Dan Bungert 59504601e7 shutdown: fix perms on /target/var/log/installer
(cherry picked from commit 2d7210557b)
2023-10-10 22:59:28 +01:00
Dan Bungert 7717b0d150
Merge pull request #1833 from dbungert/merge-variation-bug
Merge variation bug
2023-10-10 11:40:05 -06:00
Olivier Gayot 7471bb9682 filesystem: do not keep the variation info when the source changes
When the source changes, the available variations should change as well.
If we keep the old variations in the
FilesystemController._variations_info dictionary, we end up with a crash
later in the install.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit d4820497e7)
2023-10-10 18:28:53 +01:00
Olivier Gayot fd404757c2 examples: add desktop source with different variations
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 65612edc0b)
2023-10-10 18:28:53 +01:00
Dan Bungert 8b67e652d9
Merge pull request #1831 from dbungert/log-merge
log: move /var/log/installer back to 0770 root:adm
2023-10-10 07:08:44 -06:00
Dan Bungert 8b48df8a9e log: move /var/log/installer back to 0770 root:adm
(cherry picked from commit 84bcd5f7c8)
2023-10-10 13:38:40 +01:00
Dan Bungert 45310cb857
Merge pull request #1829 from dbungert/c-n-f-merge
apt: fix source.list mode
2023-10-09 09:18:34 -06:00
Dan Bungert f77176f1a6 apt: fix source.list mode
(cherry picked from commit 91e84c70ba)
2023-10-09 14:38:06 +01:00
Dan Bungert 3841511f40
Merge pull request #1824 from dbungert/mantic-translations-2023-10-05
merge 2023 10 05
2023-10-05 20:04:45 -06:00
Dan Bungert 0005ebc53e snapcraft: curtin logs change
(cherry picked from commit ca8f1ca826)
2023-10-05 19:00:03 -06:00
Dan Bungert d43b80c6d6 log: create /var/log/installer root only
(cherry picked from commit c9cfdafe0c)
2023-10-05 19:00:03 -06:00
Dan Bungert 8754456d83 several: turn off aiohttp access log
It will log arguments, so unless we are certain the arguments are clean
this will cause trouble.  Just turn it off.

(cherry picked from commit 1da5cac477)
2023-10-05 19:00:03 -06:00
Dan Bungert c36e3e7cd5 file_util: just make written files root only
(cherry picked from commit 80b144f220)
2023-10-05 19:00:03 -06:00
Dan Bungert 3fc3aa260f network: disable log line that logs psk
(cherry picked from commit f567f9e1f5)
2023-10-05 19:00:03 -06:00
Dan Bungert 83b883d979 revert en_GB, it has czech
(cherry picked from commit 0201c7756a)
2023-10-05 19:00:01 -06:00
Dan Bungert 2114201d70 po: merge from translations-export
(cherry picked from commit 6417720a2e)
2023-10-05 19:00:01 -06:00
Dan Bungert 6b4b39e827
Merge pull request #1821 from dbungert/merge-2023-10-04
Merge 2023 10 04
2023-10-05 10:00:15 -06:00
Olivier Gayot 04981fdc9f source: warn when POST /source specifies an unmatched source id
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit f6da6162e4)
2023-10-04 20:49:05 -06:00
Olivier Gayot f7c011cef3 source: do not fire a configured event again if nothing changed
When handling a POST request to /source, Subiquity sends a 'source
configured' event. This signals other controllers / models that they
need to restart their tasks that depend on the source being used.

However, if the user of the installer goes back all the way to the
source page and submits it again without changing the settings, there
should be no reason to restart the machinery.

If a call to source ends up doing no modification to the model (i.e.,
not changing the source used or the search_drivers setting), we now
avoid emitting the 'source configured' event ; except if the model has
not been configured yet.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit fff2f6591f)
2023-10-04 20:49:05 -06:00
Dan Bungert 74afeab1d4 install: create autoinstall-user-data 0400
CVE-2023-5182

As autoinstall-user-data contains a password hash hash for a user with
sudo access, create the autoinstall-user-data as 0400 root:root.

The old permissions are 0640 root:adm, and the adm group does not by
default have sudo access, so cracking that hash could lead to privilege
escallation for someone in the adm group.

Thanks to Patric Åhlin and Johan Hortling for identifying and reporting
the issue.

(cherry picked from commit 62e126896f)
2023-10-04 20:49:05 -06:00
Dan Bungert 32a3012c11 shutdown: fix owner on cloud-init logs
These have owner syslog at install time, but that is uid remapped on the
target system which may end up with a different owning user.

(cherry picked from commit ab0af6375e)
2023-10-04 20:49:05 -06:00
Dan Bungert 30c950cea9 shutdown: refactor cloud init logs logic
(cherry picked from commit d3debfcea3)
2023-10-04 20:49:05 -06:00
Dan Bungert c208ccfcab util: ensure log file is root owned
(cherry picked from commit a2b63dae13)
2023-10-04 20:49:05 -06:00
Dan Bungert f84a68577f util: set_log_perms tests
(cherry picked from commit 8ab052c200)
2023-10-04 20:49:05 -06:00
Dan Bungert b88f8a09f7 util: explicit isdir arg from set_log_perms
target already exists, we should just inspect target and find if it is a
directory or not.

(cherry picked from commit 4a4e8ba886)
2023-10-04 20:49:05 -06:00
Dan Bungert 8b0119e93a util: more control on file writer mode and group
(cherry picked from commit ddc11d8687)
2023-10-04 20:49:05 -06:00
Dan Bungert ed68c1c8c1 util: standardize on term 'mode'
(cherry picked from commit ddc3345eb6)
2023-10-04 20:49:05 -06:00
Chris Peterson 8827624a7f tests: remove obsolete commands
(cherry picked from commit 8f28063b92)
2023-10-04 20:49:04 -06:00
Chris Peterson 5d0a846f8a autoinstall: Don't use snap env when invoking early and late commands
(cherry picked from commit 39f1ea9cb6)
2023-10-04 20:49:04 -06:00
Dan Bungert 4a76b65edc
Merge pull request #1817 from ogayot/mantic-2023-10-02
Mantic merge 2023-10-02
2023-10-02 12:30:10 -06:00
Olivier Gayot 3dfce34240 workflows: differentiate CI and Snap workflows
The workflows defined respectively in build.yaml and snap.yaml were
both called "CI". On the Github web interface, it resulted in two menus
called "CI" with no easy way to know which is which.

To make things clearer, we now:
 * rename build.yaml -> ci.yaml
 * call "Snap" the workflow defined by snap.yaml

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit a34bce470f)
2023-10-02 18:17:14 +02:00
Olivier Gayot ecf56e1414 shutdown: do not try to unmount /target if install was not started
If we ask for reboot before the installation has started (i.e., if
curtin install was not invoked at least once), the following call fails
and prevents the system from rebooting.

 $ umount --recursive /target

Make sure we check that /target exists and is mounted before calling
umount.

Another approach would be to check the return value of umount but the
values are not documented.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit abef05178c)
2023-10-02 18:15:32 +02:00
Olivier Gayot 5eba140cbb codecs: skip installation when running an offline install
ubuntu-restricted-addons is a multiverse package and is not included in
the pool. Therefore, trying to get it installed when offline leads to an
obvious error.

Instead of making the whole Ubuntu installation fail, we now warn and
skip installation of the package when performing an offline install.
In a perfect world, we should not have offered to install the package in
the first place, but in practice, we can run an offline installation as
the result of failed mirror testing (bad network for instance).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 01ec1da86f)
2023-10-02 18:15:17 +02:00
Dan Bungert 17b4753027
Merge pull request #1810 from dbungert/mantic-2023-09-27
Mantic merge 2023-09-27
2023-09-27 15:32:07 -06:00
Dan Bungert 8276f61d9a filesystem: revamp udev handling
In LP: #2009141, we are hitting kernel limits and pyudev buffer limits.
We don't care about specific events, so much as getting one event,
waiting for things to calm down, then reprobing.

Outright disable the event monitor, and re-enable later.  If there is a
storm of events, testing has shown that stopping the listener is not
enough.

(cherry picked from commit b11726d398)
2023-09-27 14:12:26 -06:00
Olivier Gayot 7fa97fec2a oem: make sure storage is configured before using is_core_boot_classic
Before using fs_controller.is_core_boot_classic(), we wait for the call
to /meta/confirmation?tty=xxx. That said, in semi-automated desktop
installs, sometimes the call to /meta/confirmation happens before
marking storage configured. This leads to the following error:

  File "subiquity/server/controllers/oem.py", line 209, in apply_autoinstall_config
    await self.load_metapkgs_task
  File "subiquity/server/controllers/oem.py", line 81, in list_and_mark_configured
    await self.load_metapackages_list()
  File "subiquitycore/context.py", line 149, in decorated_async
    return await meth(self, **kw)
  File "subiquity/server/controllers/oem.py", line 136, in load_metapackages_list
    if fs_controller.is_core_boot_classic():
  File "subiquity/server/controllers/filesystem.py", line 284, in is_core_boot_classic
    return self._info.is_core_boot_classic()
AttributeError: 'NoneType' object has no attribute 'is_core_boot_classic'

Receiving the confirmation before getting the storage configured is
arguably wrong - but let's be prepared for it just in case.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 59849f7f45)
2023-09-27 13:50:06 -06:00
Olivier Gayot 0caf03c037 filesystem: don't crash if v2/orig_config is called early
When v2/orig_config is called too early, the load_probe_data function
will fail because probe_data is None:

  Traceback (most recent call last):
    File "subiquity/common/api/server.py", line 164, in handler
      result = await implementation(**args)
    File "subiquity/server/controllers/filesystem.py", line 1029, in v2_orig_config_GET
      model = self.model.get_orig_model()
    File "subiquity/models/filesystem.py", line 1428, in get_orig_model
      orig_model.load_probe_data(self._probe_data)
    File "subiquity/models/filesystem.py", line 1894, in load_probe_data
      for devname, devdata in probe_data["blockdev"].items():
  TypeError: 'NoneType' object is not subscriptable

Make sure we don't dereference model._probe_data if it is None.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 7de6f0538b)
2023-09-27 13:50:03 -06:00
Chris Peterson 8c76944e6b docs: locale must be specified to be interactive
(cherry picked from commit 4ba59a503b)
2023-09-27 13:50:01 -06:00
Olivier Gayot 0ba5f775b0 ui: have a distinct state file for rich mode over serial
We recently made sure that after doing a snap refresh, the rich mode
(i.e., either rich or basic) is preserved. This was implemented by
storing the rich mode in a state file. When the client starts, it loads
the rich mode from said state file if it exists.

Unfortunately, on s390x, it causes installs to default to basic mode.
This happens because on this architecture, a subiquity install consists
of:

 * a first client (over serial) showing the SSH password
 * a second client (logging over SSH) actually going through the
   installation UI.

Since the first client uses a serial connection, the state file is
created with rich-mode set to basic. Upon connecting using SSH, the
state file is read and the rich-mode is set to basic as well.

Fixed by storing the rich-mode in two separate files, one for clients
over serial and one for other clients.

LP: #2036096

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit c95261e0de)
2023-09-27 13:49:54 -06:00
Dan Bungert ad27603b12 snapd api: wait longer
While these changes are not supposed to take nearly this long,
per LP: #2034715 we know that they are, and that some systems will
correctly perform the finish_install() step if just given more time.

(cherry picked from commit 5a573f2cef)
2023-09-27 13:49:47 -06:00
Olivier Gayot cfb9cce8e9 Bump curtin rev for recovery key on systems using zkey
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 22b6d1258b)
2023-09-27 13:49:42 -06:00
Chris Peterson 03a3dd821a docs: reflect autoinstall options and precedence
(cherry picked from commit 28dd55f9dd)
2023-09-27 13:49:35 -06:00
Chris Peterson 090e104c34 autoinstall: allow for specifying autoinstall path on kernel command line
(cherry picked from commit 76b520afa9)
2023-09-27 13:49:35 -06:00
Dan Bungert 0d50d5a8ef
Merge pull request #1801 from dbungert/mantic-2023-09-19
Mantic merge 2023-09-19
2023-09-19 19:01:43 -06:00
Dan Bungert f22493974c snapcraft: rev curtin for deb822, large sectors
This curtin rev adds the following:
Dan Bungert (3):
      extract: log source information
      tests/data: 4k sector disk
      storage_config: handle partitions on 4k disk

Nick Rosbrook (1):
      apt: disable default deb822 migration

(cherry picked from commit ea7b683d8e)
2023-09-19 18:33:57 -06:00
Olivier Gayot e558ce2176 s390x: ensure chreipl is called before unmounting /target
For ZFS, we recently introduced a call to $(umount --recursive /target)
slighly before shutting down or rebooting. Unfortunately, on s390x, we
also had a very late call to chreipl to make the firmware boot from the
installed system.

The call to chreipl reads data from /target/boot, and it fails if the
filesystem is no longer mounted.

Fixed by calling chreipl earlier in the installation, during the
postinst phase rather than after the user clicks "reboot".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit cf828eeb8d)
2023-09-19 18:33:49 -06:00
Michael Hudson-Doyle cb8fa723b5 include api-only data when sending storage objects from client to server
Making an install that used an existing RAID failed because of an
attempt to log the size of the RAID when rendering the curtin config.

This turns out to be because when the client sends the storage objects
back to the server it loses all the "api only" data including the udev
data that is needed to display the size.

In some sense this is a bit silly, we could just drop the log statement
and it would be fine but I think it's probably better to always have the
full storage objects in the server (until we can get away from this
hackish API anyway).

(cherry picked from commit 4d24865a63)
2023-09-19 18:33:40 -06:00
Dan Bungert a4fe424df6
Merge pull request #1797 from Chris-Peterson444/faster-unattended-upgrades
Make unattended upgrades run faster by disabling minimal steps
2023-09-15 13:30:12 -06:00
Chris Peterson cd7d3dd2cb add Unattended-Upgrade::MinimalSteps "faslse" to uu_apt conf
Co-authored-by: Dan Bungert <danielbungert@gmail.com>
2023-09-15 12:20:08 -07:00