Commit Graph

6333 Commits

Author SHA1 Message Date
Dan Bungert d43b80c6d6 log: create /var/log/installer root only
(cherry picked from commit c9cfdafe0c)
2023-10-05 19:00:03 -06:00
Dan Bungert 8754456d83 several: turn off aiohttp access log
It will log arguments, so unless we are certain the arguments are clean
this will cause trouble.  Just turn it off.

(cherry picked from commit 1da5cac477)
2023-10-05 19:00:03 -06:00
Dan Bungert c36e3e7cd5 file_util: just make written files root only
(cherry picked from commit 80b144f220)
2023-10-05 19:00:03 -06:00
Dan Bungert 3fc3aa260f network: disable log line that logs psk
(cherry picked from commit f567f9e1f5)
2023-10-05 19:00:03 -06:00
Dan Bungert 83b883d979 revert en_GB, it has czech
(cherry picked from commit 0201c7756a)
2023-10-05 19:00:01 -06:00
Dan Bungert 2114201d70 po: merge from translations-export
(cherry picked from commit 6417720a2e)
2023-10-05 19:00:01 -06:00
Dan Bungert 6b4b39e827
Merge pull request #1821 from dbungert/merge-2023-10-04
Merge 2023 10 04
2023-10-05 10:00:15 -06:00
Olivier Gayot 04981fdc9f source: warn when POST /source specifies an unmatched source id
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit f6da6162e4)
2023-10-04 20:49:05 -06:00
Olivier Gayot f7c011cef3 source: do not fire a configured event again if nothing changed
When handling a POST request to /source, Subiquity sends a 'source
configured' event. This signals other controllers / models that they
need to restart their tasks that depend on the source being used.

However, if the user of the installer goes back all the way to the
source page and submits it again without changing the settings, there
should be no reason to restart the machinery.

If a call to source ends up doing no modification to the model (i.e.,
not changing the source used or the search_drivers setting), we now
avoid emitting the 'source configured' event ; except if the model has
not been configured yet.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit fff2f6591f)
2023-10-04 20:49:05 -06:00
Dan Bungert 74afeab1d4 install: create autoinstall-user-data 0400
CVE-2023-5182

As autoinstall-user-data contains a password hash hash for a user with
sudo access, create the autoinstall-user-data as 0400 root:root.

The old permissions are 0640 root:adm, and the adm group does not by
default have sudo access, so cracking that hash could lead to privilege
escallation for someone in the adm group.

Thanks to Patric Åhlin and Johan Hortling for identifying and reporting
the issue.

(cherry picked from commit 62e126896f)
2023-10-04 20:49:05 -06:00
Dan Bungert 32a3012c11 shutdown: fix owner on cloud-init logs
These have owner syslog at install time, but that is uid remapped on the
target system which may end up with a different owning user.

(cherry picked from commit ab0af6375e)
2023-10-04 20:49:05 -06:00
Dan Bungert 30c950cea9 shutdown: refactor cloud init logs logic
(cherry picked from commit d3debfcea3)
2023-10-04 20:49:05 -06:00
Dan Bungert c208ccfcab util: ensure log file is root owned
(cherry picked from commit a2b63dae13)
2023-10-04 20:49:05 -06:00
Dan Bungert f84a68577f util: set_log_perms tests
(cherry picked from commit 8ab052c200)
2023-10-04 20:49:05 -06:00
Dan Bungert b88f8a09f7 util: explicit isdir arg from set_log_perms
target already exists, we should just inspect target and find if it is a
directory or not.

(cherry picked from commit 4a4e8ba886)
2023-10-04 20:49:05 -06:00
Dan Bungert 8b0119e93a util: more control on file writer mode and group
(cherry picked from commit ddc11d8687)
2023-10-04 20:49:05 -06:00
Dan Bungert ed68c1c8c1 util: standardize on term 'mode'
(cherry picked from commit ddc3345eb6)
2023-10-04 20:49:05 -06:00
Chris Peterson 8827624a7f tests: remove obsolete commands
(cherry picked from commit 8f28063b92)
2023-10-04 20:49:04 -06:00
Chris Peterson 5d0a846f8a autoinstall: Don't use snap env when invoking early and late commands
(cherry picked from commit 39f1ea9cb6)
2023-10-04 20:49:04 -06:00
Dan Bungert 4a76b65edc
Merge pull request #1817 from ogayot/mantic-2023-10-02
Mantic merge 2023-10-02
2023-10-02 12:30:10 -06:00
Olivier Gayot 3dfce34240 workflows: differentiate CI and Snap workflows
The workflows defined respectively in build.yaml and snap.yaml were
both called "CI". On the Github web interface, it resulted in two menus
called "CI" with no easy way to know which is which.

To make things clearer, we now:
 * rename build.yaml -> ci.yaml
 * call "Snap" the workflow defined by snap.yaml

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit a34bce470f)
2023-10-02 18:17:14 +02:00
Olivier Gayot ecf56e1414 shutdown: do not try to unmount /target if install was not started
If we ask for reboot before the installation has started (i.e., if
curtin install was not invoked at least once), the following call fails
and prevents the system from rebooting.

 $ umount --recursive /target

Make sure we check that /target exists and is mounted before calling
umount.

Another approach would be to check the return value of umount but the
values are not documented.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit abef05178c)
2023-10-02 18:15:32 +02:00
Olivier Gayot 5eba140cbb codecs: skip installation when running an offline install
ubuntu-restricted-addons is a multiverse package and is not included in
the pool. Therefore, trying to get it installed when offline leads to an
obvious error.

Instead of making the whole Ubuntu installation fail, we now warn and
skip installation of the package when performing an offline install.
In a perfect world, we should not have offered to install the package in
the first place, but in practice, we can run an offline installation as
the result of failed mirror testing (bad network for instance).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 01ec1da86f)
2023-10-02 18:15:17 +02:00
Dan Bungert 17b4753027
Merge pull request #1810 from dbungert/mantic-2023-09-27
Mantic merge 2023-09-27
2023-09-27 15:32:07 -06:00
Dan Bungert 8276f61d9a filesystem: revamp udev handling
In LP: #2009141, we are hitting kernel limits and pyudev buffer limits.
We don't care about specific events, so much as getting one event,
waiting for things to calm down, then reprobing.

Outright disable the event monitor, and re-enable later.  If there is a
storm of events, testing has shown that stopping the listener is not
enough.

(cherry picked from commit b11726d398)
2023-09-27 14:12:26 -06:00
Olivier Gayot 7fa97fec2a oem: make sure storage is configured before using is_core_boot_classic
Before using fs_controller.is_core_boot_classic(), we wait for the call
to /meta/confirmation?tty=xxx. That said, in semi-automated desktop
installs, sometimes the call to /meta/confirmation happens before
marking storage configured. This leads to the following error:

  File "subiquity/server/controllers/oem.py", line 209, in apply_autoinstall_config
    await self.load_metapkgs_task
  File "subiquity/server/controllers/oem.py", line 81, in list_and_mark_configured
    await self.load_metapackages_list()
  File "subiquitycore/context.py", line 149, in decorated_async
    return await meth(self, **kw)
  File "subiquity/server/controllers/oem.py", line 136, in load_metapackages_list
    if fs_controller.is_core_boot_classic():
  File "subiquity/server/controllers/filesystem.py", line 284, in is_core_boot_classic
    return self._info.is_core_boot_classic()
AttributeError: 'NoneType' object has no attribute 'is_core_boot_classic'

Receiving the confirmation before getting the storage configured is
arguably wrong - but let's be prepared for it just in case.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 59849f7f45)
2023-09-27 13:50:06 -06:00
Olivier Gayot 0caf03c037 filesystem: don't crash if v2/orig_config is called early
When v2/orig_config is called too early, the load_probe_data function
will fail because probe_data is None:

  Traceback (most recent call last):
    File "subiquity/common/api/server.py", line 164, in handler
      result = await implementation(**args)
    File "subiquity/server/controllers/filesystem.py", line 1029, in v2_orig_config_GET
      model = self.model.get_orig_model()
    File "subiquity/models/filesystem.py", line 1428, in get_orig_model
      orig_model.load_probe_data(self._probe_data)
    File "subiquity/models/filesystem.py", line 1894, in load_probe_data
      for devname, devdata in probe_data["blockdev"].items():
  TypeError: 'NoneType' object is not subscriptable

Make sure we don't dereference model._probe_data if it is None.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 7de6f0538b)
2023-09-27 13:50:03 -06:00
Chris Peterson 8c76944e6b docs: locale must be specified to be interactive
(cherry picked from commit 4ba59a503b)
2023-09-27 13:50:01 -06:00
Olivier Gayot 0ba5f775b0 ui: have a distinct state file for rich mode over serial
We recently made sure that after doing a snap refresh, the rich mode
(i.e., either rich or basic) is preserved. This was implemented by
storing the rich mode in a state file. When the client starts, it loads
the rich mode from said state file if it exists.

Unfortunately, on s390x, it causes installs to default to basic mode.
This happens because on this architecture, a subiquity install consists
of:

 * a first client (over serial) showing the SSH password
 * a second client (logging over SSH) actually going through the
   installation UI.

Since the first client uses a serial connection, the state file is
created with rich-mode set to basic. Upon connecting using SSH, the
state file is read and the rich-mode is set to basic as well.

Fixed by storing the rich-mode in two separate files, one for clients
over serial and one for other clients.

LP: #2036096

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit c95261e0de)
2023-09-27 13:49:54 -06:00
Dan Bungert ad27603b12 snapd api: wait longer
While these changes are not supposed to take nearly this long,
per LP: #2034715 we know that they are, and that some systems will
correctly perform the finish_install() step if just given more time.

(cherry picked from commit 5a573f2cef)
2023-09-27 13:49:47 -06:00
Olivier Gayot cfb9cce8e9 Bump curtin rev for recovery key on systems using zkey
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 22b6d1258b)
2023-09-27 13:49:42 -06:00
Chris Peterson 03a3dd821a docs: reflect autoinstall options and precedence
(cherry picked from commit 28dd55f9dd)
2023-09-27 13:49:35 -06:00
Chris Peterson 090e104c34 autoinstall: allow for specifying autoinstall path on kernel command line
(cherry picked from commit 76b520afa9)
2023-09-27 13:49:35 -06:00
Dan Bungert 0d50d5a8ef
Merge pull request #1801 from dbungert/mantic-2023-09-19
Mantic merge 2023-09-19
2023-09-19 19:01:43 -06:00
Dan Bungert f22493974c snapcraft: rev curtin for deb822, large sectors
This curtin rev adds the following:
Dan Bungert (3):
      extract: log source information
      tests/data: 4k sector disk
      storage_config: handle partitions on 4k disk

Nick Rosbrook (1):
      apt: disable default deb822 migration

(cherry picked from commit ea7b683d8e)
2023-09-19 18:33:57 -06:00
Olivier Gayot e558ce2176 s390x: ensure chreipl is called before unmounting /target
For ZFS, we recently introduced a call to $(umount --recursive /target)
slighly before shutting down or rebooting. Unfortunately, on s390x, we
also had a very late call to chreipl to make the firmware boot from the
installed system.

The call to chreipl reads data from /target/boot, and it fails if the
filesystem is no longer mounted.

Fixed by calling chreipl earlier in the installation, during the
postinst phase rather than after the user clicks "reboot".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit cf828eeb8d)
2023-09-19 18:33:49 -06:00
Michael Hudson-Doyle cb8fa723b5 include api-only data when sending storage objects from client to server
Making an install that used an existing RAID failed because of an
attempt to log the size of the RAID when rendering the curtin config.

This turns out to be because when the client sends the storage objects
back to the server it loses all the "api only" data including the udev
data that is needed to display the size.

In some sense this is a bit silly, we could just drop the log statement
and it would be fine but I think it's probably better to always have the
full storage objects in the server (until we can get away from this
hackish API anyway).

(cherry picked from commit 4d24865a63)
2023-09-19 18:33:40 -06:00
Dan Bungert a4fe424df6
Merge pull request #1797 from Chris-Peterson444/faster-unattended-upgrades
Make unattended upgrades run faster by disabling minimal steps
2023-09-15 13:30:12 -06:00
Chris Peterson cd7d3dd2cb add Unattended-Upgrade::MinimalSteps "faslse" to uu_apt conf
Co-authored-by: Dan Bungert <danielbungert@gmail.com>
2023-09-15 12:20:08 -07:00
Dan Bungert 3540ad077f
Merge pull request #1791 from dbungert/lp-2034939-guided-zfs-align
storage: guided_zfs must align the swap part size
2023-09-11 11:55:19 -06:00
Dan Bungert e0cd93881b
Merge pull request #1793 from dbungert/workflow-tidy
workflows: to checkout@v4
2023-09-11 10:24:25 -06:00
Dan Bungert baa475aa15 storage: guided_zfs must align the swap part size 2023-09-11 10:13:05 -06:00
Dan Bungert 705c752320
Merge pull request #1792 from dbungert/snap-builds
Fix snap build, and CI enforce it
2023-09-11 10:12:19 -06:00
Dan Bungert a0c776008b workflows: standardize on checkout@v4 2023-09-11 10:02:28 -06:00
Dan Bungert 7ef514f79a workflows: build a snap 2023-09-11 09:53:27 -06:00
Dan Bungert d84c445612 network: fix import during snap build
Adding this import means a dependency on probert, which also means
anybody importing subiquity.common.types also has that requirement.

The make-kbd-info script imports types, and that steps was causing
snapcraft build failures due to not finding probert.
2023-09-11 09:45:28 -06:00
Dan Bungert f7c5d8c665
Merge pull request #1790 from ogayot/security-archive
Ensure the security archive is set to the proper URL
2023-09-08 15:45:21 -06:00
Olivier Gayot 75ab969d8b snapcraft: update curtin - deb822 source fix when all series disabled
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 22:59:34 +02:00
Dan Bungert afc0f3faa3
Merge pull request #1763 from s-makin/doc-move-pages
[docs] Add content and change to reST
2023-09-08 06:33:07 -06:00
Olivier Gayot 5556313652 mirror: do not let curtin decide the URL of the security archive
When the URL of the security archive is unset, curtin will set it to the
URL of the primary archive.

This is not the behavior we want for Ubuntu installations. On amd64 (and
i386), the URL of the security archive should be set to
http://security.ubuntu.com/ubuntu

On other architectures, it should be set to
http://ports.ubuntu.com/ubuntu-ports

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-09-08 10:25:02 +02:00