Commit Graph

6687 Commits

Author SHA1 Message Date
Olivier Gayot 5553793f43 mirror: allow to query or set if archive/mirror is used during install
The /mirror GET and POST endpoints now include a boolean field named
"use_during_installation".

* if set to True, the mirror information will be used during
  installation to fetch packages online.

* if set to False, we will only fetch packages from the pool.

In either case, the mirror information will still be used to build the
etc/apt/ directory in the target system.

Currently, the way use_during_installation is implemented is coupled
with the force_offline property of the network model. This means that
Ubuntu Pro and other stuff will be disabled too if we're skipping the
use of the archive.

NOTE: the default value for "use_during_installation" in the POST
endpoint is `null` ; which means that we should not change the current
value of force_offline.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-04-11 11:00:26 +02:00
Olivier Gayot 6db2ff8113
Merge pull request #1961 from ogayot/lvm-size-max
storage: fix UI crash when partition size gets capped
2024-04-09 16:58:06 +02:00
Robert Krátký 797f223705
Merge pull request #1963 from rkratky/review-of-1956
Language and formatting fixes as a follow up to #1956
2024-04-09 13:14:37 +02:00
Robert Krátký 76440813bd
Merge pull request #1962 from rkratky/FR-5798_autoinstall-intro
Better introduction to autoinstall
2024-04-09 13:13:05 +02:00
Robert Krátký ce6dd33e80 Better introduction to autoinstall. 2024-04-09 12:58:51 +02:00
Robert Krátký 219f55258d Language and formatting fixes. 2024-04-05 15:58:08 +02:00
Olivier Gayot 15f6848dc6 storage: fix UI crash when partition size gets capped
In cda6c54b87, we introduced the accurate
size vs human readable size for the size form fields. Unfortunately, we
also introduced a regression when the size gets capped (when it exceeds
the maximum size defined on the form). Currently, specifying a size that
is beyond the maximum value fails when submitting the form with:

  [...]
  File "urwid/wimp.py", line 543, in keypress
    self._emit('click')
  File "urwid/widget.py", line 461, in _emit
    signals.emit_signal(self, name, self, *args)
  File "urwid/signals.py", line 265, in emit
    result |= self._call_callback(callback, user_arg, user_args, args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "urwid/signals.py", line 295, in _call_callback
    return bool(callback(*args_to_pass))
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "subiquitycore/ui/form.py", line 486, in _click_done
    emit_signal(self, "submit", self)
  File "urwid/signals.py", line 265, in emit
    result |= self._call_callback(callback, user_arg, user_args, args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "urwid/signals.py", line 295, in _call_callback
    return bool(callback(*args_to_pass))
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "subiquity/ui/views/filesystem/partition.py", line 636, in done
    handler(self.disk, spec, partition=self.partition, gap=self.gap)
  File "subiquity/common/filesystem/manipulator.py", line 334, in logical_volume_handler
    lv.size = align_up(spec["size"])
              ^^^^^^^^^^^^^^^^^^^^^^
  File "subiquity/models/filesystem.py", line 1382, in align_up
    return (size + block_size - 1) & ~(block_size - 1)
            ~~~~~^~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'BoundFormField' and 'int'

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-04-05 11:20:34 +02:00
Chris Peterson 16bcfcf7ae
Merge pull request #1956 from Chris-Peterson444/top-level-autoinstall
Allow top-level autoinstall in all delivery methods
2024-04-04 14:32:37 -07:00
Chris Peterson 26761b4f1a doc: update wordlist 2024-04-04 14:26:12 -07:00
Chris Peterson aed2905db0 doc: consistent top-level autoinstall keyword
Also deletes a note to users about not needing the autoinstall keyword
in non cloud-config delivery methods. We should keep a hint in the reference
that this is available, but generally let's always suggest to use the
top-level autoinstall keyword.
2024-04-04 14:26:08 -07:00
Chris Peterson ecfc3a4df0 scripts: support top-level autoinstall in validator 2024-04-04 14:21:02 -07:00
Chris Peterson ba3dbb52ef autoinstall: Allow "autoinstall" as top-level key 2024-04-04 14:20:58 -07:00
Chris Peterson a25f2e03c1 server: refactor reading in autoinstall
Factor out reading the autoinstall data from the loading function.
Also add context logging to both the loading and reading functions.
2024-04-04 13:57:35 -07:00
Chris Peterson d77bfbe1e0
Merge pull request #1954 from Chris-Peterson444/cloud-init-schema-failures
Cloud init schema failures
2024-04-03 16:35:07 -07:00
Chris Peterson 1b2c6befbb client: CloudInitSchemaValidationError in error overlay 2024-04-03 11:22:47 -07:00
Chris Peterson 8a31c7e72f autoinstall: Query cloud-init for schema failures
Users attempting to do autoinstall may incorrectly send autoinstall
directives as cloud-config, which will result in cloud-init
schema validation errors. When loading autoinstall from cloud-config,
we now check to see if there are any cloud-init schema validation errors
and warn the user. Additionally, if the source of the error is from
a known autoinstall error, we inform the user and halt the installation
with a nonreportable AutoinstallError.
2024-04-03 11:22:44 -07:00
Olivier Gayot 224d9cde1e
Merge pull request #1957 from ogayot/target-resize-part-mounted
filesystem: don't suggest target resize if part is mounted
2024-04-02 09:53:31 +02:00
Dan Bungert 72bac06005
Merge pull request #1958 from dbungert/ubuntu-desktop-bootstrap
fix a few bootstrap renames
2024-03-28 18:34:25 -06:00
Dan Bungert 12934d7b97 scripts: update for bootstrap 2024-03-28 17:20:43 -06:00
Dan Bungert 3422e3be2e server: fix bootstrap log 2024-03-28 17:20:21 -06:00
Olivier Gayot c093059919 filesystem: don't suggest target resize if part is mounted
When a partition is mounted in the live environment (either by casper or
another process) and is selected for resize, curtin will fail when
running e2fsck (or equivalent) with an error such as:

  Resize requested for format ext4
  Resizing /dev/sdb4 of type ext4 down to 12546211840
  Running command ['e2fsck', '-p', '-f', '/dev/sdb4'] with allowed return codes [0] (capture=False)
  /dev/sdb4 is mounted.
  e2fsck: Cannot continue, aborting.

  An error occured handling 'disk-sdb': ProcessExecutionError - Unexpected error while running command.
  Command: ['e2fsck', '-p', '-f', '/dev/sdb4']
  Exit code: 8

This is what happens when casper creates a partition on the installer
media to store /var/log and /var/crash. Usually the partition is large
enough to install Ubuntu so a GuidedStorageTargetResize scenario gets
emitted.

This patch prevents the emission of a GuidedStorageTargetResize scenario
if the partition is marked as mounted already.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-03-28 15:59:24 +01:00
Olivier Gayot eca86c58da
Merge pull request #1955 from ogayot/grub2-wa-drop
Revert grub2 workaround for cloud_style_installation
2024-03-27 14:35:14 +01:00
Olivier Gayot 092de2e598 Revert "debconf-selections: work around grub2 bug until it is fixed"
To workaround a grub2 bug (LP: #2055294) causing 24.04 installations to
fail, we added a default debconf-selection in Subiquity.

grub2 2.12-1ubuntu4 fixes the issue and migrated to the noble release
pocket on 2024-03-06.

We can now drop the workaround.

https://launchpad.net/ubuntu/+source/grub2/2.12-1ubuntu4

This reverts commit cffce32305.
2024-03-26 10:27:00 +01:00
Olivier Gayot cdb56a0d9f
Merge pull request #1952 from ogayot/zzzz-remove
Ensure the zzzz-temp-installer-unattended-upgrade file does not end up on the target system
2024-03-26 10:07:19 +01:00
Chris Peterson f307b87426 context: also send events to log 2024-03-25 20:18:00 -07:00
Chris Peterson ce0683adec
Merge pull request #1947 from Chris-Peterson444/strict-top-level-keys
Enforce top-level keys
2024-03-25 19:59:13 -07:00
Chris Peterson d812919366 autoinstall: enforce top-level keys
Subiquity currently ignores invalid top-level keys, but this has likely
been a major source of confusion of autoinstall capabilities.

In future versions, the following autoinstall config will throw an
AutoinstallValidationError:

    version: 2
    interactive-sections:
        - identity
    literally-anything: lmao

This patch adds warnings for version 1 and will begin to throw
an AutoinstallValidationError on these instances in version 2 once
it has been enabled.
2024-03-25 19:52:39 -07:00
Chris Peterson 5da4f9b777
Merge pull request #1946 from Chris-Peterson444/interactive-context-reading
Display new reporting events in the TUI
2024-03-25 17:14:33 -07:00
Chris Peterson 6e2f256f08 progress: show log-like reporting events
Allows for warning and error type events to be printed to the progress
view.
2024-03-25 17:07:55 -07:00
Chris Peterson 735771a494
Merge pull request #1945 from Chris-Peterson444/context-logging
Improve context logging
2024-03-25 17:05:24 -07:00
Dan Bungert 5496effe45
Merge pull request #1953 from dbungert/rev-flock-ex
snapcraft: rev for flock-ex
2024-03-25 16:59:23 -06:00
Dan Bungert ed9aab0289 snapcraft: rev for flock-ex 2024-03-25 11:31:46 -06:00
Olivier Gayot 2d8a1ea9b3 install: use pathlib.Path.write_bytes construct instead of open + write
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-03-25 18:04:40 +01:00
Olivier Gayot 092cbf14df install: remove zzzz-temp-installer-unattended-upgrade after running UU
On recent installations of Ubuntu, the zzzz-installer-unattended-upgrade
file ended up in the etc/apt/apt.conf.d directory of the target system.
It is supposed to be a temporary file intended for one time invocation
of unattended-upgrade at the end of the installation.

Let's remove the file after unattended-upgrades finishes using a
try ... finally construct.

The previous implementation was doing the call to UU inside the
with open(...) block, after a manual call to .close(). This is
unnecessary, the file is automatically closed at the end of the with
block.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-03-25 18:04:40 +01:00
Dan Bungert bef10517bd
Merge pull request #1951 from ogayot/ubuntu.sources.curtin.old
snapcraft: bump curtin revision for apt-config warning fix
2024-03-25 10:53:52 -06:00
Olivier Gayot 09b79aeaf2 snapcraft: bump curtin revision for apt-config warning fix
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-03-25 14:23:06 +01:00
Dan Bungert e54354b8a3
Merge pull request #1950 from dbungert/lp-2058394-no-format-obj-on-unformatted
filesystem: handle empty string fstype partitions
2024-03-24 20:42:56 -06:00
Dan Bungert fc05e77a4c filesystem: handle empty string fstype partitions
This should not result in a format object being created, which curtin
doesn't like.  Normalize to None as the fstype to skip format object
creation.

LP: #2058394
2024-03-22 16:33:21 -06:00
Chris Peterson ed8971f0b2 server: Create an interface for server event listeners 2024-03-22 07:58:10 -07:00
Dan Bungert 03b68337d9
Merge pull request #1949 from dbungert/autoinstall-user-data-fix-comment
install: fix comment in autoinstall-user-data
2024-03-22 08:04:43 -06:00
Dan Bungert 1ec1c33da1 install: fix comment in autoinstall-user-data 2024-03-21 18:05:41 -06:00
Chris Peterson 0eb05d31e1 client: support log-like events in non-interactive installs
This changes the journal listener callback in the non-interactive
scenario to support all types of events, not just start and finish
events.

This also modifies the journal listener in the non-interactive scenario
to not seek all the way to the newest entry. In non-interactive installs,
the server can get pretty far into the installation by the time the client
can connect, so any important warnings/errors that the server reports
before then wouldn't be printed to the console by the client otherwise.
2024-03-20 16:14:28 -07:00
Chris Peterson 980411a670 reporting: add new logging type events
Adds new reporting type events "INFO", "WARNING", and "ERROR" to be
used for context logging. These can be invoked with the new `.info`,
`.warning`, and `.error` methods on the context object accordingly.
Useful for things like warning/errors on autoinstall configuartions.
2024-03-20 16:14:28 -07:00
Chris Peterson c25e28ef2c server: refactor _maybe_push_to_journal 2024-03-20 16:14:28 -07:00
Chris Peterson e0f02a74d7 server: add tests for publishing journal events
Currently there is a lot of state used to determine which events get
published to the journal. This is a baseline before changing behavor.
2024-03-20 16:14:28 -07:00
Dan Bungert 4fba6aa7d3
Merge pull request #1948 from dbungert/workflow-remove-lunar
workflows: -lunar
2024-03-20 06:24:38 -06:00
Dan Bungert 29868998cb workflows: -lunar 2024-03-19 23:02:00 -06:00
Olivier Gayot 50dacee4a1
Merge pull request #1942 from ogayot/warnings-capture
logging: capture warnings emitted using warnings.warn()
2024-03-19 13:30:29 +01:00
Olivier Gayot cc15d645f4
Merge pull request #1943 from ogayot/curl-timeout
tests: terminate curl processes on timeout
2024-03-19 09:59:57 +01:00
Olivier Gayot 612a8bbfc9 tests: terminate curl processes on timeout
When ensuring that the system-setup process can only be connected to
on the loopback interface, we spawn a bunch of `curl --interface ...`
processes. If the connection times out (which is the expectation in most
scenarios), the curl processes ended up not being terminated. Not only
this is small waste of resources, this is also causing errors on noble:

Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x745692661300>
Traceback (most recent call last):
  File "/usr/lib/python3.12/asyncio/base_subprocess.py", line 126, in __del__
    self.close()
  File "/usr/lib/python3.12/asyncio/base_subprocess.py", line 104, in close
    proto.pipe.close()
  File "/usr/lib/python3.12/asyncio/unix_events.py", line 568, in close
    self._close(None)
  File "/usr/lib/python3.12/asyncio/unix_events.py", line 592, in _close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.12/asyncio/base_events.py", line 793, in call_soon
    self._check_closed()
  File "/usr/lib/python3.12/asyncio/base_events.py", line 540, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Fixed by terminating the curl processes (and waiting for them to
terminate) before exiting the script.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2024-03-19 09:21:31 +01:00