Commit Graph

5401 Commits

Author SHA1 Message Date
Olivier Gayot d4f76d1ce8
Merge pull request #1426 from ogayot/fix-snaplist
Make snaplist more robust to races and fix snap info prefetch
2022-09-22 15:18:31 +02:00
Olivier Gayot 15151675d6 snaplist: fix cancellation of loader preventing retry
The SingleInstanceTask does not allow us to control cancellation the way
we want. We now use a standard asyncio.Task to execute the coroutine
that fetches the list of snaps.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 15:02:18 +02:00
Olivier Gayot 507b025059 snaplist: make list snap task raise on error
The list snaps task used to return successfully even in case of error.

To make the distinction between and error and a successful fetch, we
used to rely on the task to set an attribute at the loader level:
 * failed = True; or
 * failed = False

The task was responsible for setting this attribute and there were
scenarios where it would not set the attribute properly. Usually, this
was when an exception was raised in the task.

We now raise an exception when an error occurs in the task ; instead of
returning. This is better because we can know if the task:
 * got cancelled - by calling "task.cancelled()"
 * finished successfully - by calling "task.done() and not
   task.exception()"
 * failed - by calling "task.done() and task.exception()"
 * has not finished - by calling "not task.done()"

This also allows us to detect errors and retry

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 15:02:18 +02:00
Olivier Gayot beb2d46d3f snaplist: prefetch snaps info after list of snaps is retrieved
Once we have access to the list of snaps, we can start prefetching
information. We need to wait for the list snaps task to be completed
first.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 15:02:18 +02:00
Olivier Gayot d770b37859
Merge pull request #1432 from mwhudson/serialize-date-argh
fix deserializing date fields
2022-09-22 14:41:47 +02:00
Michael Hudson-Doyle fa8bc2f7f1 fix deserializing date fields 2022-09-22 14:30:52 +02:00
Olivier Gayot 62a2f3104c kvm-test: allow to specify a cloud-init file not meant for autoinstall
The --cloud-config option now replaces the old --autoinstall-file
option.

The use of --autoinstall-file always made kvm-test pass the autoinstall
argument to the kernel command line. With the --cloud-config option, we
now determine if the autoinstall argument is needed by reading the cloud
config and checking if an autoinstall section is present.

The two new options --force-autoinstall and --force-no-autoinstall can
be used to override this behaviour.

The --autoinstall option is also dropped. If we want to use the default,
hardcoded cloud-config, one can use --cloud-config-default.

This can be useful to ease debugging of the VM using a config that looks
like:

    #cloud-config
    users:
        - default
        - name: root
          ssh_import_id: lp:ogayot

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 13:09:59 +02:00
Olivier Gayot aff9ed0a7c reserved-usernames: add sgx group ahead of ubuntu
sgx is not yet in reserved-usernames in Ubuntu but I submitted a
.debdiff to get it added:

https://bugs.launchpad.net/ubuntu/+source/user-setup/+bug/1990488

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 11:09:30 +02:00
Olivier Gayot dea6cea4ef reserved-usernames: refresh local copy using upstream
Refresh the reserved-usernames local copy using:

https://git.launchpad.net/ubuntu/+source/user-setup/plain/reserved-usernames

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 11:08:50 +02:00
Olivier Gayot b45f47acf2 reserved-usernames: build using latest version from user-setup.git
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-22 11:07:10 +02:00
Michael Hudson-Doyle 9f32abd8f4 remove special cases around mounting/unmounting /target/cdrom 2022-09-22 13:54:03 +12:00
Michael Hudson-Doyle 3a95193905
Merge pull request #1423 from mwhudson/serialization-enhancements
serialization enhancements in preparation for talking to snapd api
2022-09-22 12:07:05 +12:00
Carlos Nihelton ef5984b1be
Merge pull request #1427 from CarlosNihelton/2stage-snap-wsl-87
[WSL] Two stage snap launching
2022-09-21 21:05:29 -03:00
Dan Bungert e16f81293c
Merge pull request #1428 from dbungert/lp-1989353-ai-vs-kernel
kernel: fix autoinstall and run file
2022-09-21 07:51:24 -06:00
Carlos Nihelton 09bc1b7386
Ensures this script only runs in WSL
To prevent issues with desktop or server.
2022-09-21 09:26:13 -03:00
Michael Hudson-Doyle c063717897
Merge pull request #1424 from mwhudson/api-enhancements
api extensions needed to talk to snapd API
2022-09-21 14:24:25 +12:00
Dan Bungert 884f29627c kernel: fix autoinstall and run file 2022-09-20 08:43:08 -06:00
Dan Bungert 5c636acc30
Merge pull request #1429 from ogayot/LP#1990239
cdrom: make sure we unmount the cdrom only once
2022-09-20 08:39:29 -06:00
Carlos Nihelton 0ad27c2052
Merge pull request #1422 from CarlosNihelton/fix-wrong-parameters-loaded
[WSL] Fixes wrong parameters being passed to init WslSetupOptions
2022-09-20 08:59:02 -03:00
Olivier Gayot 474491567a
Merge pull request #1419 from ogayot/LP#1980584
mirror: only start configuring APT once source & mirror are configured
2022-09-20 11:33:30 +02:00
Olivier Gayot 74cad94efa
Merge pull request #1425 from ogayot/fix-try-again-not-waiting
snaplist: do not automatically mark configured on failed GET request
2022-09-20 11:32:26 +02:00
Olivier Gayot 5f1f2ec431 cdrom: make sure we unmount the cdrom only once
Until recently, subiquity and curtin had shared ownership regarding
mounting/unmounting the cdrom to/from the target:

 * curtin was responsible for bind-mounting /cdrom to /target/cdrom
 * subiquity was responsible for unmounting /target/cdrom

We recently made subiquity responsible for bind-mounting the cdrom to
the target too, which is a good thing.

Unfortunately, we now attempt to unmount the cdrom from the target
twice, at the end of the installation:

 * once because we explicitly unmount the cdrom from the target before
   restoring the APT config on target
 * once because subiquity has a mechanism that automatically unmount
   everything that it mounted, in reverse order.

This leads to a crash at the end of the installation.

Fix this issue by making sure we only try to unmount the cdrom once. If
we unmount it before restoring the APT config, then we don't want to
unmount it automatically anymore.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-20 11:22:03 +02:00
Olivier Gayot 505318625e
Merge pull request #1418 from ogayot/LP#1988407
filesystem: fix crash when editing LVM logical volume
2022-09-20 10:04:34 +02:00
Michael Hudson-Doyle 3a55816ee1 rewrite serialization to make path tracking and errors cleaner 2022-09-20 14:11:21 +12:00
Carlos Nihelton 5331268600
Adds the added script to the snap
Following the same path and naming conventions as UDI.
That allows for reuse of the existing snap mounting/bootstrapping
approach.
2022-09-19 14:41:24 -03:00
Carlos Nihelton c0c4b3c7fe
Adds a bootstrapping script for WSL
Where we don't have a proper snapd support.
Approach similar to the one used in UDI.
For the case where we have Subiquity snap, but not UDI.
The script added is a stripped version of the UDI one.

Removed code related to:
 - GTK and GDK;
 - GI repository;
 - Xkb;

 Preserved:
 - Python;
 - XDG directory definitions;
 - font config;
2022-09-19 14:39:39 -03:00
Michael Hudson-Doyle a16f75b2b8 extend TestEndToEnd.test_defaults a bit 2022-09-19 14:56:54 +12:00
Olivier Gayot 83a7aff316 snaplist: do not automatically mark configured on failed GET request
When a client sends a GET /snaplist request and an error occurs on the
server side, we return a response of the following form:

 { "status": "FAILED", ... }

This effectively gives a chance to the client to retry the same request
and expect a different outcome (for instance when a temporary network
failure occurs).

Having said that, when the server returns "status": "FAILED", it also
automatically marks the snaplist model configured. This is wrong because
if the next GET /snaplist request succeeds, the user will be prompted to
select some snaps to install ; but the installation will already be
running in the background. Depending on the timing, it might or might
not install the snaps that the user selected.

Fixed by not marking the snaplist model configured automatically. The
client is made responsible from sending a POST /snaplist ; even in case
of error.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-16 15:46:28 +02:00
Michael Hudson-Doyle 6262d5cfbd subiquity.common.api: allow parts of the path to an API to be parameters
I've wanted to do this for ages and getting ready to talk to the snapd
API provided an excuse :-) (as, for example, to get info about a snap
you talk to the /v2/snaps/{snap_name} endpoint). As before, the server
part isn't strictly needed but I forgot about that and actually
implemented it first. Oh well, we'll need it if I implement my idea
described in a comment in apidef.py to make the network API more RESTish.
2022-09-16 17:25:21 +12:00
Michael Hudson-Doyle 63d52705bf subiquity.common.api: have make_client return an instance
I don't really remember why I didn't do it this way to start with, to be
honest.
2022-09-16 17:25:21 +12:00
Michael Hudson-Doyle ee674c0ed2 subiquity.common.api: add a way to not serialize query arguments
The snapd API does not JSON-encode query arguments, they are just all
strings. I only need the client side of this really but implementing it
server side might avoid confusion down the road.
2022-09-16 17:25:21 +12:00
Michael Hudson-Doyle 4b6e34454b subiquity.common.serialize: test that a certain style of embedding works
Some snapd APIs do this sort of thing. It works fine without code
changes, luckily.
2022-09-16 16:37:18 +12:00
Michael Hudson-Doyle 64acb295c9 subiquity.common.serialize: allow serializing enums by value
Similarly to field names, the snapd API returns objects with values that
should be part of an enumeration, but with values that are not all
Python identifiers (e.g. "system-seed"). To allow these to be mapped to
enum's in Python that are not insanely annoying to use, allow a
Serializer to serialize enum's to and from their values rather than
their names (which remains the default behaviour).
2022-09-16 16:31:35 +12:00
Michael Hudson-Doyle 0d17c2e135 subiquity.common.seralize: allow overriding field names
Many snapd APIs return JSON objects with field names that are not valid
Python identifiers (such as "display-name") so if we're going to
translate them to and from Python objects we need to be able to override
the field name in the serialized form.
2022-09-16 16:21:34 +12:00
Michael Hudson-Doyle 5e64fe1ca1 subiquity.common.serialize: allow ignoring unknown fields
Some snapd APIs return JSON objects with lots and lots of fields and I
don't want to have to declare the ones I am not interested in. Also, I
think snapd adds fields over time, so I don't want to be broken when
this happens.
2022-09-16 16:17:26 +12:00
Carlos Nihelton 2a598a8b5d
[WSL] Fixes wrong paramters being passed ...
to the WSLSetupOptions controller.

Misuse of the default_loader().
That's meant for loading /etc/wsl.conf.
WSLSetupOptions is by design not related to that config file.
2022-09-15 17:12:03 -03:00
Carlos Nihelton a04d93451b
Exercises the crash with an API test.
Required changes to make default_loader testable.
2022-09-15 17:12:03 -03:00
Carlos Nihelton b4734ad587
Makes the default_loader for wsl.conf testable 2022-09-15 16:24:18 -03:00
Michael Hudson-Doyle 60d6746a41
Merge pull request #1417 from mwhudson/install-step-refactor
refactor CurtinInstallStep a bit
2022-09-15 08:44:22 +12:00
Olivier Gayot ecb6cb6399 filesystem: fix crash when editing LVM logical volume
When editing a LVM logical volume (LV), Subiquity would crash with the
following error:

filesystem/gaps.py", line 244, in movable_trailing_partitions_and_gap_size
    pgs = parts_and_gaps(partition.device)
AttributeError: 'LVM_LogicalVolume' object has no attribute 'device'

When we changed the implementation to use
movable_trailing_partitions_and_gap_size, we did not pay attention to
the LVM use case.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-14 19:12:53 +02:00
Olivier Gayot b9f9aee399 mirror: only start configuring APT once source & mirror are configured
The mirror controller is responsible from creating the APT configuration
once its associated model is marked configured. However, the APT
configuration is also dependent on the source model being marked
configured.

When subiquity is used for the server installer, the client immediately
sends a POST /a/meta/client_variant request. This ends up marking the
source model as configured (even though the default value does not make
much sense). So we never end up in a scenario where the mirror model is
marked configured before the source model.

When Subiquity is used by a different client, such as the
ubuntu-desktop-installer, it is possible that we mark the mirror model
configured first. This ends up crashing the installer.

Fixed by adding a dependency on the source model so that we do not end
up creating the APT configuration too early.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-09-14 17:28:40 +02:00
Michael Hudson-Doyle 37500beaa1 lint 2022-09-14 22:55:55 +12:00
Michael Hudson-Doyle 7d716030b2 apply suggestion from review 2022-09-14 21:44:34 +12:00
Michael Hudson-Doyle 972c59a198 remove unused code 2022-09-14 21:14:17 +12:00
Michael Hudson-Doyle 6d724c0a5b use new flexibility to remove a bit of mystery code from SubiquityModel 2022-09-14 21:11:57 +12:00
Michael Hudson-Doyle c34f0a8bfa make CurtinInstallStep more self-contained 2022-09-14 12:01:00 +12:00
Carlos Nihelton 80869d23f2
Merge pull request #1416 from CarlosNihelton/fix-missing-common
Fix the missing `system_setup.common` error.
2022-09-13 16:18:18 -03:00
Carlos Nihelton 08d4c39387
Fix the missing `system_setup.common` error.
When Subiquity is packaged as its own snap, that subdirectory is not
present.
That is due the lack of an `__init__.py` file.
2022-09-13 15:56:37 -03:00
Olivier Gayot 3aee57f6d0
Merge pull request #1414 from ogayot/fix-ubuntu-pro-not-configured
ubuntu-pro: fixed model not being marked configured on LTS
2022-09-12 17:33:19 +02:00
Olivier Gayot ddddc4356d
Merge pull request #1413 from ogayot/LP#1986674
snaplist: avoid showing snaplist screen on failure
2022-09-12 17:32:41 +02:00