Commit Graph

6344 Commits

Author SHA1 Message Date
Olivier Gayot 5bf3f6a818 workflow: produce a diff of the mypy errors before/after a PR
A new job now runs on PRs. It will automatically run mypy on the target
branch and on the source branch. It will generate a diff of the errors,
showing the new ones and showing the ones that have been fixed.

It will also show a summary with the number of errors before/after the
PR.

Because we have so many false positive, it makes no sense to mark this
job red. So we always make it green (unless mypy can't run).

As for now, it's up to developers to go check if any new error is
introduced.

If a line that used to produce a mypy error gets moved, it will be
reported as:

 * a fixed error
 * a newly introduced error

This is suboptimal and ideally we should have a way to detect moves.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-22 10:29:20 +02:00
Olivier Gayot 058a106a9f
Merge pull request #1679 from ogayot/latest-curtin-probert
snapcraft: use latest curtin and probert
2023-05-17 08:34:23 +02:00
Dan Bungert 2b25e8f36a
Merge pull request #1675 from canonical/canary-23.04
Canary 23.04
2023-05-16 16:39:06 -06:00
Olivier Gayot 39623077d7 snapcraft: use latest curtin and probert
This will pull the fixes for the utf-8 partition names.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-16 18:00:03 +02:00
Ryan Mounce 7dc9668862 Support matching disks on DEVPATH or ID_PATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 11:25:32 +09:30
Ryan Mounce 5ba82a3974 Fix doc, disk path matcher does not consider udev DEVPATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 10:54:56 +09:30
Dan Bungert b49031c5fb mounter: handle destination not existing
bind_mount_tree really wants to mount to places that don't exist yet,
and to unmount them later.  Simply creating these files or directories
gets weird.  Manage this process in the Mounter, so that any of these
temporary mountpoints get cleaned up on unmount.
2023-05-15 16:50:27 -06:00
Dan Bungert 70549c4c70 filesystem: no bind_mount in dryrun 2023-05-15 16:50:27 -06:00
Dan Bungert d69bc0dd77 filesystem: select gap by offset
Given disk layout like so
[p1 g1 p2 p3 g2]
The code attempting to create p4 used the largest_gap() method to
attempt to select g2. If the disk was small enough, below 28GiB with
current numbers, g1 would actually be larger, but the sizing math was
done with the offset that was correct for g2, so p4 would be created
a very incorrect size (-800MiB or so).

Just select gap by offset - we have the info needed.
2023-05-15 16:50:27 -06:00
Dan Bungert e2d55f50a2 apitest: example has no bios compat part 2023-05-15 16:50:27 -06:00
Dan Bungert c9180f09c1 examples: improve enhanced-secureboot 2023-05-15 16:50:27 -06:00
Dan Bungert 925d3e0a40 filesystem: fix snapd label lookup 2023-05-15 16:50:27 -06:00
Dan Bungert 0d4119070c filesystem: also mount snaps dir
We need these for valid responses from snapd v2/systems.
2023-05-15 16:50:27 -06:00
Dan Bungert 1bcf66c89b filesystem: give Source.get_handler variation name
In order to lookup the snapd system correctly, we need the correct
variant information, and it works best by name.
2023-05-15 16:50:27 -06:00
Dan Bungert 03c48daded test/api: add a simple coreboot testcase 2023-05-15 16:50:27 -06:00
Dan Bungert 451c4d7b74 storage/v1: handle variations for guided_GET
Purposefully simple as this is not believed to be a real use case to
have multiple variations.
2023-05-15 16:50:27 -06:00
Dan Bungert 1cc39ab7ee examples: fake enhanced-secureboot-desktop 2023-05-15 16:50:27 -06:00
Michael Hudson-Doyle b5ba12bf2c filesystem: enumerate and choose variation 2023-05-15 16:50:27 -06:00
Dan Bungert ebc7e39ea8 source: test expected model characteristics 2023-05-15 16:49:15 -06:00
Michael Hudson-Doyle 5105eb6c82 parse variations out of the source catalog
This parses variations but nothing iterates over them or selects a
non-arbitrary one yet.
2023-05-15 16:49:15 -06:00
Dan Bungert 02bfe89c93
Merge pull request #1676 from dbungert/workflow-mantic
workflow: +mantic
2023-05-12 07:53:51 -06:00
Dan Bungert f21dac28f2 workflow: +mantic 2023-05-12 07:44:16 -06:00
Olivier Gayot cf1e13ac12
Merge pull request #1671 from ogayot/no-reorder-uefi
install: request curthooks not to reorder UEFI boot entries
2023-04-28 10:08:41 +02:00
Olivier Gayot a270019387 prober: turn on parallelism for probert storage
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-27 11:09:23 +02:00
Olivier Gayot 782481b357
Merge pull request #1670 from ogayot/asyncio-probert
prober: move to asyncio for probert storage
2023-04-27 11:08:33 +02:00
Olivier Gayot 0bf2d9d38d install: request curthooks not to reorder UEFI boot entries
By default, curthooks will automatically reorder the UEFI boot entries
so that the media used during installation is set as the first boot
entry.

This is something that MAAS wants because MAAS typically boots over the
network.

This is not typically something we would want for a server or desktop
installer though. Most of the time, the media is a removable device.
Ideally, we would want the new installed grub to be the first boot
entry.

Furthermore, on some UEFI implementations, the BootCurrent does not
correspond to a Boot#### entry when booting from a removable media. When
this happens, curtin fails at making the media the first boot entry,
resulting in the following error:

  Invalid BootOrder order entry valueXXXX
                                       ^
  efibootmgr: entry XXXX does not exist

Fixed by disabling the UEFI boot entries reordering when invoking
curthooks.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-26 13:42:07 +02:00
Olivier Gayot 60f2b506db prober: move to asyncio for probert storage
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-25 15:12:19 +02:00
Olivier Gayot 1607a8a95b
Merge pull request #1669 from ogayot/logical+swap
filesystem: fix logical+swap or logical+boot partition not considered logical
2023-04-25 11:27:30 +02:00
Olivier Gayot 81acb6eb9f snapcraft: bump curtin to handled logical swap partitions
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-25 10:56:58 +02:00
Olivier Gayot 36161c0040 filesystem: fix logical+swap or logical+boot partition not considered logical
Some people who installed a previous version of Ubuntu on a DOS
partition table ended up with a logical, ESP partition (which is marked
bootable).

This is an unsupported use-case. An ESP partition should always be a
primary partition. Ideally, an ESP partition should be on a GPT.

However, curtin's storage_config makes it clear that expecting the flag
of a logical partition to be set to 'logical' is a bad idea.

  # if the boot flag is set, use this as the flag, logical
  # flag is not required as we can determine logical via
  # partition number

Other people want to create a swap partition in an extended partition.
This is a perfectly valid use-case and we set flag='swap' for those
partitions.

Therefore, in subiquity code, we cannot just check if a partition has a
flag set to 'logical'. We need to look at the partition number too.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-24 14:31:35 +02:00
Dan Bungert 649dbcc5d6
Merge pull request #1667 from dbungert/merge-discourse-docs-2023-april
Merge discourse docs 2023 april
2023-04-20 05:14:32 -06:00
Dan Bungert 0c8c38e088 documentation: merge changes from Discourse 2023-04-20 12:05:25 +01:00
Dan Bungert da678c7472 documentation: makefile helper targets 2023-04-20 12:05:25 +01:00
Olivier Gayot dfd67b5039
Merge pull request #1666 from ogayot/probert-fix-follow-on
Cosmetic fixes for asynchronous probert runs fix
2023-04-19 13:44:16 +02:00
Olivier Gayot 62dafb2c3d storage: add LP bug number to non-reg test and minor reword
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-19 09:22:08 +02:00
Olivier Gayot 69d6ef91e1 storage: add type hint for locked_probe_data
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-19 09:19:33 +02:00
Dan Bungert f43dae64e5
Merge pull request #1664 from dbungert/curtin-ntfs
snapcraft: rev for ntfsresize change
2023-04-18 19:21:13 +01:00
Dan Bungert fe2b3f2d88 snapcraft: rev for ntfsresize change 2023-04-18 19:13:54 +01:00
Dan Bungert 5586382890
Merge pull request #1660 from ogayot/no-probert-during-partitioning
storage: don't let probing discard storage config in the middle of partitioning
2023-04-18 19:09:00 +01:00
Olivier Gayot 9e4cd77d9d storage: add API test to cover probert run finishing during partitioning
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 11:01:26 +02:00
Olivier Gayot d8748c27fe storage: add unit tests to cover probe data locking
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 10:56:13 +02:00
Olivier Gayot cd25ec454f storage: don't let probing discard storage config in the middle of partitioning
If a probert run finished in the middle of partitioning, the probing
data would be loaded automatically and this would essentially discard
changes made so far by the user. This is not a desirable behavior.

Upon starting partitioning, we now prevent later probert runs from
automatically loading the probe data. Instead, the data is stored and
only loaded after after a reset.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 10:52:55 +02:00
Olivier Gayot 2f160ada88
Merge pull request #1661 from ogayot/probert-verbosity-fix
snapcraft: bump probert to pickup verbosity fix
2023-04-17 09:20:08 +02:00
Olivier Gayot 55a5ebd42b snapcraft: bump probert to pickup verbosity fix
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-17 09:08:10 +02:00
Olivier Gayot d4b9068dc5
Merge pull request #1659 from ogayot/apt-get-error-on-focal
mirror: clean environment before running apt-get
2023-04-14 16:27:48 +02:00
Olivier Gayot 3bb74546e2 mirror: clean environment before running apt-get
For mirror testing, we run apt-get update on the host. By default,
external commands run by subiquity inherit the environment variables set
by the snap (including LD_LIBRARY_PATH).

We don't ship apt-get or its dependencies in the subiquity snap, so we
want to reset LD_LIBRARY_PATH and other variables when running the
command.

Not doing so leads to the following error when running the subiquity
snap on a focal-based system:

 * apt-get: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not
   found (required by /snap/subiquity/4675/usr/lib/x86_64-linux-gnu/libsystemd.so.0)
 * apt-get: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not
   found (required by /snap/subiquity/4675/usr/lib/x86_64-linux-gnu/libsystemd.so.0)
 * apt-get: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not
   found (required by /snap/subiquity/4675/usr/lib/x86_64-linux-gnu/libsystemd.so.0)
 * apt-get: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not
   found (required by /snap/core22/current/lib/x86_64-linux-gnu/libcap.so.2)

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-14 16:13:36 +02:00
Olivier Gayot 0e957dfb9d
Merge pull request #1652 from ogayot/swap
filesystem: fix crash when editing swap partition in v2
2023-04-14 15:16:11 +02:00
Olivier Gayot aa6274c38e filesystem: fix crash when editing swap partition in v2
When we create a swap partition using v2_add_partition, the mount
parameter always has the value of None.

We do however, create a fake mountpoint object and assign it the value
of "" (empty string) so we know the swap partition is used although it's
not really mounted anywhere in the filesystem hierarchy.

Subsequently, when editing the swap partition, the mount parameter is
set to the empty string, not None.

This causes trouble in the create_mount function, which normally returns
immediately upon encountering None as the mountpoint.

Fixed by skipping the call to create_mount for a swap partition.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-14 09:51:33 +02:00
Dan Bungert b99544dfe6
Merge pull request #1657 from CarlosNihelton/seeded-cmds-wsl-490
[system_setup] Fixed paths for language-related commands
2023-04-13 14:22:38 -06:00
Carlos Nihelton 2e67005403
Fixed paths for lsc and locales commands
Subiquity snap doesn't stage language-selector-common nor locales
packages.
Attempting to run those commands require respecting the env previously
prepared by the caller, as well as fixing the commands paths
(or changing the PATH env var in the env preparation step).
Since the apt commands were already ran with a fixed path,
I'm applying the same principle in here.
Yet, for now, Jammy doesn't ship the l-s-c package seeded,
so a condition was preserved to avoid future crashes.
2023-04-13 16:11:05 -03:00