Commit Graph

6350 Commits

Author SHA1 Message Date
Olivier Gayot 1e88591003
Merge pull request #1684 from ogayot/pep8-removed-from-mantic
apt-deps: drop pep8 which is unused and does not exist anymore in mantic
2023-05-24 14:51:32 +02:00
Olivier Gayot e1863d9fb6 apt-deps: drop pep8 which is unused and does not exist anymore in mantic
The pep8 binary package was a transitional package. It would install a
symlink /usr/bin/pep8 -> pycodestyle

pep8 does not exist anymore in mantic.

pycodestyle would be the right binary package going forward. That said,
we stopped using the pep8 command back in 2018 with this commit:

  af9427561 Make tox and Makefile more similar.

Let's drop the dependency instead.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-24 11:03:06 +02:00
Michael Hudson-Doyle 2c3cb2491d
Merge pull request #1680 from mwhudson/simplify-curtin_install
simplify InstallController.curtin_install a bit
2023-05-24 12:57:42 +12:00
Michael Hudson-Doyle 7417f812d5 simplify InstallController.curtin_install a bit
curtin_install before this assembled a list of steps and then executed
them one by one. As far as I can tell/remember (I was at least partly
responsible for this) there is no reason to do it this way: it feels
more Pythonic to just execute the steps!

This required removing some indirection in how the config for each step
was constructed, which I think also makes things a bit clearer.
2023-05-23 12:53:28 +12:00
Olivier Gayot e58cf53f26
Merge pull request #1677 from ogayot/ci-diff-mypy
workflow: produce a diff of the mypy errors before/after a PR
2023-05-22 10:41:42 +02:00
Olivier Gayot ac36ce5ddd filesystem: fix use of type hint for dictionary
For a dictionary, the type hint should be dict[key_type, value_type] as
opposed to dict[key_type: value_type].

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-22 10:29:20 +02:00
Olivier Gayot 5bf3f6a818 workflow: produce a diff of the mypy errors before/after a PR
A new job now runs on PRs. It will automatically run mypy on the target
branch and on the source branch. It will generate a diff of the errors,
showing the new ones and showing the ones that have been fixed.

It will also show a summary with the number of errors before/after the
PR.

Because we have so many false positive, it makes no sense to mark this
job red. So we always make it green (unless mypy can't run).

As for now, it's up to developers to go check if any new error is
introduced.

If a line that used to produce a mypy error gets moved, it will be
reported as:

 * a fixed error
 * a newly introduced error

This is suboptimal and ideally we should have a way to detect moves.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-22 10:29:20 +02:00
Olivier Gayot 058a106a9f
Merge pull request #1679 from ogayot/latest-curtin-probert
snapcraft: use latest curtin and probert
2023-05-17 08:34:23 +02:00
Dan Bungert 2b25e8f36a
Merge pull request #1675 from canonical/canary-23.04
Canary 23.04
2023-05-16 16:39:06 -06:00
Olivier Gayot 39623077d7 snapcraft: use latest curtin and probert
This will pull the fixes for the utf-8 partition names.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-16 18:00:03 +02:00
Ryan Mounce 7dc9668862 Support matching disks on DEVPATH or ID_PATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 11:25:32 +09:30
Ryan Mounce 5ba82a3974 Fix doc, disk path matcher does not consider udev DEVPATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 10:54:56 +09:30
Dan Bungert b49031c5fb mounter: handle destination not existing
bind_mount_tree really wants to mount to places that don't exist yet,
and to unmount them later.  Simply creating these files or directories
gets weird.  Manage this process in the Mounter, so that any of these
temporary mountpoints get cleaned up on unmount.
2023-05-15 16:50:27 -06:00
Dan Bungert 70549c4c70 filesystem: no bind_mount in dryrun 2023-05-15 16:50:27 -06:00
Dan Bungert d69bc0dd77 filesystem: select gap by offset
Given disk layout like so
[p1 g1 p2 p3 g2]
The code attempting to create p4 used the largest_gap() method to
attempt to select g2. If the disk was small enough, below 28GiB with
current numbers, g1 would actually be larger, but the sizing math was
done with the offset that was correct for g2, so p4 would be created
a very incorrect size (-800MiB or so).

Just select gap by offset - we have the info needed.
2023-05-15 16:50:27 -06:00
Dan Bungert e2d55f50a2 apitest: example has no bios compat part 2023-05-15 16:50:27 -06:00
Dan Bungert c9180f09c1 examples: improve enhanced-secureboot 2023-05-15 16:50:27 -06:00
Dan Bungert 925d3e0a40 filesystem: fix snapd label lookup 2023-05-15 16:50:27 -06:00
Dan Bungert 0d4119070c filesystem: also mount snaps dir
We need these for valid responses from snapd v2/systems.
2023-05-15 16:50:27 -06:00
Dan Bungert 1bcf66c89b filesystem: give Source.get_handler variation name
In order to lookup the snapd system correctly, we need the correct
variant information, and it works best by name.
2023-05-15 16:50:27 -06:00
Dan Bungert 03c48daded test/api: add a simple coreboot testcase 2023-05-15 16:50:27 -06:00
Dan Bungert 451c4d7b74 storage/v1: handle variations for guided_GET
Purposefully simple as this is not believed to be a real use case to
have multiple variations.
2023-05-15 16:50:27 -06:00
Dan Bungert 1cc39ab7ee examples: fake enhanced-secureboot-desktop 2023-05-15 16:50:27 -06:00
Michael Hudson-Doyle b5ba12bf2c filesystem: enumerate and choose variation 2023-05-15 16:50:27 -06:00
Dan Bungert ebc7e39ea8 source: test expected model characteristics 2023-05-15 16:49:15 -06:00
Michael Hudson-Doyle 5105eb6c82 parse variations out of the source catalog
This parses variations but nothing iterates over them or selects a
non-arbitrary one yet.
2023-05-15 16:49:15 -06:00
Dan Bungert 02bfe89c93
Merge pull request #1676 from dbungert/workflow-mantic
workflow: +mantic
2023-05-12 07:53:51 -06:00
Dan Bungert f21dac28f2 workflow: +mantic 2023-05-12 07:44:16 -06:00
Olivier Gayot cf1e13ac12
Merge pull request #1671 from ogayot/no-reorder-uefi
install: request curthooks not to reorder UEFI boot entries
2023-04-28 10:08:41 +02:00
Olivier Gayot a270019387 prober: turn on parallelism for probert storage
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-27 11:09:23 +02:00
Olivier Gayot 782481b357
Merge pull request #1670 from ogayot/asyncio-probert
prober: move to asyncio for probert storage
2023-04-27 11:08:33 +02:00
Olivier Gayot 0bf2d9d38d install: request curthooks not to reorder UEFI boot entries
By default, curthooks will automatically reorder the UEFI boot entries
so that the media used during installation is set as the first boot
entry.

This is something that MAAS wants because MAAS typically boots over the
network.

This is not typically something we would want for a server or desktop
installer though. Most of the time, the media is a removable device.
Ideally, we would want the new installed grub to be the first boot
entry.

Furthermore, on some UEFI implementations, the BootCurrent does not
correspond to a Boot#### entry when booting from a removable media. When
this happens, curtin fails at making the media the first boot entry,
resulting in the following error:

  Invalid BootOrder order entry valueXXXX
                                       ^
  efibootmgr: entry XXXX does not exist

Fixed by disabling the UEFI boot entries reordering when invoking
curthooks.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-26 13:42:07 +02:00
Olivier Gayot 60f2b506db prober: move to asyncio for probert storage
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-25 15:12:19 +02:00
Olivier Gayot 1607a8a95b
Merge pull request #1669 from ogayot/logical+swap
filesystem: fix logical+swap or logical+boot partition not considered logical
2023-04-25 11:27:30 +02:00
Olivier Gayot 81acb6eb9f snapcraft: bump curtin to handled logical swap partitions
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-25 10:56:58 +02:00
Olivier Gayot 36161c0040 filesystem: fix logical+swap or logical+boot partition not considered logical
Some people who installed a previous version of Ubuntu on a DOS
partition table ended up with a logical, ESP partition (which is marked
bootable).

This is an unsupported use-case. An ESP partition should always be a
primary partition. Ideally, an ESP partition should be on a GPT.

However, curtin's storage_config makes it clear that expecting the flag
of a logical partition to be set to 'logical' is a bad idea.

  # if the boot flag is set, use this as the flag, logical
  # flag is not required as we can determine logical via
  # partition number

Other people want to create a swap partition in an extended partition.
This is a perfectly valid use-case and we set flag='swap' for those
partitions.

Therefore, in subiquity code, we cannot just check if a partition has a
flag set to 'logical'. We need to look at the partition number too.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-24 14:31:35 +02:00
Dan Bungert 649dbcc5d6
Merge pull request #1667 from dbungert/merge-discourse-docs-2023-april
Merge discourse docs 2023 april
2023-04-20 05:14:32 -06:00
Dan Bungert 0c8c38e088 documentation: merge changes from Discourse 2023-04-20 12:05:25 +01:00
Dan Bungert da678c7472 documentation: makefile helper targets 2023-04-20 12:05:25 +01:00
Olivier Gayot dfd67b5039
Merge pull request #1666 from ogayot/probert-fix-follow-on
Cosmetic fixes for asynchronous probert runs fix
2023-04-19 13:44:16 +02:00
Olivier Gayot 62dafb2c3d storage: add LP bug number to non-reg test and minor reword
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-19 09:22:08 +02:00
Olivier Gayot 69d6ef91e1 storage: add type hint for locked_probe_data
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-19 09:19:33 +02:00
Dan Bungert f43dae64e5
Merge pull request #1664 from dbungert/curtin-ntfs
snapcraft: rev for ntfsresize change
2023-04-18 19:21:13 +01:00
Dan Bungert fe2b3f2d88 snapcraft: rev for ntfsresize change 2023-04-18 19:13:54 +01:00
Dan Bungert 5586382890
Merge pull request #1660 from ogayot/no-probert-during-partitioning
storage: don't let probing discard storage config in the middle of partitioning
2023-04-18 19:09:00 +01:00
Olivier Gayot 9e4cd77d9d storage: add API test to cover probert run finishing during partitioning
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 11:01:26 +02:00
Olivier Gayot d8748c27fe storage: add unit tests to cover probe data locking
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 10:56:13 +02:00
Olivier Gayot cd25ec454f storage: don't let probing discard storage config in the middle of partitioning
If a probert run finished in the middle of partitioning, the probing
data would be loaded automatically and this would essentially discard
changes made so far by the user. This is not a desirable behavior.

Upon starting partitioning, we now prevent later probert runs from
automatically loading the probe data. Instead, the data is stored and
only loaded after after a reset.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-18 10:52:55 +02:00
Olivier Gayot 2f160ada88
Merge pull request #1661 from ogayot/probert-verbosity-fix
snapcraft: bump probert to pickup verbosity fix
2023-04-17 09:20:08 +02:00
Olivier Gayot 55a5ebd42b snapcraft: bump probert to pickup verbosity fix
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-17 09:08:10 +02:00