Commit Graph

5981 Commits

Author SHA1 Message Date
Dan Bungert 3ac413e88b
Merge pull request #1689 from dbungert/zfs-autoinstall
zfs autoinstall
2023-06-08 12:24:38 -06:00
Dan Bungert 0e1da11c36 filesystem: allow zpool to satisfy mount 2023-06-08 12:06:45 -06:00
Dan Bungert 895f6a6384 filesystem: handle zpool export 2023-06-08 12:06:45 -06:00
Dan Bungert 7e74184fdc filesystem: add zfs/zpool fsobjs 2023-06-08 12:06:45 -06:00
Dan Bungert 50c7ff3d36 shutdown: add pre-shutdown pubsub channel 2023-06-08 12:06:45 -06:00
Dan Bungert c0741ab715 examples: autoinstalls showing zfs root
Add example autoinstalls.  Include several necessary steps for
everything to go well that will be moved out of the autoinstall in
following PRs.
2023-06-08 12:06:33 -06:00
Olivier Gayot aec441d9e9 ubuntu-drivers: by default in dry-run, consider that drivers are available
We only search for drivers if search_drivers is enabled in the source
screen. When testing in dry-run mode, we need to check the
search_drivers checkbox AND remember to pass the has-drivers debug flag.
I think it makes sense to consider drivers available by default in
dry-run mode.

This also means that OEM meta-packages are available by default (and
_should_ be installed automatically) in dry-run now.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-06-08 15:34:08 +02:00
Olivier Gayot e657b7e0e7 ubuntu-drivers: add method to list OEM meta-packages
The ubuntu_drivers.list_drivers() method currently returns names of
drivers + name of OEM meta-packages.

We now define a new method to be used for the OEM meta-packages only.

For now, the has-drivers, no-drivers, run-drivers debug flags affect the
presence of the OEM meta-packages as well. Going forward, we should
define dry-run directives controlling drivers & OEM independently.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-06-08 15:33:55 +02:00
Michael Hudson-Doyle 0e043679f4 stop passing source to every curtin step 2023-06-08 13:10:16 +12:00
Michael Hudson-Doyle c85ad513ec handle spaces in curtin step names better 2023-06-08 12:43:21 +12:00
Michael Hudson-Doyle 9b7a395347 skip symlinks when populating rp 2023-06-08 12:43:21 +12:00
Michael Hudson-Doyle 16ff550a2c update curtin 2023-06-08 12:43:18 +12:00
Ondrej Kubik f55f2fc94a bin/console-conf-wrapper: prefer use of snapd/modeenv when determining snapd mode
Signed-off-by: Ondrej Kubik <ondrej.kubik@canonical.com>
2023-06-08 00:19:11 +01:00
Michael Hudson-Doyle e59e4e7c7a copy /cdrom to the reset partition, if configured 2023-06-08 10:57:14 +12:00
Michael Hudson-Doyle c9be41c227 add an async contextmanager to temporarily mount a device 2023-06-08 10:57:14 +12:00
Michael Hudson-Doyle 285a44d54f have Mounter.mount create a tmpdir mountpoint if none is passed 2023-06-08 10:57:14 +12:00
Michael Hudson-Doyle 0dca036101 allow an autoinstall file to request creation of a recovery partition 2023-06-08 10:57:11 +12:00
Olivier Gayot 919ca5ebd4
Merge pull request #1688 from ogayot/fix-list-drivers
drivers: fix empty list of drivers shown when it has not yet been retrieved
2023-06-07 17:39:13 +02:00
Olivier Gayot 1f9d824e46 drivers: fix empty list of drivers shown when it has not yet been retrieved
The /drivers GET endpoint is supposed to have {"drivers": null} when the
list of drivers has not (yes) been retrieved.

However, this behavior got broken by the following patch:

 66c82b322 drivers: query again list of drivers if the source variant changes

.. which resets the drivers variable to an empty list when querying the
list of drivers (this happens automatically when the user submits on the
source screen).

Therefore, instead of getting {"drivers": null}, we would get
{"drivers": []} which has a different meaning.

The patch also incorrectly declared the drivers variable in the
initializer (not taking None into account). The variable was actually
already declared as a class variable with the right type.

Fixed by resetting the variable to None when re-querying the list of
drivers and dropped the class variable in favor of the instance variable
(with the right type this time).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-06-07 17:05:16 +02:00
Sally Makin 8b16c25363 Add README.md and rename folder 2023-06-07 10:14:31 +01:00
Dan Bungert 607eb80bdc
Merge pull request #1682 from dbungert/interactive-sections-api
meta: add interactive_sections endpoint
2023-06-01 12:56:26 -06:00
Dan Bungert f76a648fc2 meta: interactive-sections API test 2023-06-01 12:27:42 -06:00
Olivier Gayot 5924509127
Merge pull request #1683 from ogayot/mypy-diff-add-local-script
scripts: add script to run mypy and compare with output in another revision
2023-06-01 09:57:18 +02:00
Olivier Gayot 0db2da32ec
Merge pull request #1668 from ogayot/parallel-probert
Run probert probes in parallel - resulting in faster block probing
2023-06-01 09:24:59 +02:00
Olivier Gayot a13b2d6f46 mirror: add a bunch of assert statements to make mypy happy
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-06-01 09:24:05 +02:00
Olivier Gayot 3d1ea39e20 scripts: add script to run mypy and compare with output in another revision
The script can be invoked in different ways:

Run mypy in the current working directory and display the output:

 $ scripts/run-mypy.py

Run mypy in a clean copy of the HEAD revision:

 $ scripts/run-mypy.py --checkout-head

Run mypy in the current working directory and compare the result with
another revision (here main):

 $ scripts/run-mypy.py --diff-against main

Run mypy in a clean copy of the HEAD revision and compare with another
revision (here main):

 $ scripts/run-mypy.py --diff-against main --checkout-head

The produced result might be slightly different from what the CI does
because it also clones checks out curtin and probert (at the right
revision). This is something we might want to do in the CI as well.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-06-01 09:24:05 +02:00
Dan Bungert 266fb4a65b meta: handle None in interactive-sections API 2023-05-31 12:59:47 -06:00
Dan Bungert 74e19903c7 meta: expand interactive-sections: ['*'] in API 2023-05-30 17:34:08 -06:00
Dan Bungert e4c94780b6 meta: add interactive_sections endpoint
A feature request for Desktop was to add the ability to retrieve, all at
once, the raw interactive-sections value from autoinstall.
2023-05-30 17:34:08 -06:00
Dan Bungert 7e41eee464
Merge pull request #1580 from acres-com-au/fix-storage-path-match
Fix storage 'path' matcher doc & add new matchers to do what 'path' claimed to
2023-05-30 16:19:22 -06:00
Sally Makin e90d85e78c docs: add files for RTD setup
- Add docs directory to keep files together
- Add files necessary for RTD
- Add style sheets for "Ubuntu" style
- Add Diataxis outline and framework
2023-05-26 16:58:16 +01:00
Olivier Gayot 1e88591003
Merge pull request #1684 from ogayot/pep8-removed-from-mantic
apt-deps: drop pep8 which is unused and does not exist anymore in mantic
2023-05-24 14:51:32 +02:00
Olivier Gayot e1863d9fb6 apt-deps: drop pep8 which is unused and does not exist anymore in mantic
The pep8 binary package was a transitional package. It would install a
symlink /usr/bin/pep8 -> pycodestyle

pep8 does not exist anymore in mantic.

pycodestyle would be the right binary package going forward. That said,
we stopped using the pep8 command back in 2018 with this commit:

  af9427561 Make tox and Makefile more similar.

Let's drop the dependency instead.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-24 11:03:06 +02:00
Michael Hudson-Doyle 2c3cb2491d
Merge pull request #1680 from mwhudson/simplify-curtin_install
simplify InstallController.curtin_install a bit
2023-05-24 12:57:42 +12:00
Michael Hudson-Doyle 7417f812d5 simplify InstallController.curtin_install a bit
curtin_install before this assembled a list of steps and then executed
them one by one. As far as I can tell/remember (I was at least partly
responsible for this) there is no reason to do it this way: it feels
more Pythonic to just execute the steps!

This required removing some indirection in how the config for each step
was constructed, which I think also makes things a bit clearer.
2023-05-23 12:53:28 +12:00
Olivier Gayot e58cf53f26
Merge pull request #1677 from ogayot/ci-diff-mypy
workflow: produce a diff of the mypy errors before/after a PR
2023-05-22 10:41:42 +02:00
Olivier Gayot ac36ce5ddd filesystem: fix use of type hint for dictionary
For a dictionary, the type hint should be dict[key_type, value_type] as
opposed to dict[key_type: value_type].

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-22 10:29:20 +02:00
Olivier Gayot 5bf3f6a818 workflow: produce a diff of the mypy errors before/after a PR
A new job now runs on PRs. It will automatically run mypy on the target
branch and on the source branch. It will generate a diff of the errors,
showing the new ones and showing the ones that have been fixed.

It will also show a summary with the number of errors before/after the
PR.

Because we have so many false positive, it makes no sense to mark this
job red. So we always make it green (unless mypy can't run).

As for now, it's up to developers to go check if any new error is
introduced.

If a line that used to produce a mypy error gets moved, it will be
reported as:

 * a fixed error
 * a newly introduced error

This is suboptimal and ideally we should have a way to detect moves.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-22 10:29:20 +02:00
Olivier Gayot 058a106a9f
Merge pull request #1679 from ogayot/latest-curtin-probert
snapcraft: use latest curtin and probert
2023-05-17 08:34:23 +02:00
Dan Bungert 2b25e8f36a
Merge pull request #1675 from canonical/canary-23.04
Canary 23.04
2023-05-16 16:39:06 -06:00
Olivier Gayot 39623077d7 snapcraft: use latest curtin and probert
This will pull the fixes for the utf-8 partition names.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-05-16 18:00:03 +02:00
Ryan Mounce 7dc9668862 Support matching disks on DEVPATH or ID_PATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 11:25:32 +09:30
Ryan Mounce 5ba82a3974 Fix doc, disk path matcher does not consider udev DEVPATH
Related to:
https://bugs.launchpad.net/subiquity/+bug/1952228
https://github.com/canonical/subiquity/pull/1389
2023-05-16 10:54:56 +09:30
Dan Bungert b49031c5fb mounter: handle destination not existing
bind_mount_tree really wants to mount to places that don't exist yet,
and to unmount them later.  Simply creating these files or directories
gets weird.  Manage this process in the Mounter, so that any of these
temporary mountpoints get cleaned up on unmount.
2023-05-15 16:50:27 -06:00
Dan Bungert 70549c4c70 filesystem: no bind_mount in dryrun 2023-05-15 16:50:27 -06:00
Dan Bungert d69bc0dd77 filesystem: select gap by offset
Given disk layout like so
[p1 g1 p2 p3 g2]
The code attempting to create p4 used the largest_gap() method to
attempt to select g2. If the disk was small enough, below 28GiB with
current numbers, g1 would actually be larger, but the sizing math was
done with the offset that was correct for g2, so p4 would be created
a very incorrect size (-800MiB or so).

Just select gap by offset - we have the info needed.
2023-05-15 16:50:27 -06:00
Dan Bungert e2d55f50a2 apitest: example has no bios compat part 2023-05-15 16:50:27 -06:00
Dan Bungert c9180f09c1 examples: improve enhanced-secureboot 2023-05-15 16:50:27 -06:00
Dan Bungert 925d3e0a40 filesystem: fix snapd label lookup 2023-05-15 16:50:27 -06:00
Dan Bungert 0d4119070c filesystem: also mount snaps dir
We need these for valid responses from snapd v2/systems.
2023-05-15 16:50:27 -06:00