Commit Graph

5380 Commits

Author SHA1 Message Date
Dan Bungert 4b66178ada model/fs: test swap size 0 2022-10-25 18:39:29 -06:00
Dan Bungert bccdff2792
Merge pull request #1459 from mfoliveira/lp1927103
filesystem: fix curtin config to disable swapfile
2022-10-25 18:38:10 -06:00
Olivier Gayot b01f92874b
Merge pull request #1464 from ogayot/autoinstall-doc-fix-discourse-links
doc: fix missing leading / before some discourse links
2022-10-25 17:35:06 +02:00
Olivier Gayot 5acff6d7d0 doc: fix missing leading / before some discourse links
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-25 16:08:14 +02:00
Olivier Gayot 51db154cb8
Merge pull request #1463 from ogayot/autoinstall-doc-update
Autoinstall doc update
2022-10-25 15:45:20 +02:00
Olivier Gayot 0bab5dd61e doc: double the memory allocated to kvm in quickstart guide
Although 1GiB of RAM seems enough to install a server ISO and boot it
after, if feels rather on the low end nowadays.
The quickstart for s390x already uses 2GiB so it feels right to update
the generic guide to use 2GiB as well.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-25 09:32:44 +02:00
Olivier Gayot 952ca2f8a3 doc: update quickstart guide for 22.10 release
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-25 09:31:44 +02:00
Olivier Gayot 218eb36ec3 doc: fix default shutdown action which is to reboot
If the autoinstall configuration does not declare a shutdown section,
the default action is not to do nothing ; but to reboot.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-25 09:30:41 +02:00
Olivier Gayot 5d3d83dcbb doc: reword autoinstall introduction that still mentions 20.04
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-25 09:30:23 +02:00
Dan Bungert 0bccbbeed1
Merge pull request #1461 from dbungert/clarify-cloud-init
doc: clarify cloud-init interactions
2022-10-22 10:39:35 -06:00
Dan Bungert 4980206804
doc: improve cloud-init interaction wording
Co-authored-by: Chad Smith <chad.smith@canonical.com>
2022-10-22 10:29:06 -06:00
Olivier Gayot 277c7c9a29
Merge pull request #1460 from ogayot/autoinstall-reference-update
autoinstall reference documentation update
2022-10-21 10:03:06 +02:00
Olivier Gayot 0c4e6872a2 doc: document "kernel" section in autonistall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-21 09:50:33 +02:00
Olivier Gayot 37b1db83bd doc: document "shutdown" section in autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-21 09:50:33 +02:00
Olivier Gayot f581afb855 doc: document "updates" section in autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-21 09:50:33 +02:00
Olivier Gayot 3aa20a2b89 doc: document "timezone" section in autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-21 09:50:33 +02:00
Dan Bungert 01f2dee8ce doc: clarify cloud-init interactions 2022-10-20 17:23:18 -06:00
Olivier Gayot de389c8fc5 doc: document "ubuntu-pro" section in autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 16:10:27 +02:00
Olivier Gayot 102f2fa584 doc: document "drivers" section to autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 16:10:27 +02:00
Olivier Gayot 8951cf0e04 doc: document "source" section in autoinstall-reference
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 16:10:20 +02:00
Olivier Gayot ebe22945da doc: consistency fixes for autoinstall doc
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 15:48:15 +02:00
Olivier Gayot cf657c1161 doc: import latest JSON schema in autoinstall doc
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 15:47:49 +02:00
Olivier Gayot ba40c4c6eb doc: escape asterisks in markdown
Vim seems lost and breaks syntax highlighting when it is dealing with
unescaped asterisks in the markdown content. Let's escape the asterisks
to make it happy.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 15:46:52 +02:00
Olivier Gayot 75e727af4f
Merge pull request #1458 from ogayot/autoinstall-doc-import
Import autoinstall docs from discourse and add ability to run link substitutions before exporting
2022-10-20 11:09:08 +02:00
Olivier Gayot c811cc72f8 doc: implement link substitution mechanism
Running 'make' or 'make html' will now convert the links to .md files
into links to .html files so that the documentation can be browsed
locally as HTML files. This will also produce the HTML output under
html/.

Running 'make discourse' will convert the links to .md files into links
to the existing discourse pages. The output MD files will be placed
under discourse/

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 09:50:10 +02:00
Olivier Gayot 14ce18f8b2 doc: use better links in s390x page
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 09:50:09 +02:00
Olivier Gayot 85e1a858a8 doc: build with pandoc
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 09:50:09 +02:00
Olivier Gayot 747cfc50f6 doc: make links to anchors work with pandoc
For some reason, pandoc seems to struggle to create anchor links if
there is no space between the <a> tag and the title. Add a newline to
make it happy.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 09:50:09 +02:00
Olivier Gayot a4f70ce464 doc: import autoinstall markdown documents from discourse
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-20 09:50:09 +02:00
Mauricio Faria de Oliveira 286b0312c5 filesystem: fix curtin config to disable swapfile
If subiquity detects it should not add a swapfile (e.g., swap partition
present, or root filesystem on btrfs), it renders this curtin config:

  swap:
    swap: 0

But that should be:

  swap:
    size: 0

Per the curtin docs [1]:

  swap
  size: <Size string>
  Configure the exact size of the swapfile. Setting size to 0 will disable swap.

[1] https://curtin.readthedocs.io/en/latest/topics/config.html#swap

Tests: (build snap/modify ISO as in README.md)

- before:

  $ cat /proc/swaps
  Filename	...
  /swap.img	...
  /dev/vda3	...

  $ sudo grep -A1 ^swap: \
    /var/log/installer/curtin-install/subiquity-partitioning.conf
  swap:
    swap: 0

- after:

  $ cat /proc/swaps
  Filename	...
  /dev/vda3	...

  $ sudo grep -A1 ^swap: \
    /var/log/installer/curtin-install/subiquity-partitioning.conf
  swap:
    size: 0

LP: #1927103

Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
2022-10-19 18:55:27 -03:00
Olivier Gayot 43d1206855
Merge pull request #1456 from ogayot/LP1993257
storage: make GET /storage return the last successful probe result when requested
2022-10-18 17:40:55 +02:00
Olivier Gayot 0a954645d6 storage: make GET /storage return the last successful probe when requested
In the following patch:

  4c59e1ee never return a PROBING response from guided_POST

we fixed a UI crash occurring when a storage probing operation was
ongoing while the user would enter the partitioning screen with "Use
an entire disk".

To address the issue, we made POST /storage/guided return the last
(cached) successful probe result rather than returning a potential
PROBING status result.

That being said, when "Custom Storage Layout" is selected, we are using
a different HTTP call: GET /storage which can also return a PROBING
status result. So our fix was only partial.

This patch adds a "use_cached_result" parameter (which defaults to
false) to GET /storage. This makes the call return the last successful
probe result.

When the client selects "Custom Storage Layout" we now explicitly
request a cached result so that we don't have to deal with a potential
PROBING status response.

Ideally, we should either:

 * make the client aware that a PROBING status can be returned and act
   accordingly.
 * pass the wait=true argument to GET /storage or POST /storage/guided

But doing so has other implications and the kinetic release is imminent.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-18 15:27:51 +02:00
Dan Bungert 87a9876ef7
Merge pull request #1453 from ogayot/LP#1992977
drivers: make overlay unmount error not fatal
2022-10-17 09:22:35 -06:00
Olivier Gayot 6954b309e6 drivers: do not consider overlay cleanup error fatal
For some reason that we have not yet determined, the overlay that we use
for the drivers code sometimes fails to unmount with EBUSY.

When it happens, the _list_drivers task would fail, making subsequent
GET calls to /drivers?wait=true fail as well and produce an error
report.

We now mark failures in the overlay unmount operation as non fatal.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-17 12:42:30 +02:00
Olivier Gayot aae52dd6ba apt: introduce OverlayCleanupError in overlay context manager
The apt.overlay construct unmounts the overlay at the end of the with
body. To do so, it leans on subprocess.run(["umount", ...], check=True).

If we want to catch the unmount error, we need to do:

  try:
      with apt.overlay():
          # do stuff
  except subprocess.CalledProcessError:
     # handle the unmount exception

However, we do not want the subprocess exceptions that happen in
the with body to be caught in the same except block.

If the umount call fails, we now wrap the exception in a
OverlayCleanupError ; therefore we can make the distinction between an
unmount error and a subprocess error that would happen in the body.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-17 12:31:31 +02:00
Dan Bungert 7387625e8d
Merge pull request #1452 from dbungert/lp-1992990-flash-kernel
curtin: rev for flash-kernel fix
2022-10-14 16:20:49 -06:00
Dan Bungert 1c59596913 curtin: rev for flash-kernel fix
LP: #1992990
2022-10-14 16:10:36 -06:00
Dan Bungert 36705829ff
Merge pull request #1450 from dbungert/guided-lvm-bad-offset
filesystem: ensure bootfs at aligned size
2022-10-12 08:42:52 -06:00
Dan Bungert 62ed92505a filesystem: ensure bootfs at aligned size
Certain disk sizes could trigger unaligned bootfs size.  This can result
in the next partition overlapping into the bootfs partition.

LP: #1992541
2022-10-11 20:45:57 -06:00
Dan Bungert c2e1daa35c
Merge pull request #1449 from dbungert/autoinstall-umount
apt: workaround umount failure
2022-10-11 18:54:13 -06:00
Dan Bungert 24c28a89c2 apt: workaround umount failure
When the apt overlay fails to umount, this directory removal will also
fail due to the mount still being present.

LP: #1992531
2022-10-11 17:07:26 -06:00
Michael Hudson-Doyle c1dd281040
Merge pull request #1444 from mwhudson/device-paths
track fs model objects device paths
2022-10-11 13:09:41 +13:00
Olivier Gayot f0e5c19ee7 loop: replace use of asyncio.get_event_loop
The behavior of asyncio.get_event_loop() will change in a future Python
version. It is deprecated starting Python 3.10.

The functions that we can use instead are:

 * asyncio.new_event_loop() - which creates a new event loop
 * asyncio.get_running_loop() - which returns the event loop only if it
   is already running

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-07 18:45:11 +02:00
Olivier Gayot 1fbef3354c client: move aiohttp connector creation in coroutine
Creating a aiohttp connector when no event loop is running is not
considered safe. Under the hood, aiohttp invokes
asyncio.get_event_loop() which:

 1. currently creates a new event loop if one does not already exist
 2. will stop creating a new event loop in a future Python release
 3. is deprecated starting Python 3.10

We want to make sure Subiquity creates the event loop itself, so we now
create the aiohttp connector in a coroutine.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-07 18:36:39 +02:00
Olivier Gayot d27fec6480 filesystem: split logical and primary using more_itertools.partition
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-07 09:57:32 +02:00
Olivier Gayot 01060fc14b filesystem: add complex use-case as a test for offsets and sizes
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-06 16:08:01 +02:00
Olivier Gayot b831d7ccae filesystem: fix use of negative size in gaps calculations
When size: -1 is specified on a given partition, we attempt to determine
the its right size by looping through the partitions and computing a
list of available gaps.

That said, when looping through the partitions, we also include the one
which has an incomplete (i.e. = -1) size. This makes the computation
rely on a negative value and this often leads to incorrect result.

Fixed by excluding the current partition when determining its
appropriate size.

https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1991929

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-06 16:08:01 +02:00
Olivier Gayot 78ac4408d5 filesystem: guess missing offsets in autoinstall
When storage version 2 is used, partitions require an offset. When
loading partitions via an autoinstall file, we now make sure to assign
offsets automatically if the user did not specify offsets.

https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1991413

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-06 16:08:01 +02:00
Olivier Gayot 06f5e24904 filesystem: fix implem of available_for_partitions
The available_for_partitions function wrongly assumed that the
underlying disk had a GPT partition table. Updated the implementation to
use the alignment data as expected.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-06 15:52:39 +02:00
Olivier Gayot dd2553723f filesystem: mark ptable msdos in tests at the right place
In the new tests recently added to test_filesystem.py, the initial disk
object was marked having a msdos partition table.

This is fine for computing offsets manually. However, what really needs
to be done is to mark the partition table as msdos in the YAML itself.
Otherwise, the disk is reformatted as GPT.

This makes gap functions use the right alignment data when computing the
list of gaps available.

Sadly it breaks the tests because of other issues, so I added FIXME tags
in the broken tests.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-10-06 15:52:39 +02:00