Commit Graph

5763 Commits

Author SHA1 Message Date
Dan Bungert 8533f57e9d
Merge pull request #1633 from dbungert/lunar-sizing-policy
Lunar sizing policy
2023-04-04 08:10:26 -06:00
Dan Bungert d49e37edd0 utils: matching_dicts handle key=None, typing
Co-authored-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-04-03 15:22:30 -06:00
Dan Bungert 6c2e52a028 storage: autoinstall sizing-policy
Co-authored-by: Ryan Mounce <ryan@mounce.com.au>
2023-04-03 15:22:30 -06:00
Dan Bungert 9eeb5669f2 storage: implement unscaled LV sizing for guided 2023-04-03 15:22:30 -06:00
Dan Bungert dcb8fe7920 sizing: move lv scaling logic 2023-04-03 15:22:30 -06:00
Dan Bungert e0e856475c util: add matching_dicts
Move some of this logic to a common util, so it can be used in
unittests.  The curtin version is close but expects to work on a storage
config, which is not a flat list.
2023-04-03 15:22:30 -06:00
Dan Bungert 2d74167614
Merge pull request #1632 from ogayot/lunar-2023-04-03
Lunar 2023 04 03
2023-04-03 14:46:32 -06:00
Olivier Gayot 7d3222dab9 snapcraft: update curtin to fix disk lookups with wwn extensions
Adjusted to pick commit from ubuntu/lunar branch.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
(cherry picked from commit 4c4ab16f7d)
2023-04-03 21:28:14 +02:00
Carlos Nihelton c36f4a81eb Falls back to seeded l-s-c
UDI snap ships the package, thus its data dir is part of the snap
subiquity snap does not.
This tries l-s-c data dir in the base system, outside of the snap
and gives up if the dir does not exits.

(cherry picked from commit 1105da2368)
2023-04-03 21:26:39 +02:00
Carlos Nihelton 53473feda4 Umounts in the reversed order.
Sometimes unmounting order matters. Let's take the defensive approach.

(cherry picked from commit cda5a0f250)
2023-04-03 21:26:12 +02:00
Carlos Nihelton c0e7390c61 Bind mounts
realm --install <target> requires system bind-mounts to work properly.

(cherry picked from commit 37f3348135)
2023-04-03 21:25:58 +02:00
Carlos Nihelton 9cf9a156c6 (fix) Join target instead of live session
LP: #2013079
Found during beta testing that the live session is joining, instead of
the target.

(cherry picked from commit cd6e69187a)
2023-04-03 21:24:40 +02:00
Dan Bungert 596c14056a
Merge pull request #1629 from dbungert/lunar-2023-03-31
Lunar 2023 03 31
2023-03-31 07:38:32 -06:00
Olivier Gayot 83ff5597e7 snapcraft: bump version of curtin to pickup ventoy fix
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-03-31 07:30:40 -06:00
Dan Bungert e5602ef5e5 cmd: log environment on startup 2023-03-31 07:26:30 -06:00
Dan Bungert 7da6b832d4 utils: orig_environ cleans LD_LIBRARY_PATH
LD_LIBRARY_PATH is set earlier than some of the other environment
variables like PATH or PYTHONPATH, so trying to save a clean version in
snapcraft is not viable.  Remove it here.
2023-03-31 07:26:30 -06:00
Dan Bungert e33b0d9107
Merge pull request #1626 from dbungert/lunar-2023-03-30
Lunar 2023 03 30
2023-03-30 08:27:28 -06:00
Olivier Gayot ade3416d62 filesystem: fix path property from Raid - override attribute
The following commmit:
  ce146ab28 add .path to Raid so for_client(raid-with-partitions) works

introduced a (read only) path property for Raid objects, returning an
imaginary value to make for_client work with Raid objects. Later on, the
following commit:
  8e658998e add "path" attributes to fs model objects that curtin now
            provides a path for

introduced a (read/write) path attribute to all filesystem objects. For
Raid however, the property still takes precedence over the new attribute
of the same name, so doing raid.path = x is invalid (no setter) and
results in the following exception:

AttributeError: can't set attribute 'path'

Fixed by using a @property + @setter attribute for path. The getter
returns the actual path if it exists, otherwise returns the same
imaginary value as before.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-03-30 08:20:17 -06:00
Dan Bungert d66c193e54 errorreport: all recent syslog
Some outputs only appear in the journal.
2023-03-30 07:53:09 -06:00
Dan Bungert 8759105e08 system-setup: fix disappearing ubuntu-wsl-setup
Make a copy of this file to INSTALL, as the organize step renames it,
causing it to disappear from the source directory.
2023-03-30 07:53:09 -06:00
Dan Bungert 0207d1a1e7 network: run `netplan apply` without snap env
netplan isn't staged in the snap, and the environment variables being
passed around don't help it.
2023-03-30 07:53:08 -06:00
J-P Nurmi 2764719a30 Add test source catalog for desktop variants 2023-03-30 07:53:08 -06:00
Dan Bungert 48c3b0a7c7 system-setup: restore accidentally deleted file 2023-03-30 07:53:08 -06:00
Chad Smith 481e5c85db cloud-init: preserve ephemeral cloud-init logs in target to aid in debug 2023-03-30 07:53:08 -06:00
Michael Hudson-Doyle 5311840ad2 fix "make lint" when run with the new pycodestyle/flake8 2023-03-30 07:53:08 -06:00
Dan Bungert 7fd3e607ba workflow: run both linters always
If the linter is broken in the distro in the devel series but not the
LTS, such as right now, it's interesting to allow the LTS linter to
finish.
2023-03-30 07:53:08 -06:00
Dan Bungert 49bf10022c
Merge pull request #1607 from dbungert/lunar-2023-03-21
Lunar 2023 03 21
2023-03-21 19:02:01 -06:00
Dan Bungert e1686e4e80 ad: add hack to fix non-autoinstall cases 2023-03-21 18:38:25 -06:00
Carlos Nihelton 4ff3e645c3 Fix missing scrip in snap. 2023-03-21 18:37:24 -06:00
Olivier Gayot 113427345d network: fix check of wlan network state
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-03-21 18:37:05 -06:00
Dan Bungert fe8cf93d67 api: additional checks for controllers running 2023-03-21 18:36:45 -06:00
Olivier Gayot d9c29b919d filesystem: rename done btn which overrides a meth. of the same name
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-03-21 18:36:05 -06:00
Dan Bungert 7b78cbf3cf
Merge pull request #1599 from dbungert/lunar-2023-03-17
Lunar 2023 03 17
2023-03-17 10:20:07 -06:00
Olivier Gayot 57c132729f snapcraft: upgrade to core22
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-03-17 08:42:37 -06:00
Dan Bungert 22c3987db5 api: delay most endpoints until controllers start
In LP: 2009797, an exception of this form happens:
AttributeError: 'FilesystemController' object has no attribute '_start_task'

The installer client, u-d-i, is asking for storage information ASAP
after the socket starts listening, and in this case that happened before
all controllers were started.  The sync primitive the probe is waiting
on wasn't created yet.

With one known exception, /meta/status, we really shouldn't be
responding to random API calls, and the startup sequence of the
controllers should be relatively quick (sub 1 second to be sure).
Just delay them, except for the special one.
2023-03-17 08:42:04 -06:00
Dan Bungert a960f2bbfa storage: use SingleInstanceTask improvements 2023-03-17 08:39:01 -06:00
Dan Bungert f7cb06f72a async_helpers: simplify task done check
Another version of the task-created-yet problem is the non-blocking
check to see if it is done or not.  Add a wrapper here to simplify
calling code.
2023-03-17 08:39:01 -06:00
Dan Bungert 71b71425d3 async_helpers: make SingleInstanceTask.wait safer
SingleInstanceTask has distinct steps for creation of the object, and
starting the task.  If a different coroutine is waiting on the
SingleInstanceTask, it isn't safe to directly call
SingleInstanceTask.wait() as the task may or may not have been created
yet.

Existing code usage of SingleInstanceTask is in 4 categories, with
reguards to SingleInstanceTask.wait():
1) using SingleInstanceTask without using SingleInstanceTask.wait().
   This is unchanged.
2) using SingleInstanceTask.wait without a check on task is not None.
   This may be safe now, but is fragile in the face of innocent-looking
   refactors around the SingleInstanceTask.
3) using SingleInstanceTask.wait after confirming that the task is not
   None.  This is fine but a leaky abstraction.
4) directly waiting on the SingleInstanceTask.task.  Another leaky
   abstraction, but it's solving a cancellation problem.  Leaving this
   alone.

By enhancing SingleInstanceTask.wait(), cases 2 and 3 are improved.  The
code not checking the task today is made safer, and the code checking
the task today can be simplified.
2023-03-17 08:39:01 -06:00
Dan Bungert 0b3f84f7cd make: timeout on unit tests
We aren't close to hitting this timeout in unit tests that I can see,
but some of the asyncio tests I'm writing for now risk a hang and I
don't want to get CI stuck.
2023-03-17 08:39:01 -06:00
Dan Bungert 8db8cbba09 docs: update expectations for password 2023-03-17 08:35:03 -06:00
Dan Bungert f24c76bc50
Merge pull request #1590 from dbungert/block-probing-timeouts
storage: lengthen and log probing time
2023-03-08 12:35:31 -07:00
Dan Bungert 79c38b086d
Merge pull request #1589 from dbungert/lp-2009151-manual-part-add-boot
storage/v2: endpoint for boot-plausible disks
2023-03-07 22:24:36 -07:00
Dan Bungert 9d78aed696 storage: lengthen and log probing time
A reasonably common type of bug report is hitting the block probing
timeout.  That timeout is at 15 seconds today.  Extend that to 90
seconds, and log the time it takes.

As a performance exercise, it would be good to know where that time is
being spent and why it takes 10x longer or more for some people.
2023-03-07 19:10:15 -07:00
Dan Bungert 8524f657b4 storage: s/get_guided_disks/potential_boot_disks/
This routine only answers the bootable question, it doesn't consider
other guided-required criteria such as sizing.
2023-03-07 16:07:57 -07:00
Dan Bungert 10e22a96d7 storage/v2: endpoint for boot-plausible disks 2023-03-07 15:41:47 -07:00
Carlos Nihelton 65e100eed9
Merge pull request #1581 from CarlosNihelton/ad-autoinstall-deeng-587
Active Directory - Partial autoinstall support #&1F979
2023-03-07 12:40:46 -03:00
Carlos Nihelton 3565027e02
Moves the AD autoinstall test to the API testing
Test looks much simpler now.
One small addition in the Server testing class needed to expose the
output_base dir so we can inspect the logs.
2023-03-07 08:37:38 -03:00
Dan Bungert cfeb3fe163
Merge pull request #1584 from ogayot/mirror-testing-apt-warning
apt: force downloads to be unsandboxed when running mirror testing
2023-03-06 12:00:11 -07:00
Carlos Nihelton 82cb7bd33e
Merge pull request #1585 from CarlosNihelton/ad-split-domain-check
Active Directory - Split the domain validation endpoint in two
2023-03-06 09:00:56 -03:00
Carlos Nihelton 2625c233b0
Updates the integration to set variant and update AD
I couldn't find a way to set the variant as desktop in the autoinstall
file or command line.
Thus some shell scripting hackery runs curl to set the variant.
Also, GET /active_directory must provide user and domain, thus updating
the password and curl'ing again configures the AD controller.
2023-03-05 15:03:43 -03:00