Commit Graph

5038 Commits

Author SHA1 Message Date
Olivier Gayot 2e6fffe02b
Merge pull request #1179 from ogayot/FR-2025
Server-side implementation of third-party drivers
2022-03-01 10:21:59 +01:00
Dan Bungert e35cc87a69
Merge pull request #1193 from dbungert/the-os-prober-saga
os-prober: installer newer version
2022-02-28 16:20:55 -07:00
Dan Bungert 1f8569fcf1
Merge pull request #1176 from dbungert/FR-1182-dual-boot
storage/v2: expose Gap object
2022-02-28 16:20:43 -07:00
Dan Bungert b8db34bb21 test/api: add test for partition to existing table 2022-02-28 16:05:12 -07:00
Dan Bungert 838b83b0b5 test/api: add test for multi-gap case
This test should work once multi-gap is ready, but skip for now since
the gap is reported as one large one.
2022-02-28 16:05:12 -07:00
Dan Bungert e1911cba93 test/api: add notes about debug usage 2022-02-28 16:05:12 -07:00
Dan Bungert e64b3781f9 ui: ignore Gap object in Partition list summary 2022-02-28 16:05:12 -07:00
Dan Bungert e5770f029a storage/v2: start showing Gaps on the v2 api 2022-02-28 16:05:12 -07:00
Dan Bungert 16d0325cf8 storage/v2: reject partition add with no space 2022-02-28 16:05:12 -07:00
Dan Bungert c68a32fe42 examples: add disk with free space at end 2022-02-28 16:01:49 -07:00
Dan Bungert 5124762de4 probert: rev for os-prober, mdadm 2022-02-28 15:56:04 -07:00
Dan Bungert 9179777aa2 os-prober: use snapified source
Use my branch of os-prober which adds env variables for finding the lib
& share directories.  And set those env variables.
2022-02-28 15:56:04 -07:00
Dan Bungert 3b6f6024ae os-prober: build from a git
Build os-prober from a git repo instead of staging the focal version.
Point to my mirror of os-prober as salsa is currently down.
2022-02-28 15:00:25 -07:00
Dan Bungert 1ebdc91c7c
Merge pull request #1192 from dbungert/lp-1962025-update-answers
refresh: cancel the task waiting on update
2022-02-28 14:28:54 -07:00
maces 7c7b6da7f6 fixing swap path + support partition size units
- The 20.04 autoinstaller expects a swap path of 'none'
- Added support for the working units for partition sizes
2022-02-28 13:00:13 -07:00
Dan Bungert c8a4b09376 refresh: cancel the task waiting on update
Per LP: #1962025, the update screen can get stuck in test automation.
Cancel the task waiting to see if there is an update, which makes sense
more broadly anyhow in that if a regular UI user has declined the
update then the check task is no longer helpful.
2022-02-28 10:30:29 -07:00
Olivier Gayot 0ca56056ca Use systemd-run --user when non-root
Instead of using systemd-cat in dry-run mode and systemd-run in normal
mode, we now use systemd-run all the time but pass the --user switch
when running as non-root.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:13:17 +01:00
Olivier Gayot 5108f4cb1e Use private mounts when running curtin in-target & apt-config commands
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:13:16 +01:00
Olivier Gayot 87c249d5d0 Add type annotations for command execution methods
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:05:56 +01:00
Olivier Gayot 689173ad22 Add option to run commands with private mounts
The LoggedCommandRunner now has the option to start a command with
private mounts enabled - see mount_namespaces(7).

To use this feature, we now rely on systemd-run instead of systemd-cat.
The main benefit is that it should allow us not to worry about mounts
refusing to be un-mounted after executing a curtin in-target command.

On the downside, systemd-run requires root privileges (unless --user is
available) so we can't use it in dry-run mode. For dry-run mode, we
still need to rely on systemd-cat.

Another downside is that the commands passed to systemd-run are run in a
clean environment. Therefore, we must explicitly set the environment
variables that we want to pass to the commands to execute.

I came up with a short list of environment variables that we need but we
might need to add more later.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:05:56 +01:00
Olivier Gayot fe6d0878fc Make sure the FS unmounted were previously mounted in dry-run mode
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 3c9e5ca98f Use body for parameters of POST /drivers
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 98860b10ca Fix unmount of overlay broken by merge
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 382f73150d Add unit-tests for ubuntu-drivers
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot f50b5c0057 Make sure the drivers ctrler calls .configured until we have client ctrler
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot de148289aa Move the calls to ubuntu-drivers out of the drivers controller
We now provide implementation for different interfaces to list and
install drivers using ubuntu-drivers. The implementations are outside
the drivers controller.

We have:
 * the normal interface that calls ubuntu-drivers in the root directory
   specified (i.e., in /target or an overlay).
 * a dry-run interface that calls the system's ubuntu-drivers for
   listing the drivers available (but does not install anything).
   The listing of drivers will fail if --recommended is not an available
   option in the system's ubuntu-drivers implementation.
 * a dry-run interface that returns an hard-coded list of drivers and
   does nothing on installation.
 * a dry-run interface that returns an empty list of drivers.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot b0eb063288 Return a different flavor of driver when running on server vs desktop
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot d6f48f8d9c Make sure we decode the output of ubuntu-drivers
Using run_curtin_command, we have no way to specify the encoding or
something like universal_newlines=True. This is actually not supported
by asyncio.create_subprocess_exec(...).

We have partial support for this in Subiquity when using arun_command()
but run_curtin_command() actually relies on astart_command() + wait().

Therefore, we now decode the output at the very end, just before reading
the outout of ubuntu-drivers.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot d72fae0c8f Fix dry-run with overlay
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot abf05cf31e Pass the list of drivers instead of a boolean
In the reponse to the /a/drivers GET call, we now include a list of
packages / drivers instead of just storing a boolean telling if any
driver has been found.

Each element from the list corresponds to the name of a package to
install.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle 6241144681 check for ubuntu-drivers before trying to run it
[Olivier Gayot]
 * Drop TODO that is now implemented
2022-02-27 20:16:14 +01:00
Olivier Gayot 6f183a39ca Add do_install state in API response for drivers
The drivers GET call returned only the value of "has_drivers" (when
known). In order to have the client be able to know the current value of
install/do_install, we now embed the value into the response as well:

Before, the call would return either:

true   -> meaning that there are drivers available to install
false  -> meaning that there are no drivers available to install
null   -> retry later - we don't know yet

Now we have:

  {
      "has_drivers": boolean or null if we don't know yet
      "install": boolean
  }

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 43f71a02be Improve autoinstall schema for third-party drivers
The autoinstall data for the drivers was only constituted of a boolean,
e.g.:

  drivers: true

or

  drivers: false

It was not immediately clear that the boolean value meant (install/don't
install the drivers).

We now store the boolean in the "install" sub-element instead, e.g.:

  drivers:
    - install: true

  drivers:
    - install: false

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle e26f35ad10 remove duplicate lines 2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle ad26914a60 slightly insane gyrations to invoke ubuntu-drivers in better environment 2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle 6e65c801d4 only run ubuntu-drivers in dry-run mode if asked for 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle 9936fdb77f actually run ubuntu-drivers install 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle 5a4141050a actually run ubuntu-drivers 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle b2cd25b299 skeleton of a ubuntu-drivers controller and api
[Olivier Gayot]
 * update copyright year 2018 -> 2022
 * drop explicit inheritance from object

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 19:26:14 +01:00
Olivier Gayot 43090a46ba
Merge pull request #1188 from ogayot/modernize-overlay
Add type hints to overlayfs helpers and add some unit tests.
2022-02-27 17:12:09 +01:00
Dan Bungert 0f987ee6ba
Merge pull request #1191 from ogayot/consistent-loggers
Use logger names consistent with directory structure
2022-02-25 12:16:40 -07:00
Olivier Gayot 812bba1de1 Use logger names consistent with directory structure
Refreshed name of loggers to make them consistent with the directory
structure.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-25 17:35:45 +01:00
Dan Bungert b67c0b7dd0
Merge pull request #1190 from mwhudson/no-pip-bson
snapcraft.yaml: remove bson from python-packages
2022-02-22 15:23:35 -07:00
Michael Hudson-Doyle de4c2af81e snapcraft.yaml: remove bson from python-packages 2022-02-23 10:12:21 +13:00
Dan Bungert bd4da8b08c
Merge pull request #1189 from dbungert/os-probert-vs-devmapper
Revert probert again to remove os-prober usage
2022-02-22 13:43:45 -07:00
Dan Bungert 4f012918f8 Revert probert again to remove os-prober usage
os-prober is leaving around invalid devices and we don't know what to do
about it yet, but we do know that we don't require os-prober for
20.04.4.  Back this out.  LP: #1961640 LP: #1961628
2022-02-22 13:31:14 -07:00
Olivier Gayot 1c282546cc Use attr classes to define Mountpoint and OverlayMountpoint
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 16:18:23 +01:00
Olivier Gayot 7fb0c9b79d Accept only sequence of lowers in setup_overlays
AptConfigurer.setup_overlays() used to accept either a single Lower
directory or a sequence of them. It oftentimes felt confusing so we now
accept only a sequence of Lower's. If we want to pass a single Lower, we
can just pass a list with a single element.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:18:07 +01:00
Olivier Gayot 18c897f2d6 Add type hintings to overlay methods
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:18:07 +01:00
Olivier Gayot ef2a570e81 Add unit tests for apt.lowerdir_for
Added some unit tests for translating different "types" of directories
into a list of options suitable for mount -t overlay.

The unit tests can also be used as examples to visualize how a given
overlay would be configured depending on the parameters that are given
for its creation.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:07:54 +01:00