Commit Graph

4720 Commits

Author SHA1 Message Date
Olivier Gayot 87c249d5d0 Add type annotations for command execution methods
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:05:56 +01:00
Olivier Gayot 689173ad22 Add option to run commands with private mounts
The LoggedCommandRunner now has the option to start a command with
private mounts enabled - see mount_namespaces(7).

To use this feature, we now rely on systemd-run instead of systemd-cat.
The main benefit is that it should allow us not to worry about mounts
refusing to be un-mounted after executing a curtin in-target command.

On the downside, systemd-run requires root privileges (unless --user is
available) so we can't use it in dry-run mode. For dry-run mode, we
still need to rely on systemd-cat.

Another downside is that the commands passed to systemd-run are run in a
clean environment. Therefore, we must explicitly set the environment
variables that we want to pass to the commands to execute.

I came up with a short list of environment variables that we need but we
might need to add more later.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-28 14:05:56 +01:00
Olivier Gayot fe6d0878fc Make sure the FS unmounted were previously mounted in dry-run mode
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 3c9e5ca98f Use body for parameters of POST /drivers
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 98860b10ca Fix unmount of overlay broken by merge
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 382f73150d Add unit-tests for ubuntu-drivers
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot f50b5c0057 Make sure the drivers ctrler calls .configured until we have client ctrler
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot de148289aa Move the calls to ubuntu-drivers out of the drivers controller
We now provide implementation for different interfaces to list and
install drivers using ubuntu-drivers. The implementations are outside
the drivers controller.

We have:
 * the normal interface that calls ubuntu-drivers in the root directory
   specified (i.e., in /target or an overlay).
 * a dry-run interface that calls the system's ubuntu-drivers for
   listing the drivers available (but does not install anything).
   The listing of drivers will fail if --recommended is not an available
   option in the system's ubuntu-drivers implementation.
 * a dry-run interface that returns an hard-coded list of drivers and
   does nothing on installation.
 * a dry-run interface that returns an empty list of drivers.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot b0eb063288 Return a different flavor of driver when running on server vs desktop
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot d6f48f8d9c Make sure we decode the output of ubuntu-drivers
Using run_curtin_command, we have no way to specify the encoding or
something like universal_newlines=True. This is actually not supported
by asyncio.create_subprocess_exec(...).

We have partial support for this in Subiquity when using arun_command()
but run_curtin_command() actually relies on astart_command() + wait().

Therefore, we now decode the output at the very end, just before reading
the outout of ubuntu-drivers.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot d72fae0c8f Fix dry-run with overlay
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot abf05cf31e Pass the list of drivers instead of a boolean
In the reponse to the /a/drivers GET call, we now include a list of
packages / drivers instead of just storing a boolean telling if any
driver has been found.

Each element from the list corresponds to the name of a package to
install.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle 6241144681 check for ubuntu-drivers before trying to run it
[Olivier Gayot]
 * Drop TODO that is now implemented
2022-02-27 20:16:14 +01:00
Olivier Gayot 6f183a39ca Add do_install state in API response for drivers
The drivers GET call returned only the value of "has_drivers" (when
known). In order to have the client be able to know the current value of
install/do_install, we now embed the value into the response as well:

Before, the call would return either:

true   -> meaning that there are drivers available to install
false  -> meaning that there are no drivers available to install
null   -> retry later - we don't know yet

Now we have:

  {
      "has_drivers": boolean or null if we don't know yet
      "install": boolean
  }

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Olivier Gayot 43f71a02be Improve autoinstall schema for third-party drivers
The autoinstall data for the drivers was only constituted of a boolean,
e.g.:

  drivers: true

or

  drivers: false

It was not immediately clear that the boolean value meant (install/don't
install the drivers).

We now store the boolean in the "install" sub-element instead, e.g.:

  drivers:
    - install: true

  drivers:
    - install: false

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle e26f35ad10 remove duplicate lines 2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle ad26914a60 slightly insane gyrations to invoke ubuntu-drivers in better environment 2022-02-27 20:16:14 +01:00
Michael Hudson-Doyle 6e65c801d4 only run ubuntu-drivers in dry-run mode if asked for 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle 9936fdb77f actually run ubuntu-drivers install 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle 5a4141050a actually run ubuntu-drivers 2022-02-27 19:26:14 +01:00
Michael Hudson-Doyle b2cd25b299 skeleton of a ubuntu-drivers controller and api
[Olivier Gayot]
 * update copyright year 2018 -> 2022
 * drop explicit inheritance from object

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-27 19:26:14 +01:00
Olivier Gayot 43090a46ba
Merge pull request #1188 from ogayot/modernize-overlay
Add type hints to overlayfs helpers and add some unit tests.
2022-02-27 17:12:09 +01:00
Dan Bungert 0f987ee6ba
Merge pull request #1191 from ogayot/consistent-loggers
Use logger names consistent with directory structure
2022-02-25 12:16:40 -07:00
Olivier Gayot 812bba1de1 Use logger names consistent with directory structure
Refreshed name of loggers to make them consistent with the directory
structure.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-25 17:35:45 +01:00
Dan Bungert b67c0b7dd0
Merge pull request #1190 from mwhudson/no-pip-bson
snapcraft.yaml: remove bson from python-packages
2022-02-22 15:23:35 -07:00
Michael Hudson-Doyle de4c2af81e snapcraft.yaml: remove bson from python-packages 2022-02-23 10:12:21 +13:00
Dan Bungert bd4da8b08c
Merge pull request #1189 from dbungert/os-probert-vs-devmapper
Revert probert again to remove os-prober usage
2022-02-22 13:43:45 -07:00
Dan Bungert 4f012918f8 Revert probert again to remove os-prober usage
os-prober is leaving around invalid devices and we don't know what to do
about it yet, but we do know that we don't require os-prober for
20.04.4.  Back this out.  LP: #1961640 LP: #1961628
2022-02-22 13:31:14 -07:00
Olivier Gayot 1c282546cc Use attr classes to define Mountpoint and OverlayMountpoint
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 16:18:23 +01:00
Olivier Gayot 7fb0c9b79d Accept only sequence of lowers in setup_overlays
AptConfigurer.setup_overlays() used to accept either a single Lower
directory or a sequence of them. It oftentimes felt confusing so we now
accept only a sequence of Lower's. If we want to pass a single Lower, we
can just pass a list with a single element.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:18:07 +01:00
Olivier Gayot 18c897f2d6 Add type hintings to overlay methods
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:18:07 +01:00
Olivier Gayot ef2a570e81 Add unit tests for apt.lowerdir_for
Added some unit tests for translating different "types" of directories
into a list of options suitable for mount -t overlay.

The unit tests can also be used as examples to visualize how a given
overlay would be configured depending on the parameters that are given
for its creation.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:07:54 +01:00
Dan Bungert 00b0e8279d
Merge pull request #1187 from CarlosNihelton/fix-c-utf8
Fixes missing directory for system_setup integration test
2022-02-17 10:13:14 -07:00
Carlos Nihelton e79e4ea9a9
Removes unnecessary check `sourceDirs is None` 2022-02-17 07:07:19 -03:00
Carlos Nihelton a7fd83a384
Fixes missing dir on system_setup integration test
By providing alternative directories to find LC_* locale definitions.
2022-02-16 22:59:05 -03:00
Dan Bungert 4914e537ae
Merge pull request #1186 from dbungert/FR-2018-boot-efi-sizing
filesystem: add disk scaling for /boot and efi
2022-02-16 17:26:13 -07:00
Dan Bungert caab859a4f filesystem: add disk scaling for /boot and efi
Add a scaling system to adjust the size of /boot and efi partitions in
response to the size of the disk.
2022-02-16 16:51:46 -07:00
Dan Bungert eab1758e75
Merge pull request #1182 from dbungert/log-visibility
Log and config file generation as 0640 root:adm
2022-02-14 11:40:05 -07:00
Dan Bungert 42db747a0d file_util: remove omode / copy_mode
omode used to be in use, but has been standardized to 'w'.
copy_mode was unused.  Remove both.
2022-02-14 11:23:39 -07:00
Dan Bungert f4a9836143 file_util generate_config -> generate_config_yaml
The two usages of generate_config were both for yaml, so make that
simpler.
2022-02-14 11:21:56 -07:00
Dan Bungert b7349241a9 standardize on useage of file_util methods 2022-02-14 11:14:10 -07:00
Dan Bungert fae24f3247 logging: log files 0640 root:adm 2022-02-14 11:14:10 -07:00
Dan Bungert 9b5513f311 file_util: add open_perms and generate_config
Create open_perms context manager for custom or multiple writes.
Create generate_config for a small removal of redundancy.
0640 root:adm the resulting files.
2022-02-14 11:14:10 -07:00
Dan Bungert 7762dfaec0
Merge pull request #1178 from ogayot/FR-1953
Expose endpoint to get list of UA activable services
2022-02-14 09:45:37 -07:00
Dan Bungert 08e41ad745
Merge pull request #1181 from dbungert/lp-1960162
Lp 1960162
2022-02-11 16:30:05 -07:00
Dan Bungert 78fe7ccf7f ubuntu advantage: drop client token logging 2022-02-11 16:19:19 -07:00
Dan Bungert 824f6bbaee endpoints: add body logging
Add body logging for POST methods that accept a Payload for data.
Use repr=False on UbuntuAdvantageForm to not log the token.
2022-02-11 16:19:19 -07:00
Dan Bungert 07b8b50399 api: drop body from context reporting
Some API methods have sensitive information in the body of the call.
Stop logging the body unconditionally.
2022-02-11 16:19:19 -07:00
Dan Bungert f316ea5e06 storage: change guided method to Payload 2022-02-11 16:19:19 -07:00
Dan Bungert 9bd3ea6a6c filesystem: passphrase in answers, failing test 2022-02-11 16:19:19 -07:00