Commit Graph

5874 Commits

Author SHA1 Message Date
Olivier Gayot 5177a6a8f6 mirror: add new endpoint to submit mirror without marking configured
The POST handler for /mirror applies the URL passed but also marks the
model configured. That said, we want the ability to:

 1. supply a mirror URL
 2. run the mirror test
 3. mark the model configured if the test is successful

This patches creates a new endpoint: /mirror/candidate that does the
same as /mirror except that it does not mark the model configured.

It is therefore suitable for step 1.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-24 12:50:23 +01:00
Olivier Gayot 86fb20ec04 mirror: add API to run mirror checking
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-24 12:50:15 +01:00
Olivier Gayot 3dd648238b
Merge pull request #1538 from ogayot/warn-warning
logging: replace deprecated calls to log.warn by log.warning
2023-01-24 12:46:51 +01:00
Olivier Gayot 4d6e1340ad logging: replace deprecated calls to log.warn by log.warning
Since Python 3.3, log.warn has been deprecated in favor of log.warning.
Running the unit tests raises the following warning:

subiquity/server/tests/test_geoip.py::TestGeoIPBadData::test_lookup_error
  /home/olivier/dev/canonical/subiquity/subiquity/server/geoip.py:112:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    log.warn("geoip lookup failed: %r", le)

I replaced all the calls to log.warn by calls to log.warning

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 16:25:12 +01:00
Olivier Gayot fd9e4ace47 apt: create the partial directory before running apt-get update
apt-get complains if the var/lib/apt/lists/partial directory does not
exist or is not owned by the _apt user. Make sure that it is.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:52:48 +01:00
Olivier Gayot 0451c9de3b apt: allow to store output of apt config check
When running the apt config check, we can now pass a stream that will be
populated with the output of apt-get update (stdout + stderr combined).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:51:51 +01:00
Olivier Gayot ed96088130 apt: add function to test currently applied apt configuration
The AptConfigurer object now has a method that runs apt-get update on
the applied configuration and returns the result Strictly speaking, it
does more than just checking if the mirror is in a good shape, thus the
naming shows "apt config" rather than "apt mirror".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:49:35 +01:00
Olivier Gayot 8c29f34754 mirror: detach primary section from config object
Going forward, we need the mirror model to keep the data that it can
handle separately from what goes straight to curtin. This patch
separates the primary section from the config blob.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:49:14 +01:00
Olivier Gayot c13edb07f8 mirror: detach disabled-components from config
Going forward, we need the mirror model to the data that it can handle
separately from what goes straight to curtin. This patch separates the
disabled-components from the config blob.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:47:22 +01:00
Olivier Gayot 1decdffd75 mirror: add function to add country code to URI
Instead of transforming on the fly the archive URL into the country mirror URL
using string substitutions, we can now use the countrify_uri function that does
the job and has unit tests.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:44:23 +01:00
Olivier Gayot 927882c642 apt: make apt-configure tests not rely on DEFAULT
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:44:23 +01:00
Olivier Gayot 9b6405adc8 mounter: remove conflicting instructions about sources.list.d
In dry-run mode, when generating the fake overlay for apt, we copy the
content of /etc/apt from the host to the temporary directory.
For some reason, we then removed the sources.list.d directory in the
destination.

When I noticed the absence of sources.list.d in the fake overlay, I
added another call to cp(1) to get it populated. However, a more
sensible thing to do is to get rid of the instruction that removed
sources.list.d.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-20 10:44:23 +01:00
Dan Bungert c6564de0dc
Merge pull request #1535 from dbungert/no-except-on-geoip-failure
geoip: do not raise on geoip failure
2023-01-18 11:26:47 -07:00
Dan Bungert 11d89ec0b9 geoip: do not raise on geoip failure
This is non-fatal, and users find the Traceback confusing.

Sample log statement:
  2023-01-17 18:18:19,958 DEBUG subiquity.server.geoip:113 geoip lookup
  failed: ClientConnectorError(ConnectionKey(host='lmao', port=443,
  is_ssl=True, ssl=None, proxy=None, proxy_auth=None,
  proxy_headers_hash=None), gaierror(-5, 'No address associated with
  hostname'))
2023-01-18 11:02:03 -07:00
Carlos Nihelton d3099728ea
Merge pull request #1534 from CarlosNihelton/fix-none-wslsetupoptions
Fix wslsetupotions being None
2023-01-13 06:56:14 -03:00
Olivier Gayot 9aeb3d2bb3
Merge pull request #1531 from ogayot/fix-discarded-passphrase
storage: fix creation of encrypted VG in manual partitioning
2023-01-13 09:15:34 +01:00
Carlos Nihelton 41a2427704
Fix wslsetupotions being None
Recent changes in GUI turned possible to have that field as None,
leading to a crash.
This avoids the crash while preserving the default behavior of
installing the language packs.
2023-01-12 16:57:52 -03:00
Olivier Gayot 9612a7eafc storage: use "passphrase" for LVM/LUKS in the server
The client expected the LUKS passphrase to be called "passphrase" but
the server expected it to be called "password".

To keep the implementation consistent, we now use "passphrase"
everywhere except in the API (i.e., storage/guided and
storage/v2/guided) where "password" is still used for backward
compatibility reasons.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-12 10:21:56 +01:00
Olivier Gayot 8cb6318456
Merge pull request #1533 from ogayot/apt-dry-run-copy-sources.list.d
apt: in dry-run code, also copy etc/apt/sources.list.d
2023-01-12 09:20:38 +01:00
Michael Hudson-Doyle dd45be1457
Merge pull request #1526 from mwhudson/show-encryption-unavailable-reason
show the FDE "unavailable-reason" from snapd to the user
2023-01-12 15:17:37 +13:00
Michael Hudson-Doyle ef371a652b address review comments 2023-01-12 14:10:29 +13:00
Michael Hudson-Doyle d7dfc5ed7f show the FDE "unavailable-reason" from snapd to the user 2023-01-12 14:10:29 +13:00
Olivier Gayot ec29fa9a0a apt: in dry-run code, also copy etc/apt/sources.list.d
In dry-run mode, we used to only copy etc/apt/sources.list to the
fake overlay. However, if the host uses deb822, the sources.list file is
usually empty.

This patch also makes sure to copy the deb822 sources from
etc/apt/sources.list.d/

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-11 14:22:43 +01:00
Olivier Gayot dcc66fa346 storage: fix crash when editing encrypted VG created in guided storage
When editing an encrypted VG that was created in the guided storage
screen, the VG information is originating from the server. However, the
server does not send the LUKS key over the wire. Instead it sends the
path to a keyfile which contains the key. The client may or may not have
read access to this keyfile so it does not have a reliable way to
determine the key.

This causes problem when editing the VG because the GUI expects to
receive a key when encryption is enabled.

If the VG object only contains a keyfile, the passphrase is set to None
and this result in the GUI crashing.

This patch fixes the crash by passing an empty passphrase instead of a
None value when the VG object only contains a keyfile.

This means the user gets forced to supply a passphrase again when
editing an encrypted VG that was created in the guided partition screen.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-10 16:13:41 +01:00
Olivier Gayot 723bd1ebe6 storage: fix creation of encrypted VG in manual partitioning
The following patch changed the name of the luks passphrase field from
"password" to "passphrase" to make it consistent across the screens:

  commit d63b44c014
  storage: use fields named passphrase for passphrases

  Because the storage views lean on the implementation of
  setup_password_validation from the identity screen, we were forced to
  use a form with fields named "password" and "password_confirm".

  This makes the code confusing because we use the "passphrase"
  terminology in the storage forms.

  We now leave up to the caller to specify which fields he wants to be
  part of the validation ; instead of making him provide the full form.

However, this change broke the ability to create an encrypted VG in
manual partitioning mode.

This happened because the values input in the VG edit form are passed
as-is (mostly) to functions that are also used by the server.

On one hand, we have the client which deals with passphrases named
"passphrase" and "confirm_passphrase".
On the other hand, we have the server which deals with passphrases
named "password" and "confirm_password".

This inconsistency makes it hard to work with shared code.

To work around the issue, we now rename the passphrase key (passphrase
-> password) before passing it to the shared code.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-10 12:41:50 +01:00
Michael Hudson-Doyle 09c498f01e
Merge pull request #1521 from mwhudson/all-deb-dependencies
snapcraft.yaml: Get all Python dependencies from the archive
2023-01-10 09:28:40 +13:00
Olivier Gayot 9ec0ce73ee ubuntu-pro: enable for LTS releases
The version of ubuntu-advantage-tools present in focal-updates contains
all we need with regard to the magic attach implementation.

Although we are still missing the ubuntu.com/pro/attach screens, they
should get active in the near future. It should be enough for testing.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-09 17:41:58 +01:00
Olivier Gayot 9563b543ce
Merge pull request #1527 from ogayot/ssh-import-lp
integration-tests: import SSH keys from LP instead of GH
2023-01-09 17:40:52 +01:00
Olivier Gayot ec80a1cdee integration-tests: import SSH keys from LP instead of GH
As part of our integrations tests, we import mwhudson's public SSH
key(s) from GitHub. At the moment, however, the GitHub API is rate
limiting the number of queries from our CI.

Upon exceeding the rate limit, our HTTP queries are responded with a 403:

┌────────────────────────────────────────────────────────────────────────┐
│  Importing keys failed:                                                │
│                                                                        │
│  2023-01-08 23:43:40,562 ERROR GitHub REST API rate-limited this IP    │
│  address. See https://developer.github.com/v3/#rate-limiting .         │
│  status_code=403 user=mwhudson                                         │
└────────────────────────────────────────────────────────────────────────┘

Currently, upon pushing to GitHub, the CI runs the integrations tests
against 4 different Ubuntu images (focal, jammy, kinetic, lunar).

This ends up doing 4 SSH import queries at roughly the same time ; which
often exceeds the rate limit and makes some of the tests fail.

This patch makes integration tests import SSH keys from Launchpad
instead of GitHub. Maybe a better approach would be to mock the calls to
ssh-import-id in the CI instead.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-09 09:43:53 +01:00
Didier Roche-Tolomelli f8dbff5224
Merge pull request #1522 from CarlosNihelton/adv-config-wsl-350
[WSL] WslConfigAdvanced only on reconfiguration variant
2023-01-06 15:20:40 +01:00
Olivier Gayot 69264ad7f2
Merge pull request #1525 from ogayot/confirmation-overlay
Add helper to create confirmation dialog and use it for Ubuntu Pro
2023-01-05 10:50:54 +01:00
Olivier Gayot 5c2d4463b6 ubuntu-pro: use confirmation overlay for ubuntu pro
Instead of defining a ContinueAnyway widget specific to Ubuntu Pro, we
now lean on the new ConfirmationOverlay to obtain the same result.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-05 09:54:51 +01:00
Olivier Gayot 51b6772175 ui: add a helper to open a confirmation dialog and get the response
The new ConfirmationOverlay object along with the
BaseView.ask_confirmation helper can be used to open a confirmation
dialog and get back the decision from the user.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-05 09:54:51 +01:00
Olivier Gayot 301d9ddcba
Merge pull request #1524 from ogayot/fix-astart-stdin
utils: fix unused stdin parameter in astart_command
2023-01-05 09:52:53 +01:00
Olivier Gayot 108b26d76d utils: fix unused stdin parameter in astart_command
Specifying the value of subprocess.PIPE as the stdin argument of
astart_command did not have any effect. This happened because the
parameter was not forwarded to the subprocess function. The parameter
was effectively unused.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-03 19:00:16 +01:00
Carlos Nihelton 5319633d78
Makes the wsl_conf imune to None 2022-12-16 13:28:11 +00:00
Carlos Nihelton 4f0b45798b
Answers no longer relevant for wsl_setup
Integration tests on wsl_setup variant should not write wsl.conf.
2022-12-16 13:28:11 +00:00
Carlos Nihelton 3e04dab619
Interactive wsl_setup no longer writes wsl.conf
That variant would only apply configs if in autoinstall.
There are no more screens available related to those settings in
wsl_setup.
Reconfiguration variant is the only one able to write that file.
2022-12-16 13:28:10 +00:00
Carlos Nihelton 0540c521fe
Moves WslConfBase to the reconfiguration workflow 2022-12-16 13:28:10 +00:00
Dan Bungert 59f167c12b
Merge pull request #1486 from dbungert/workflow-lunar
workflow: +lunar
2022-12-15 08:00:27 -07:00
Dan Bungert 4db88f5075 workflow: +lunar 2022-12-15 07:20:44 -07:00
Dan Bungert 1851531c54
Merge pull request #1520 from dbungert/pep632
distutils migration related items
2022-12-15 07:19:52 -07:00
Michael Hudson-Doyle ceff8a6095 snapcraft.yaml: Get all Python dependencies from the archive
Once we do this, there is no reason to use the 'python' plugin, so
switch to the 'nil' plugin with an override-build that calls pip for
each of the subiquity, curtin, and probert parts.
2022-12-15 11:24:10 +13:00
Dan Bungert 78d55da71b setup.py: start migration away from distutils
Follow guidance from PEP 632 and move some of this over to setuputils.
build.build lacks a straightforward answer, use the vendored copy of
distutils found in setuptools but delaying the import until after
setuptools.
2022-12-14 10:32:20 -07:00
Dan Bungert c62b72923f aptdeps: +gettext 2022-12-14 09:19:13 -07:00
Olivier Gayot 6e868ca771
Merge pull request #1519 from ogayot/cancel-update
refresh: remove the cancel update button since it is broken
2022-12-14 12:39:12 +01:00
Olivier Gayot 8f42b8404e refresh: remove the cancel update button since it is broken
Cancelling the installer self refresh does not work. Clicking on the
button simply goes back to the screen asking the user if they want to
update ; but the refresh operation still continues in the background.

Since we do not have a simple way to implement refresh cancellation at
the moment, we are just removing the button.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-12-13 17:27:21 +01:00
Dan Bungert 0eddccd75a
Merge pull request #1471 from kenvandine/users_group_not_lxd
Ensure the first user is in the users group and not in the lxd group
2022-12-13 08:11:06 -07:00
Carlos Nihelton d330449dbf
Merge pull request #1517 from CarlosNihelton/no-switch-shell-wsl-325
Removes the switch to shell widget
2022-12-12 14:29:14 +00:00
Olivier Gayot e47f667e15
Merge pull request #1324 from ogayot/remove_overlay_no_overlay_ok
ui: avoid crashing when removing overlay that does not exist
2022-12-12 09:09:19 +01:00