Commit Graph

5625 Commits

Author SHA1 Message Date
Olivier Gayot 6c3ae3c6dd mirror: add support for falling back to an offline install
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-15 18:09:39 +01:00
Olivier Gayot 10a2e6ac22 mirror: allow to combine different filters for the candidates
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-15 14:14:20 +01:00
Olivier Gayot 1350ef8d1b mirror: do not raise from POST /mirror if no usable mirror
Raising from the POST /mirror handler makes it difficult for the client
to distinguish an internal error from the lack of usable mirror.

We now return an enumeration with the following possible values:
 * ok
 * no-usable-mirror

It is up to the client to either:
 1. adjust the network settings and retry
 2. give up on the install

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-15 11:35:31 +01:00
Olivier Gayot 4247846270 network: add override mechanism to force offline install
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-15 09:05:14 +01:00
Olivier Gayot 58e524ae6e
Merge pull request #1552 from ogayot/fallback-mirror
Support of multiple candidates mirrors with automatic selection in autoinstall & desktop installs
2023-02-15 09:04:11 +01:00
Dan Bungert 25aaf00792
Merge pull request #1541 from dbungert/more-fde-strings
More fde strings
2023-02-14 15:51:47 -07:00
Olivier Gayot 12fbea7321 doc: update autoinstall reference & schema for mirror selection
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-14 22:18:13 +01:00
Olivier Gayot ff46c48d60 mirror: use apt->mirror-selection instead of relying on version
We used to rely on a version key under apt autoinstall section to
specify if we want the old implementation (i.e., a single candidate
primary mirror) or the new implementation (i.e., multiple primary mirror
candidates).

Instead, we now introduce the apt->mirror-selection autoinstall section,
and move the primary section under it.

If the primary section if under apt, we want the old implementation.
If the primary section is under apt->mirror-selection, we want the new
implementation.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-14 19:44:57 +01:00
Dan Bungert c2ce50f809
Merge pull request #1562 from dbungert/use-run-bg-task
many: s/create_task/run_bg_task/
2023-02-14 07:18:17 -07:00
Olivier Gayot 7848ccee46
Merge pull request #1559 from ogayot/ssh-import-mock
Stop calling ssh-import-id as part of the CI - use mock responses instead
2023-02-14 11:23:56 +01:00
Olivier Gayot 538a802118
Merge pull request #1560 from ogayot/fix-unittest-schroot
Fix unittest using timedatectl failing in chroot
2023-02-14 10:43:41 +01:00
Dan Bungert 9d12871f15 many: s/create_task/run_bg_task/
create_task has the following note:
  Important: Save a reference to the result of this function, to avoid a
  task disappearing mid-execution.

Convert existing usage of create_task to run_bg_task, if that
create_task is not actually storing the result.
2023-02-13 14:56:07 -07:00
Olivier Gayot fe8ab8a1e8 ssh: don't run ssh-import-id as part of the CI
Running ssh-import-id as part of the CI has limitations. First it
requires that we have access to the Internet. Second, it is affected by
github and launchpad service disruptions and or rate limiting policies.

Make sure we don't call ssh-import-id as part of the CI. Instead use a
username that is configured to produce fake successful key imports.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-13 16:41:47 +01:00
Olivier Gayot 2120054887 ssh: implement different dry-run strategies for ssh keys imports
We now have 3 different dry-run strategies for ssh key imports:

 * success: fake a successful key import
 * failure: fake a failed key import
 * run-on-host: actually run ssh-import-id

Through a dry-run-config file, we can choose which strategy applies
based on the username.

By default, the policy is as follows:
 * username 'heracles' uses the fake successful key import
 * username 'sisyphus' uses the fake failed key import
 * any other username runs ssh-import-id

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-13 12:31:10 +01:00
Olivier Gayot 093019b4ca mirror: log the the autoinstall repr. of a mirror when iterating
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-13 10:16:37 +01:00
Dan Bungert ffa29c4fcb
Merge pull request #1557 from dbungert/api-cleanup-match
test/api: match cleanup
2023-02-12 19:40:31 -07:00
Olivier Gayot 64d77470b0 tests: fix test using timedatectl failing in chroot
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-11 12:38:33 +01:00
Dan Bungert bd4e463177 ui/guided: tweak wording for ENCRYPTED + AVAILABLE 2023-02-10 15:48:30 -07:00
Dan Bungert 3c3e467966 server/fs: FDE string improvements 2023-02-10 15:32:24 -07:00
Dan Bungert b8ca9c54e8 ui/guided: FDE string improvements
Remove references to the "model", which isn't a concept we expect the
user to know about.  General text tweaks on top of that.
2023-02-10 15:32:24 -07:00
Dan Bungert 4855fd316d
Merge pull request #1554 from jpnurmi/netplan-generate
network: reset SNAP when calling netplan in dry-run mode
2023-02-10 13:45:40 -07:00
Olivier Gayot 51b82e7897 mirror: document model and use more relevant wording
* LegacyPrimarySection => LegacyPrimaryEntry
* PrimaryElement => BasePrimaryEntry
* PrimaryEntry => PrimaryEntry

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 915aeb6058 mirror: make sure we can get final configuration when offline
When offline, it is possible when requesting the final configuration
that the mirror model does not have an elected primary mirror.

This happens every time in an automated offline install.

Make sure that we return something sensible instead of raising an
exception.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 8533ed1bbe apt: be implementation agnostic when requesting a model's apt config
Models that participate in building the apt configuration (i.e.,
currently proxy & mirror) now have the same method signature
get_apt_config having "final" as a parameter.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot eb255c6996 apt: use wording final instead of elected to reduce coupling with mirror
The mirror model has the notion of primary candidates, staged primary
candidate and elected primary candidate. This is purely specific to the
mirror model and does not necessarily make sense from the outside.

In code that does not specifically need to know implmentation details of
the mirror model, we now prefer the wording "final configuration" over
"elected configuration".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot c5a0b567b4 mirror: add unit tests for find_and_elect_candidate_mirror
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 6fd4559763 mirror: make resolved country mirror entries distinguishable
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 5d1d761dd3 mirror: drop assign_elected and replace_candidate functions
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot a385e55197 mirror: improve /mirror API
* GET /mirror now returns:
  * the URL of all the candidates
  * the URL of the candidate staged for testing (if any - otherwise null)
  * the URL of the elected candidate (if any - otherwise null)

* POST /mirror can now be invoked in different ways:
  * Make fr.archive.ubuntu.com be the elected mirror ; and mark the
    mirror model configured:
  {"elected": "http://fr.archive.ubuntu.com/ubuntu"}

  * Mark us.archive.ubuntu.com as staged for testing.
    This replaces POST /mirror/candidate that was recently introduced
    (except it does not override model.candidates)
  {"staged": "http://us.archive.ubuntu.com/ubuntu"}

  * Replace the list of candidates with the new values. Not used on
    practice, but can be used for testing / debugging.
  {"candidates": ["http://us.archive.ubuntu.com/ubuntu"]}

  * Explicit way to trigger the election of a mirror automatically.
    If a mirror is elected, the model will be marked configured ;
    otherwise an exception is raised.  This would be the recommended
    option for the desktop installer NOTE: maybe something like
  null

A combination of multiple key can also be used.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 188a457792 mirror: produce usable autoinstall data in apt/mirror version 2
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 7dbd0177e4 mirror: add autoinstall tests for apt/mirror version 2
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 38bb82f43f mirror: skip unresolved mirrors on GET /mirror and autoinstalls
Country mirrors start with no URI. Make sure we skip those if we don't
receive a geoip query response.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 5711660d5d mirror: run check automatically if mirror is marked configured
Although it is not recommended, a client can mark the mirror model
configured by way of meta/mark_configured. If this happens, we will now
automatically run mirror testing to find the right candidate.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 378e6bd6ce mirror: extend endpoint for clients without a mirror screen
Clients that do not have a mirror screen can now rely on mirror testing
in a way similar to autoinstall. They can do so by sending a POST
request to /mirror with null in the body.

The call will block and will only return after Subiquity has either:
 * found a suitable mirror and elected it (mirror gets marked
   configured)
 * failed to find a suitable mirror ; a NoUsableMirrorError extception
   gets raised.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:03 +01:00
Olivier Gayot 272697201d mirror: handle the architectures properly
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 5c466e5765 mirror: start supporting both primary sections formats
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 037a5ac8ed mirror: add functions to make a section as staged or elected
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot f7cd813254 mirror: introduce new primary entry format but do not use yet
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 23db4b597b mirror: use properties to get and set mirror uri
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 927df0b845 mirror: do not assign an elected mirror automatically
When Subiquity starts, we no longer assign the elected mirror. This
means that marking the mirror model configured without selecting a
mirror is possible, and will cause trouble during install.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 6541ea055c mirror: use PrimarySection objects for candidates and elected
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot f8f6ca1fc2 mirror: add PrimarySection class with tests
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 54a06c6e5d mirror: try all candidates in autoinstall
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 099db0bfdd mirror: call relevant version of apt_config
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-09 11:43:01 +01:00
Dan Bungert 312e010d81 test/api: refactor out a few match usages 2023-02-08 19:56:45 -07:00
Dan Bungert e86d152a69 test/api: remove first / find 2023-02-08 19:56:45 -07:00
Dan Bungert c2f23f9cf2
Merge pull request #1556 from dbungert/format-blank
storage/v2: allow unformatted partition creation
2023-02-08 13:33:12 -07:00
Dan Bungert 0749f69c9b storage/v2: allow unformatted partition creation
Previously unformatted partition creation was blocked, but supplying
None does seem to do the right thing, and we want to support creation of
unformatted partitions.  Allow it and add a API test to help document it
as an intended feature.
2023-02-08 13:09:16 -07:00
Dan Bungert fa7220512d
Merge pull request #1555 from dbungert/api-test-timeouts
test/api: mark all with timeout
2023-02-08 12:59:26 -07:00
Dan Bungert 17b42fe797 test/api: mark all with timeout 2023-02-08 12:33:34 -07:00