Commit Graph

5599 Commits

Author SHA1 Message Date
Olivier Gayot ff46c48d60 mirror: use apt->mirror-selection instead of relying on version
We used to rely on a version key under apt autoinstall section to
specify if we want the old implementation (i.e., a single candidate
primary mirror) or the new implementation (i.e., multiple primary mirror
candidates).

Instead, we now introduce the apt->mirror-selection autoinstall section,
and move the primary section under it.

If the primary section if under apt, we want the old implementation.
If the primary section is under apt->mirror-selection, we want the new
implementation.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-14 19:44:57 +01:00
Olivier Gayot 093019b4ca mirror: log the the autoinstall repr. of a mirror when iterating
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-13 10:16:37 +01:00
Olivier Gayot 51b82e7897 mirror: document model and use more relevant wording
* LegacyPrimarySection => LegacyPrimaryEntry
* PrimaryElement => BasePrimaryEntry
* PrimaryEntry => PrimaryEntry

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 915aeb6058 mirror: make sure we can get final configuration when offline
When offline, it is possible when requesting the final configuration
that the mirror model does not have an elected primary mirror.

This happens every time in an automated offline install.

Make sure that we return something sensible instead of raising an
exception.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 8533ed1bbe apt: be implementation agnostic when requesting a model's apt config
Models that participate in building the apt configuration (i.e.,
currently proxy & mirror) now have the same method signature
get_apt_config having "final" as a parameter.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot eb255c6996 apt: use wording final instead of elected to reduce coupling with mirror
The mirror model has the notion of primary candidates, staged primary
candidate and elected primary candidate. This is purely specific to the
mirror model and does not necessarily make sense from the outside.

In code that does not specifically need to know implmentation details of
the mirror model, we now prefer the wording "final configuration" over
"elected configuration".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot c5a0b567b4 mirror: add unit tests for find_and_elect_candidate_mirror
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 6fd4559763 mirror: make resolved country mirror entries distinguishable
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 5d1d761dd3 mirror: drop assign_elected and replace_candidate functions
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot a385e55197 mirror: improve /mirror API
* GET /mirror now returns:
  * the URL of all the candidates
  * the URL of the candidate staged for testing (if any - otherwise null)
  * the URL of the elected candidate (if any - otherwise null)

* POST /mirror can now be invoked in different ways:
  * Make fr.archive.ubuntu.com be the elected mirror ; and mark the
    mirror model configured:
  {"elected": "http://fr.archive.ubuntu.com/ubuntu"}

  * Mark us.archive.ubuntu.com as staged for testing.
    This replaces POST /mirror/candidate that was recently introduced
    (except it does not override model.candidates)
  {"staged": "http://us.archive.ubuntu.com/ubuntu"}

  * Replace the list of candidates with the new values. Not used on
    practice, but can be used for testing / debugging.
  {"candidates": ["http://us.archive.ubuntu.com/ubuntu"]}

  * Explicit way to trigger the election of a mirror automatically.
    If a mirror is elected, the model will be marked configured ;
    otherwise an exception is raised.  This would be the recommended
    option for the desktop installer NOTE: maybe something like
  null

A combination of multiple key can also be used.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 188a457792 mirror: produce usable autoinstall data in apt/mirror version 2
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 7dbd0177e4 mirror: add autoinstall tests for apt/mirror version 2
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 38bb82f43f mirror: skip unresolved mirrors on GET /mirror and autoinstalls
Country mirrors start with no URI. Make sure we skip those if we don't
receive a geoip query response.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 5711660d5d mirror: run check automatically if mirror is marked configured
Although it is not recommended, a client can mark the mirror model
configured by way of meta/mark_configured. If this happens, we will now
automatically run mirror testing to find the right candidate.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:05 +01:00
Olivier Gayot 378e6bd6ce mirror: extend endpoint for clients without a mirror screen
Clients that do not have a mirror screen can now rely on mirror testing
in a way similar to autoinstall. They can do so by sending a POST
request to /mirror with null in the body.

The call will block and will only return after Subiquity has either:
 * found a suitable mirror and elected it (mirror gets marked
   configured)
 * failed to find a suitable mirror ; a NoUsableMirrorError extception
   gets raised.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:43:03 +01:00
Olivier Gayot 272697201d mirror: handle the architectures properly
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 5c466e5765 mirror: start supporting both primary sections formats
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 037a5ac8ed mirror: add functions to make a section as staged or elected
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot f7cd813254 mirror: introduce new primary entry format but do not use yet
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 23db4b597b mirror: use properties to get and set mirror uri
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 927df0b845 mirror: do not assign an elected mirror automatically
When Subiquity starts, we no longer assign the elected mirror. This
means that marking the mirror model configured without selecting a
mirror is possible, and will cause trouble during install.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 6541ea055c mirror: use PrimarySection objects for candidates and elected
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot f8f6ca1fc2 mirror: add PrimarySection class with tests
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 54a06c6e5d mirror: try all candidates in autoinstall
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-10 15:41:16 +01:00
Olivier Gayot 099db0bfdd mirror: call relevant version of apt_config
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-09 11:43:01 +01:00
Olivier Gayot 5bfd6fda03 mirror: add theoretical support multiple primary sections
This patchs makes Subiquity internally support multiple primary
sections. Emphasis on "internally" because we only accept a single
section for autoinstalls. This is a first step into making it possible
to use multiple candidates mirrors.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-08 15:26:08 +01:00
Carlos Nihelton 7e2ceb9f94
Merge pull request #1550 from CarlosNihelton/ad-api-deeng-576
Active Directory API Part 1
2023-02-08 09:25:17 -03:00
Carlos Nihelton 7962bdcc9f
Shorter if clause 2023-02-08 09:05:00 -03:00
Olivier Gayot daeae3b004
Merge pull request #1547 from ogayot/apt-translated
Make sure mirror test output is translated if translation is available
2023-02-07 19:00:35 +01:00
Olivier Gayot d522487313 locale: generate the locale definition in the background
When the user selects a language, we now attempt to generate the
relevant locale definition. The generation can fail for multiple reasons
(including running as non-root) so a failure is not considered fatal.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-07 18:18:16 +01:00
Olivier Gayot 7bb5742685 apt: request translation when executing the mirror check
By setting the LANG variable and passing clean_locale=False, the apt-get
command will be translated if the locale definition exists on the
system.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-07 18:18:16 +01:00
Olivier Gayot f284d25757 utils: add parameter to avoid clearing the LC_* variables
We used to set LC_ALL=C unconditionally when executing a command. This
is a problem if we want the output of a specific command to be
translated (provided a locale definition and an actual translation exist
for the requested language).

This patch adds the clean_locale parameter, which can be used to specify
if we want the locale variable to be cleaned, to most of the helpers
that execute commands.

The value defaults to True so the default behavior is preserved.
If one wants a command to be translated, they have to  explicitly pass
clean_locale=False.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-02-07 18:18:16 +01:00
Carlos Nihelton f77971238f
Clearer logic on ADModel.target_packages method. 2023-02-07 14:17:49 -03:00
Carlos Nihelton fe25d07eca
Explains the return value of active_directory.POST
Returning a list of errors seems uncommon.
2023-02-06 22:52:49 -03:00
Carlos Nihelton 0ea977a6cc
Test enhancements:
- timeout
- assert equals because the success case wil return a very specific list
  of a single value: 'OK'
2023-02-06 22:37:16 -03:00
Carlos Nihelton 70819367de
Implements AD Model and Server Controller
Plus the first lines of test.

**Notice that the AD Controller is not required.**

POST'ing still incomplete: just a placeholder method to enable cycling
between testing and writing code.

Now testing the API is possible.
2023-02-06 22:31:06 -03:00
Carlos Nihelton ceda86031f
Defines the first API endpoint
/active_directory

Clients GET the current model or POST a new one.
Not implemented yet.
2023-02-06 22:31:05 -03:00
Carlos Nihelton bb5f6362c5
Adds vocabulary type for Active Directory support
ADConnectionInfo to be the model data submited by clients
ADValidationResult models the enum returned from a POST request.
(input must be validated in the server)
2023-02-06 22:31:03 -03:00
Dan Bungert d410022340
Merge pull request #1551 from dbungert/antirequirementism
requirements: remove
2023-02-06 17:53:23 -07:00
Dan Bungert cb9c2263e7 runtests: allow longer socket wakeup 2023-02-06 16:39:06 -07:00
Dan Bungert 6f6178c601 requirements: remove
Get em from apt
2023-02-06 16:13:22 -07:00
Dan Bungert f47874bdc9
Merge pull request #1545 from dbungert/lp-2004040
tui: turn off systemd ShowStatus
2023-02-02 07:05:53 -07:00
Dan Bungert 8dd14957c2 tui: turn off systemd ShowStatus
During TUI launch, turn systemd ShowStatus off. We don't want to do that
earlier, if we did then the easier path would be quiet boot or something
similar. This is done here so that service completion messages don't
overwrite the console.
2023-02-01 14:45:23 -07:00
Olivier Gayot e5f0e302c3
Merge pull request #1544 from ogayot/fixes-mirror-testing
Fixes and more tests for mirror testing
2023-02-01 17:58:05 +01:00
Olivier Gayot 190a469da4 mirror: add unit test for run_mirror_testing
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-31 10:15:42 +01:00
Olivier Gayot bc9ed2f0e1 mirror: fix primary and disabled components not in autoinstall data
In the following patches, we detached the disabled components and the
primary section from the config object in the mirror model.

  c13edb07f mirror: detach disabled-components from config
  8c29f3475 mirror: detach primary section from config object

However, the patches introduced a regression. The disable_components &
primary autoinstall section are not included in the
autoinstall-user-data file anymore.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-31 10:15:42 +01:00
Olivier Gayot a8610c424f mirror: show exception type when logging mirror check error
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-31 09:28:14 +01:00
Olivier Gayot ac98851b7c mirror: fix types hints in mirror screen
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-31 09:28:11 +01:00
Michael Hudson-Doyle 845e3971f7
Merge pull request #1542 from ogayot/with-pkgresources
snapcraft: add pkg_resources to the snap
2023-01-30 10:51:59 +13:00
Olivier Gayot 85af81718e snapcraft: add pkg_resources to the snap
From subiquity, we sometimes execute entry point python scripts such as:

 * usr/bin/ubuntu-advantage
 * usr/bin/ssh-import-id

Those scripts require pkg_resources on core20 to run properly:

  from pkg_resources import load_entry_point

On core22, the scripts will be a bit more flexible but for now, we need
python3-pkg-resources in the snap.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2023-01-27 10:43:56 +01:00