Commit Graph

4889 Commits

Author SHA1 Message Date
Olivier Gayot ef2a570e81 Add unit tests for apt.lowerdir_for
Added some unit tests for translating different "types" of directories
into a list of options suitable for mount -t overlay.

The unit tests can also be used as examples to visualize how a given
overlay would be configured depending on the parameters that are given
for its creation.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-21 15:07:54 +01:00
Dan Bungert 00b0e8279d
Merge pull request #1187 from CarlosNihelton/fix-c-utf8
Fixes missing directory for system_setup integration test
2022-02-17 10:13:14 -07:00
Carlos Nihelton e79e4ea9a9
Removes unnecessary check `sourceDirs is None` 2022-02-17 07:07:19 -03:00
Carlos Nihelton a7fd83a384
Fixes missing dir on system_setup integration test
By providing alternative directories to find LC_* locale definitions.
2022-02-16 22:59:05 -03:00
Dan Bungert 4914e537ae
Merge pull request #1186 from dbungert/FR-2018-boot-efi-sizing
filesystem: add disk scaling for /boot and efi
2022-02-16 17:26:13 -07:00
Dan Bungert caab859a4f filesystem: add disk scaling for /boot and efi
Add a scaling system to adjust the size of /boot and efi partitions in
response to the size of the disk.
2022-02-16 16:51:46 -07:00
Dan Bungert eab1758e75
Merge pull request #1182 from dbungert/log-visibility
Log and config file generation as 0640 root:adm
2022-02-14 11:40:05 -07:00
Dan Bungert 42db747a0d file_util: remove omode / copy_mode
omode used to be in use, but has been standardized to 'w'.
copy_mode was unused.  Remove both.
2022-02-14 11:23:39 -07:00
Dan Bungert f4a9836143 file_util generate_config -> generate_config_yaml
The two usages of generate_config were both for yaml, so make that
simpler.
2022-02-14 11:21:56 -07:00
Dan Bungert b7349241a9 standardize on useage of file_util methods 2022-02-14 11:14:10 -07:00
Dan Bungert fae24f3247 logging: log files 0640 root:adm 2022-02-14 11:14:10 -07:00
Dan Bungert 9b5513f311 file_util: add open_perms and generate_config
Create open_perms context manager for custom or multiple writes.
Create generate_config for a small removal of redundancy.
0640 root:adm the resulting files.
2022-02-14 11:14:10 -07:00
Dan Bungert 7762dfaec0
Merge pull request #1178 from ogayot/FR-1953
Expose endpoint to get list of UA activable services
2022-02-14 09:45:37 -07:00
Dan Bungert 08e41ad745
Merge pull request #1181 from dbungert/lp-1960162
Lp 1960162
2022-02-11 16:30:05 -07:00
Dan Bungert 78fe7ccf7f ubuntu advantage: drop client token logging 2022-02-11 16:19:19 -07:00
Dan Bungert 824f6bbaee endpoints: add body logging
Add body logging for POST methods that accept a Payload for data.
Use repr=False on UbuntuAdvantageForm to not log the token.
2022-02-11 16:19:19 -07:00
Dan Bungert 07b8b50399 api: drop body from context reporting
Some API methods have sensitive information in the body of the call.
Stop logging the body unconditionally.
2022-02-11 16:19:19 -07:00
Dan Bungert f316ea5e06 storage: change guided method to Payload 2022-02-11 16:19:19 -07:00
Dan Bungert 9bd3ea6a6c filesystem: passphrase in answers, failing test 2022-02-11 16:19:19 -07:00
Dan Bungert 7fba591bcf
Merge pull request #1183 from ogayot/geoip-aiohttp
Implement geoip using asyncio
2022-02-11 16:18:55 -07:00
Dan Bungert a6268011f5 test: fix installdeps 2022-02-11 16:08:34 -07:00
Dan Bungert b5f0d1fa73 geoip: fix lints 2022-02-11 16:03:35 -07:00
Olivier Gayot ebdf1db636 Move to aiohttp for GeoIP requests
GeoIP requests used to run on the default executor thread and would
prevent the application from exiting if the GeoIP service would not
respond quickly enough. We witnessed an obvious impact during an
incident on geoip.ubuntu.com.

Move to aiohttp so that the HTTP calls are non blocking.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-11 22:10:27 +01:00
Olivier Gayot 58df7d6583 Add dry-run mode for GeoIP info retrieval
We used to make real HTTP calls to geoip.ubuntu.com in dry-run mode.
This might have an impact on the service when running automatic testing.

We now provide an hardcoded value for the GeoIP information in dry-run
mode.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-11 22:10:27 +01:00
Olivier Gayot 8f024eb038 Only keep the name and description of activable services
The u-a-c returns a set of information about each service. After
filtering out the services that we don't want using the fields
"entitled" and "available", we now only keep the name and the
description of the service.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-09 14:43:22 +01:00
Olivier Gayot 98b887d696 Move UA module from common/ to server/
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-09 14:43:22 +01:00
Olivier Gayot 455e98aab9 Make sure to initialize _check_token_task in the UA controller
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-09 14:43:22 +01:00
Olivier Gayot c171aac2c7 Expose endpoint to get list of UA activable services
The list of activable UA services was only retrieved from the
client-side of Subiquity (using ubuntu-advantage-tools). Therefore, the
desktop installer would need to reimplement the same logic should they
need access to the list of services ; which is inconvenient.

We now expose a new endpoint in the API that takes the token as
a parameter and returns a status (+ a list of services if the token is
valid and not expired).

  $ curl \
    --unix-socket .subiquity/socket \
    --header 'Content-Type: application/json' \
    http://a/ubuntu_advantage/check_token \
    -d '"C123456"'

The token parameter is expected to be in the body of the request -
rather than in the query string - to avoid ending up in the access logs.

This new endpoint is a read-only GET endpoint. It is not designed as a
replacement for the POST to /a/ubuntu_advantage that the client must
(still) do to pass the token to the model.

We now use this new endpoint internally in Subiquity so that the
retrieval of the activable services is done on the server-side only.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-02-09 14:43:22 +01:00
Michael Hudson-Doyle aa659e8911
Merge pull request #1180 from dbungert/workflow-remove-hirsute
workflow: -hirsute
2022-02-04 10:45:22 +13:00
Dan Bungert c260df2894 workflow: -hirsute 2022-02-03 12:21:04 -07:00
Dan Bungert ffc6efb968
Merge pull request #1177 from dbungert/serializer-list-union
serializer: fix and test for KeyError '$type'
2022-02-02 09:32:50 -07:00
Dan Bungert 37cbfb0224 serializer: fix and test for KeyError '$type' 2022-01-28 16:52:16 -07:00
Dan Bungert d158d4e701
Merge pull request #1174 from ogayot/ua-entitled-services-only
Don't list UA services that are not entitled
2022-01-28 11:13:09 -07:00
Olivier Gayot 7dfe722686 Don't list UA services that are not entitled
Instead of only checking if a given UA service is available, we now also
check if it is entitled.

 - the available field for a service refers to its availability on the
   current machine (e.g. on Focal running on a amd64 CPU) ; whereas
 - the entitled field tells us if the contract covers the service.

Therefore, we need to make sure that we only list the services that are
both "available" and "entitled".

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-28 18:58:39 +01:00
Dan Bungert 3803ae0e80
Merge pull request #1172 from ogayot/validate-output-autoinstall-data
Validate output autoinstall data in integration tests
2022-01-26 10:44:46 -07:00
Dan Bungert 59199548aa
Merge pull request #1175 from dbungert/apitest-parallel
api test: run in parallel
2022-01-26 10:44:24 -07:00
Dan Bungert 72fbbc2868 api test: parallelize
The auto setting will choose based on the number of CPUs, including
virtual cores.  For me this is a 7x speedup.
2022-01-26 10:39:23 -07:00
Dan Bungert 0f9c9eef3f options: add output_base for dryrun use
To enable parallel API testing, add an output_base argument to replace
the default '.subiquity' that is sprinkled everywhere.
2022-01-26 10:39:23 -07:00
Dan Bungert 90a2bd6f7a
Merge pull request #1173 from dbungert/os-prober-arch
snap: fix os-prober related non-amd64 build failures
2022-01-25 07:47:09 -07:00
Dan Bungert ee20eb335a
Merge pull request #1167 from dbungert/lp-1952603-nonet
snaplist: handle cancel error paths
2022-01-25 07:46:46 -07:00
Dan Bungert ee528fe6a7 controller/snaplist: protect GET against timeout 2022-01-24 17:05:30 -07:00
Dan Bungert 853c39bae7 ui/snaplist: fix initialization 2022-01-24 17:03:31 -07:00
Dan Bungert a6901ddc25 kvm-test: allow deadnet simulation with --nets -1 2022-01-24 17:03:00 -07:00
Dan Bungert 4927affbe8 snapcraft: better file selection for os-prober sed 2022-01-24 13:18:49 -07:00
Dan Bungert 54ca91038e snapcraft: show os-prober as available app 2022-01-24 12:37:48 -07:00
Dan Bungert 8318fc4617 Revert "os-prober: patch instead of sed"
This reverts commit 7bdeb6707c.
2022-01-24 12:18:41 -07:00
Dan Bungert 229e364956
Merge pull request #1171 from ogayot/bump-curtin-version
Bump curtin version to fix APT preferences being discarded
2022-01-21 17:17:16 -07:00
Didier Roche 34b621ee66
Merge pull request #1165 from CarlosNihelton/fix-wsl-snap-env
[SystemSetup] Fix L-S-C behavior under WSL snap environment
2022-01-21 15:56:42 +01:00
Olivier Gayot e2216c1d53 Run script to validate autoinstall-user-data in integration tests
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-21 13:48:43 +01:00
Olivier Gayot 2bbd4fffe6 Update example states for Subiquity to cover valid token values
Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>
2022-01-21 13:48:43 +01:00