api: encode x-error-msg as JSON - so it does not contain <CR> or <LF>

When the server raises an exception in a HTTP request handler context,
more often than not, the exception is sent back to the client in the
body.

Additionally, the message of the exception (if any), is also copied as
is in a x-error-msg HTTP header.

That said, HTTP headers must obey strict rules. The "\r\n" sequence
indicate the end of the current HTTP header. When using aiohttp, the
library rejects any header that has a "\r" or "\n" in its value:

  ValueError: Newline or carriage return character detected in HTTP status message or header. This is a potential security issue.

As an example, any curtin.util.ProcessExecutionError exception will
contain "\n" characters when converted into a string.

We now encode the error message as JSON before copying it in the HTTP
header.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

subiquity/common/api/server.py

@@ -14,6 +14,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import inspect
+import json
 import logging
 import os
 import traceback
@@ -173,7 +174,10 @@ def _make_handler(
                     headers={
                         "x-status": "error",
                         "x-error-type": type(exc).__name__,
-                        "x-error-msg": str(exc),
+                        # aiohttp will reject a header if its value contains a
+                        # "\r" or "\n" character. By using compact JSON, we
+                        # ensure those characters are escaped.
+                        "x-error-msg": json.dumps(str(exc), indent=None),
                     },
                 )
                 resp["exception"] = exc

subiquity/common/api/tests/test_server.py

@@ -122,7 +122,7 @@ class TestBind(unittest.IsolatedAsyncioTestCase):
             self.assertEqual(resp.headers["x-status"], "error")
             self.assertEqual(resp.headers["x-error-type"], "TypeError")
             self.assertEqual(
-                resp.headers["x-error-msg"], 'missing required argument "arg"'
+                resp.headers["x-error-msg"], '"missing required argument \\"arg\\""'
             )
 
     async def test_error(self):