File Storage: Part 1

Improved TLS versions ensure secure data transport from client to server without extra client-side encryption. In this note, I''ll explain how to add unnecessary encryption when uploading files in the Phoenix web framework. In the rare case, a zero-day is discovered in the TLS protocol, causing data to leak. Adding an additional layer of encryption and integrity could be the solution. This note will explain how I set up client-side encryption and server-side decryption using forge.js, the Web Crypto API and Erlang''s :crypto and :public_key libraries NB! This solution increases the overhead of uploading files and is only an example. I do not recommend it for direct production usage. It is not Post-Quantum safe either, but a Post-Quantum Hybrid Key Exchange may be future work.

2023-12-25 17:40:22.200891+00

The idea

• The client and server complete a secure key exchange, preferably using ephemeral Diffie Hellman in elliptic curve format (not finite fields). Recommended by TLS 1.3.
• Encrypt files using the secret key generated from the exchange before users transfer them to the Phoenix web server (hybrid encryption).
• Decrypt the files on the server using the secret key generated from the exchange.
• Re-encrypt the files on the server using a randomly generated key before transferring them to the S3 bucket (also unnecessary since S3, by default, encrypts them).

Implementation

Code implementation

Key Exchange

Server libraries: Erlang's public_key
Client libraries: Web Crypto API

Provided that the web page containing the file form is successfully loaded, the subsequent series of operations can facilitate the key exchange protocol. It is essential to execute each operation in the prescribed order to ensure the successful completion of the exchange:

1. The server generates an ephemeral elliptic curve key pair.
   1. Base64 encodes the public key.
   2. DER encodes the encoded public key and signs the result with an existing static elliptic curve private key
      1. Note. The public key is DER encoded in order for both the client and server to read and use the public key sent from the opposing party.
2. The public key and its signature are transferred to the client by inserting them in the HTML of the webpage (unconventional but straightforward).
   1. The cryptographic data can be read when the client mounts the component.
3. The client validates the public key and its signature.
   1. The client validates the signature of the server's ephemeral public key using a public key corresponding to the static private key of the server.
4. The client generates an ephemeral elliptic curve key pair.
   1. DER encodes the public key
   2. Sends the encoded public key to the server.
5. The server and client compute the shared secret and derivates the secret key (KDF: PBKDF2).

Client Encryption

Client libraries: forge.js

The subsequent series of operations can facilitate the file encryption if the client and server have successfully computed the same secret key:

1. Initialize encryption
   1. Generate random IV.
   2. Initialize AES encryption in CTR mode using the IV and shared secret key.
   3. Initialize SHA512 HMAC using the shared secret key.
2. Iteratively encrypt the file content
   1. Read file content in chunks
      1. Encrypt the chunk
      2. Update the HMAC state
3. Concatenate the encrypted chunks and compute the HMAC digest.
4. Send the encrypted file with HMAC and IV to the server
   1. HMAC and IV are concatenated the original filename and transferred as the name of the file

Server Decryption and Re-encryption

Server libraries: Erlang's public_key and crypto

The subsequent series of operations can facilitate the file decryption if the client and server have successfully computed the same secret key and the file was correctly encrypted:

1. Decode filename to retrieve the HMAC and IV.
2. Iteratively reads the file as chunks.
   1. Initialize the FileWriter -> Phoenix's interface for reading the file.
      1. Initialize the decryption in CTR mode using the retrieved IV and shared secret.
      2. Initialize the SHA512 HMAC using the shared secret key.
      3. Generate a new secret key and IV.
      4. Initialize the re-encryption using the new secret key and IV.
      5. Initialize a re-encryption SHA512 HMAC using the new secret key.
      6. Open a temporary file.
   2. Decode Base64 file chunk.
   3. Decrypt the decoded chunk.
      1. Update the decrypt HMAC using the decoded chunk (not decrypted).
   4. Immediately re-encrypt the decrypted file chunk.
      1. Update the re-encrypt HMAC using the re-encrypted chunk.
   5. Write the re-encrypted chunk to the temporary file.
3. Validate that the computed decrypt HMAC digest against the HMAC retrieved from the client.
   1. If they match continue.
   2. Otherwise, throw an error.
4. Store the re-encryption metadata
   1. Re-encryption HMAC digest
   2. New IV
   3. New key
      1. Is encrypted using the user/group's secret key.
      2. User/group's secret key is also encrypted by server-wide KEK (Key Encryption Key).
      3. It should be stored more securely than in a database (currently not).
5. Transfer the re-encrypted file to the S3 bucket.