[Metadata encryption]

Metadata encryption

Abstract

LSP completion, hover, and signature help depend on a large builtin metadata document derived from the live evaluator surface. In open development the JSON is plaintext and git-tracked. For compiled Nuitka binaries, the build pipeline can ship an encrypted blob so the binary does not leave a trivially scrapable metadata file on disk.

Mechanism: Fernet (symmetric, cryptography package) + optional SHA256 integrity sidecar.

Conceptual model

Interface surface

Files

Environment variables

Code entrypoints

• Encrypt stage: scripts/build/compile.py (encrypt_metadata), scripts/build/ci_build.py (stage_metadata)
• Decrypt: src/pynescript/langserver/providers/metadata_decrypt.py
• Loader preference: plaintext if present, else encrypted (get_metadata_cached)

Internals

Generation (always code-derived)

python scripts/generate_builtin_metadata.py

Do not hand-edit the JSON for permanent feature work — re-run the generator after adding builtins.

Encrypt (local sketch)

from cryptography.fernet import Fernet
key = Fernet.generate_key()  # or load stable secret
fernet = Fernet(key)
encrypted = fernet.encrypt(plaintext_bytes)

Build scripts chmod the key file to 0o600 when writing .metadata.key.

Decrypt integrity check

After Fernet decrypt, if a .sha256 sidecar exists, the loader compares sha256(plaintext)[:16] and raises on mismatch.

Dev vs binary

Invariants & edge cases

1. Stable key ⇒ reproducible .enc. Random key every CI run yields a different ciphertext even if plaintext is identical — not a functional bug, but pollutes diffs and caches.
2. Plaintext remains in git for open-source transparency of the language surface. Encryption protects the bundled binary layout, not the secret of the API catalog.
3. Truncated hash is integrity, not secrecy. It detects bitrot / wrong plaintext, not attackers with the key.
4. Never commit .metadata.key. Rotate if leaked; regenerate .enc with the new key for binary builds.
5. After adding builtins: regenerate metadata, re-encrypt with the same CI key, commit updated JSON (and .enc if tracked).

Worked examples

Regenerate + local encrypt

python scripts/generate_builtin_metadata.py
pip install cryptography
python scripts/build/compile.py --check   # stages metadata paths as configured

CI snippet

- name: Build LSP binary
  run: python scripts/build/ci_build.py --jobs 4
  env:
    CRYPTO_KEY: ${{ secrets.METADATA_KEY }}

Runtime with env key

export PYNESCRIPT_METADATA_KEY="$(cat scripts/build/.metadata.key)"
pynescript-lsp

Failure modes

See also

• Nuitka build
• LSP builtin metadata
• Release
• Security