6. Design Rationale

RFC-SECOPS-0002                                                   Section 6
Category: Standards Track                               Design Rationale

6.1 Why This Section Exists

This section documents alternatives considered during the design of the interim secret management architecture and explains why each was rejected. Each alternative is evaluated against the invariants defined in Section 2.4 and the practical constraints of the current platform (on-premises infrastructure, no Kubernetes, no Vault).

6.2 HashiCorp Vault Without Prerequisites

Description: Deploy HashiCorp Vault directly on on-premises servers and use it as the secret store for the interim period, bypassing the need for AWS Secrets Manager entirely.

Why It Was Attractive:

• Eliminates the interim architecture altogether; the target system (RFC-SECOPS-0001) becomes the only system.
• No migration cost from interim to target.
• Full Vault feature set (rotation, audit, dynamic secrets) available immediately.

Why It Was Rejected:

• Vault on on-premises infrastructure requires significant operational investment: storage backend selection, TLS configuration, unsealing procedures, high availability setup, and ongoing maintenance.
• Vault without Kubernetes and ESO means applications still consume secrets via SDK (Vault HTTP API or Vault Agent), which defers but does not eliminate the SDK-coupling problem.
• The operational overhead of running Vault on on-premises infrastructure contradicts the design goal of minimal operational scope (Section 2.2.4).
• Vault deployment and hardening is itself a prerequisite for RFC-SECOPS-0001, and rushing it creates risk of misconfiguration.

Conclusion: Vault is the correct long-term solution, but deploying it without the supporting infrastructure (Kubernetes, ESO, GitOps) creates operational burden disproportionate to the interim need.

6.3 Environment Variables and .env Files

Description: Continue using the existing approach of .env files and environment variables as the primary secret storage mechanism, with improved documentation and operational discipline.

Why It Was Attractive:

• No additional infrastructure or services required.
• Zero migration effort; the current system continues unchanged.
• Simple and well-understood by all operators.

Why It Was Rejected:

• Violates Invariant 1 (sole secret store). Secrets remain scattered across servers with no single authoritative source.
• Violates Invariant 3 (secret values MUST NOT exist in Git) if .env files are version-controlled, which is a common failure pattern.
• No encryption at rest. Secrets exist as plaintext on VM filesystems.
• No access control beyond OS-level file permissions.
• No versioning, audit trail, or lifecycle management.
• Operational discipline alone has proven insufficient to prevent drift, duplication, and loss of visibility (Section 1.2).

Conclusion: The current approach is the problem this RFC solves. Continuing it with more discipline does not address the structural deficiencies.

6.4 Git-Encrypted Secrets (SOPS, Sealed Secrets)

Description: Store encrypted secret values directly in Git using Mozilla SOPS, Bitnami Sealed Secrets, or a similar encryption-at-rest mechanism.

Why It Was Attractive:

• Secrets are version-controlled alongside application configuration.
• Encryption at rest protects values in the repository.
• No external service dependency for secret storage.
• Works without Kubernetes (SOPS works with any Git repository).

Why It Was Rejected:

• Violates the spirit of Invariant 3 (secret values MUST NOT exist in Git). While values are encrypted, they are present in the repository history permanently. Key compromise exposes the entire secret history.
• Rotation requires Git commits (conflicts with RFC-SECOPS-0001 Invariant 3, which states rotation MUST NOT require Git changes). Adopting a rotation-requires-Git pattern now creates a convention that must be explicitly unlearned during migration.
• No runtime access control. Anyone with repository access and the decryption key can read all secrets.
• Sealed Secrets specifically requires Kubernetes, which is not available.
• SOPS solves storage encryption but does not provide centralized retrieval, access logging, or secret lifecycle management.

Conclusion: Git-encrypted secrets solve a different problem (secret storage in version control) and introduce patterns that actively conflict with the target architecture.

6.5 External Secrets Operator Without Kubernetes

Description: Find a non-Kubernetes equivalent of External Secrets Operator that runs on on-premises infrastructure and synchronizes secrets from a cloud store to local application configuration.

Why It Was Attractive:

• Decouples applications from the AWS SDK, matching the Phase 1 consumption model.
• Applications consume secrets as files or environment variables without knowing the backend.

Why It Was Rejected:

• ESO is a Kubernetes-native operator. No production-grade equivalent exists for non-Kubernetes environments.
• Building a custom synchronization daemon replicates ESO functionality outside its intended environment, creating maintenance burden without community support.
• The complexity of a custom sync daemon contradicts the design goal of minimal operational scope (Section 2.2.4).
• The AWS SDK approach is simpler, well-supported, and achieves the same functional outcome (secret retrieval) with less infrastructure.

Conclusion: ESO is the correct abstraction layer, but it requires Kubernetes. Attempting to replicate it outside Kubernetes introduces unjustified complexity.

6.6 Cloud-Native Secret Stores With Abstraction Layers

Description: Use AWS Secrets Manager (or another cloud secret store) but introduce an abstraction layer (e.g., a sidecar process, a local proxy, or a wrapper service) that presents secrets as files or environment variables, hiding the AWS SDK from applications.

Why It Was Attractive:

• Applications are decoupled from the AWS SDK immediately.
• Migration to Vault requires changing only the abstraction layer, not application code.
• Mimics the ESO consumption model without Kubernetes.

Why It Was Rejected:

• The abstraction layer itself is custom infrastructure that must be built, tested, deployed, and maintained.
• Running a sidecar or proxy on virtual machines introduces operational complexity (process management, failure handling, restart logic) that exceeds the complexity of direct SDK calls.
• The centralized Secret Config Module (Invariant 4) already provides the abstraction boundary. Replacing the module's internals during migration achieves the same decoupling benefit with less infrastructure.
• This approach over-engineers the interim solution for a problem that is fully solved by the target architecture.

Conclusion: The abstraction is correct in principle, but the application-level Secret Config Module provides sufficient isolation without requiring additional infrastructure.

6.7 Azure Key Vault

Description: Use Azure Key Vault as the interim cloud secret store instead of AWS Secrets Manager. The architectural pattern would be identical — centralized cloud store, SDK-based retrieval, service credentials on each server — with Microsoft Entra ID replacing AWS IAM for access control.

Why It Was Attractive:

• Azure Key Vault is a mature, widely adopted managed secret store with comparable functionality to AWS Secrets Manager.
• Microsoft Entra ID provides integrated identity management for organisations already using Microsoft 365.
• Extensive community documentation and SDK support across all major languages.
• Feature parity with AWS Secrets Manager for the interim use case (store, retrieve, encrypt at rest, access control scoping).

Why It Was Rejected:

• The organisation's executive leadership issued a directive to consolidate cloud service consumption under Amazon Web Services. The organisation already maintains an existing commercial relationship with AWS for core infrastructure services. Introducing Azure as a second cloud vendor would expand vendor surface area without a compelling technical justification.
• AWS IAM, which serves as the identity and access management layer for the organisation's existing AWS infrastructure, is directly reusable as the authentication and authorization layer for AWS Secrets Manager. Using Azure would require establishing a separate identity system (Microsoft Entra ID) with no relation to the existing organisational identity infrastructure.
• Billing, procurement, and support are already established with AWS. Adding Azure introduces a parallel commercial relationship for a single interim service.
• Azure Key Vault does not support / in secret names, requiring a -- separator workaround that differs from the target Vault path schema. AWS Secrets Manager supports / natively, eliminating separator remapping during migration.
• AWS CloudTrail provides audit logging by default for all Secrets Manager API calls, whereas Azure Monitor diagnostic logging for Key Vault requires explicit enablement and configuration.
• From a technical standpoint, there is no meaningful security or capability difference between Azure Key Vault and AWS Secrets Manager for this use case. The choice between them is therefore correctly determined by organisational and commercial alignment rather than technical merit.

Conclusion: Azure Key Vault is a technically equivalent alternative, but the organisational directive to consolidate on Amazon Web Services — driven by the existing AWS relationship and the desire to avoid multi-vendor complexity — combined with the technical advantage of native / path support, makes AWS Secrets Manager the correct choice for the interim architecture.