Storage Backend dialog: move endpoint out of the core model into MinIO's config

Goal

Fix a styling/UX bug in Admin > Storage's "Edit/Add Storage Backend" dialog (StorageBackendFormModal in frontend/src/StorageBackendsAdminPanel.jsx): the Endpoint field is currently rendered unconditionally, above the protocol selector, for every protocol (MinIO, GCS, S3). It's only meaningful for MinIO — the model column comment says so explicitly:

endpoint = Column(String(255), nullable=True)  # MinIO URL; empty for real cloud

endpoint is a MinIO concept, not a generic storage-backend concept — real cloud protocols (S3, GCS) don't have a customer-facing "endpoint URL," they talk to a fixed provider API. Confirmed with the user: the fix is not to keep endpoint as a shared/core column and merely change where it's rendered — it should stop being a core StorageBackend column at all, and become a MinIO-protocol-owned config field, exactly like admin_access_key/admin_secret_key already are.

Not in scope: bucket_prefix

bucket_prefix (nullable=False at the DB level, mandatory for every backend regardless of protocol) stays as a core column, rendered unconditionally in the parent dialog. Unlike endpoint, it isn't MinIO-specific — the S3ProtocolHandler/GcsProtocolHandler stub docstrings (backend/services/storage_protocols/s3.py, gcs.py) both describe their future provision() as creating "... + a bucket," implying bucket naming is a concern every protocol will need once implemented, not just MinIO. Confirmed with the user in an earlier round of this discussion.

Rejected alternative (earlier draft of this plan)

An earlier version of this plan kept endpoint as a core StorageBackend column and only moved rendering into MinioStorageForm.jsx via two new threaded props (endpoint/onEndpointChange) alongside the existing value/onChange for config. The user rejected this: endpoint being a MinIO concept means it belongs in MinIO's own config, not in the shared model with a special-case prop pair bolted onto the generic storage_protocol_forms contract. This revision does that instead — a real (if small) schema change, not just a frontend re-render.

Background — current state (confirmed by reading the code)

Design decisions (settled in discussion)

Phase 1 — Backend: per-key-aware secret masking

1.1 backend/services/storage_protocols/__init__.py

Add a class attribute to StorageProtocolHandler:

class StorageProtocolHandler:
    """..."""

    # Names of `config` keys that hold credential/secret material and must be masked as "****" in
    # admin API responses. None (the default) means "mask every key" — the conservative default
    # for any handler, including third-party plugins, that hasn't explicitly opted a key out.
    SECRET_CONFIG_KEYS = None

    def provision(self, project, backend) -> dict:
        ...

1.2 backend/services/secret_masking.py

def mask_config(config, secret_keys=None):
    """secret_keys=None masks every key (default, conservative). A protocol handler that declares
    SECRET_CONFIG_KEYS restricts masking to just those keys — other keys (e.g. MinIO's endpoint)
    pass through in plaintext."""
    config = config or {}
    if secret_keys is None:
        return {k: MASKED for k in config}
    return {k: (MASKED if k in secret_keys else v) for k, v in config.items()}

resolve_config/resolve_secret are unchanged — see Design decisions above.

Phase 2 — Backend: MinIO handler owns endpoint

2.1 plugins/ymerflow-minikube/minikube_plugin/storage_protocol.py

class MinioProtocolHandler(StorageProtocolHandler):
    SECRET_CONFIG_KEYS = frozenset({"admin_access_key", "admin_secret_key"})

    def provision(self, project, backend) -> dict:
        return setup_project_storage(
            project.id, backend.config["endpoint"], backend.bucket_prefix,
            backend.config["admin_access_key"], backend.config["admin_secret_key"],
        )

    def mint(self, project, backend) -> dict:
        raise NotImplementedError(...)  # unchanged

    async def test_connection(self, backend) -> None:
        client = get_minio_client_for_backend(
            backend.config["endpoint"], backend.config["admin_access_key"],
            backend.config["admin_secret_key"],
        )
        await asyncio.to_thread(lambda: list(client.list_buckets()))

Also update the module docstring (currently says "provision()/test_connection() are genuinely per-backend-parametrized: they read backend.endpoint/backend.bucket_prefix/backend.config") to say backend.bucket_prefix/backend.config (endpoint is now part of config).

Phase 3 — Migration

New Alembic revision (generate the id per CLAUDE.md rule 9 — alembic revision -m "..." or python3 -c "import uuid; print(uuid.uuid4().hex[:12])", verified unique via grep -rn "revision = '<id>'" --include=*.py .; down_revision must chain from whatever alembic heads reports as current at implementation time, not necessarily 604260b878e3 — other migrations may land first):

"""move storage_backends.endpoint into config['endpoint'] (minio-only concept)

Revision ID: <generate>
Revises: <current head at implementation time>
Create Date: <implementation date>
"""
from alembic import op
import sqlalchemy as sa
import json


def upgrade() -> None:
    conn = op.get_bind()
    rows = conn.execute(sa.text("SELECT id, endpoint, config FROM storage_backends")).fetchall()
    for row in rows:
        if not row.endpoint:
            continue
        config = row.config if isinstance(row.config, dict) else json.loads(row.config or "{}")
        config["endpoint"] = row.endpoint
        conn.execute(
            sa.text("UPDATE storage_backends SET config = :config WHERE id = :id"),
            {"id": row.id, "config": json.dumps(config)},
        )
    with op.batch_alter_table("storage_backends") as batch_op:
        batch_op.drop_column("endpoint")


def downgrade() -> None:
    with op.batch_alter_table("storage_backends") as batch_op:
        batch_op.add_column(sa.Column("endpoint", sa.String(255), nullable=True))
    conn = op.get_bind()
    rows = conn.execute(sa.text("SELECT id, config FROM storage_backends")).fetchall()
    for row in rows:
        config = row.config if isinstance(row.config, dict) else json.loads(row.config or "{}")
        endpoint = config.pop("endpoint", None)
        if endpoint is None:
            continue
        conn.execute(
            sa.text("UPDATE storage_backends SET endpoint = :endpoint, config = :config WHERE id = :id"),
            {"id": row.id, "endpoint": endpoint, "config": json.dumps(config)},
        )

The isinstance(row.config, dict) guard handles both possible raw-driver representations of the JSON column (already-deserialized dict vs. raw text) — verify against whichever DB backend is actually in use (dev: check backend/config.py/alembic.ini connection string) when implementing; 182d880e84c7 treated config as a JSON string via json.dumps/literal '{}' comparison, so this should hold, but confirm before relying on it.

3.1 backend/models/storage_backend.py

Remove the endpoint column and its to_dict() entry:

class StorageBackend(Base):
    __tablename__ = "storage_backends"

    id = Column(String(36), primary_key=True, default=lambda: str(uuid.uuid4()))
    name = Column(String(255), nullable=False)
    protocol = Column(String(32), nullable=False)          # s3, gcs, az, file
    bucket_prefix = Column(String(255), nullable=False)
    credential_strategy = Column(String(32), nullable=False, default="static-key")
    config = Column(JSON, nullable=False, default=dict)
    created_at = Column(DateTime, default=datetime.utcnow, nullable=False)
    sort_order = Column(Integer, nullable=False, default=0)
    active = Column(Boolean, nullable=False, default=True)

    def to_dict(self):
        return {
            "id": self.id,
            "name": self.name,
            "protocol": self.protocol,
            "bucket_prefix": self.bucket_prefix,
            "credential_strategy": self.credential_strategy,
            "created_at": self.created_at.isoformat(),
            "sort_order": self.sort_order,
            "active": self.active,
        }

Phase 4 — Backend: admin router cleanup (backend/routers/admin.py)

Phase 5 — Frontend

5.1 frontend/src/StorageBackendsAdminPanel.jsx

5.2 frontend/src/storageProviders/MinioStorageForm.jsx

Add the Endpoint field ahead of the existing admin key/secret fields, bound to value/onChange exactly like they are:

export default function MinioStorageForm({ value, onChange }) {
  return (
    <>
      <Form.Group className="mb-3">
        <Form.Label>Endpoint</Form.Label>
        <Form.Control
          value={value.endpoint || ''}
          onChange={e => onChange({ ...value, endpoint: e.target.value })}
        />
      </Form.Group>
      <Form.Group className="mb-3">
        <Form.Label>Admin Access Key</Form.Label>
        ...

5.3 frontend/src/storageProviders/GcsStorageForm.jsx, S3StorageForm.jsx

No changes.

Manual verification

Open Questions