Imports

Normally a client pushes bytes to CloudPDF (init → upload → commit). Imports invert that: the server pulls a PDF from where it already lives — a presigned URL, or a bucket you register — then verifies and commits it through the exact same pipeline. Nothing about the document is different afterwards; only who moved the bytes changed.

Two source models exist, and they differ by where authority comes from:

Import policy#

Imports are enabled by default with a safe posture: HTTPS-only, public networks only, bounded size and time.

Synchronous imports hold the HTTP response open for the whole transfer, so keep MAX_BYTES × your slowest source comfortably inside TIMEOUT_MS — the defaults give a same-region 128 MiB pull a wide margin. Larger or slower transfers belong in async mode (below), which is not bound by the request.

Connections#

A connection is a named source you register once; import requests then reference it by id plus an object key — no presigning, no credentials on the wire, ever. Declare the list, then configure each name:

CLOUDPDF_IMPORT_CONNECTIONS=customer-archive
 
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_KIND=s3
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_S3_BUCKET=customer-documents
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_S3_REGION=eu-west-1

That is a complete, working setup: the whole bucket becomes importable by key, restricted to the operator API token (the default — see Authorization below).

Names use [A-Za-z0-9_-] and are upper-snake-cased in the variable names (customer-archive → CUSTOMER_ARCHIVE). Two names that collide after normalization refuse to boot.

S3 and S3-compatible#

CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_KIND=s3
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_S3_BUCKET=customer-documents
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_S3_REGION=eu-west-1
# MinIO / R2 / Wasabi:
CLOUDPDF_IMPORT_CONNECTION_CUSTOMER_ARCHIVE_S3_ENDPOINT=https://minio.internal:9000

Credentials follow the standard AWS provider chain — an instance role, IRSA, or AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY — exactly like storage. Nothing credential-shaped goes in the connection itself. Grant the role s3:GetObject on the source bucket (plus s3:GetObjectVersion if you pin versions).

Google Cloud Storage#

CLOUDPDF_IMPORT_CONNECTION_LEGAL_SCANS_KIND=gcs
CLOUDPDF_IMPORT_CONNECTION_LEGAL_SCANS_GCS_BUCKET=legal-scans
# Optional: CLOUDPDF_IMPORT_CONNECTION_LEGAL_SCANS_GCS_PROJECT_ID=my-project

Authenticates via Application Default Credentials (workload identity, or a key file referenced by GOOGLE_APPLICATION_CREDENTIALS).

Azure Blob Storage#

CLOUDPDF_IMPORT_CONNECTION_HR_DOCS_KIND=azure-blob
CLOUDPDF_IMPORT_CONNECTION_HR_DOCS_AZURE_BLOB_CONTAINER=hr-documents
CLOUDPDF_IMPORT_CONNECTION_HR_DOCS_AZURE_BLOB_ACCOUNT_NAME=contosoprod
# Optional custom endpoint:
# CLOUDPDF_IMPORT_CONNECTION_HR_DOCS_AZURE_BLOB_ENDPOINT=https://...

Authenticates via the standard Azure identity chain (managed identity, workload identity, or az login locally). Grant the identity blob read access on the container.

Filesystem#

CLOUDPDF_IMPORT_CONNECTION_LOCAL_DROP_KIND=fs
CLOUDPDF_IMPORT_CONNECTION_LOCAL_DROP_FS_ROOT=/srv/import-inbox

Reads files from a directory on the server host — useful for migration workflows (“drop the files on the volume, import by path”). Keys are /-separated paths relative to the root; absolute paths, .. segments, and symlinks that escape the root are all refused.

Authorization: who may use a connection#

Registering a connection is the moment you grant access — so each connection carries its own rules, all enforced before a single byte moves:

The default is deliberately strict: whole-bucket access belongs to the operator’s API token only. Importing is effectively reading — whoever can import an object can then view it — so opening a connection to tenant tokens is only allowed when the connection also says which slice each tenant gets:

• shared-prefixes grants a fixed slice to every allowed caller — right for a common assets folder that all tenants may read.
• tenant-template gives each tenant its own slice, computed from the authenticated tenant id at request time. One line scales to any number of tenants with zero per-tenant configuration:

CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_KIND=s3
CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_S3_BUCKET=app-documents
CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_S3_REGION=eu-west-1
CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_CREDENTIALS=api-token,tenant-jwt
CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_SCOPE=tenant-template
CLOUDPDF_IMPORT_CONNECTION_ACME_DOCS_SCOPE_TEMPLATE=tenants/{tenantId}/

With this, tenant acme can import keys under tenants/acme/ and nothing else — including no tenants/acme-other/, because templates must end with / precisely so tenant slices can never prefix-collide.

All of this is validated at boot, fail-closed. A connection that opens tenant-jwt on a whole bucket, a template without exactly one {tenantId} placeholder or without the trailing slash, mismatched scope variables, or a connection pointing at the server’s own storage backend (which would let imports read CloudPDF’s own bucket) — each refuses to start with a message naming the connection and the rule.

Async imports and the worker#

Clients may submit an import with mode: "async": the API answers 202 immediately and a background worker performs the transfer with automatic retries. There is nothing to deploy or configure for this — the worker runs inside the server process (like the cleanup sweeper), starts whenever imports are enabled, and is safe with multiple replicas: jobs are claimed with expiring leases, and a replica that dies mid-transfer has its job picked up by another.

Two knobs you already know shape it: CLOUDPDF_IMPORT_MAX_CONCURRENT bounds transfers per replica (sync and async share the budget), and CLOUDPDF_IMPORT_TIMEOUT_MS bounds each attempt. Failed attempts retry with exponential backoff (30s doubling, capped at 15 minutes, five attempts) before the document is marked failed.

Next steps#