Cloudflare D1 is a serverless SQLite-compatible database that runs at the edge. For most teams, the primary interaction happens inside Workers — you bind a D1 database to a Worker and query it directly using the env.DB.prepare() API. That works well for real-time reads. However, there is a second, less-discussed use case: programmatic database exports using the Cloudflare REST API, driven by external tooling rather than from within a Worker itself.
This article focuses on that second path. You will learn how to export a D1 database using the official cloudflare-go SDK (v6), why the export endpoint uses a polling model instead of streaming, and where the sharp edges are when you try to wire this into an automated pipeline.
Why Export D1 Programmatically
Before getting into code, the motivation matters.
D1 is designed for edge-local queries with low latency. It is not designed as a data warehouse. If you need to run analytics, seed a staging environment from production data, or back up to object storage like R2, you need a way to extract a full SQL dump from outside the Workers runtime.
Cloudflare provides exactly this via their REST API. The D1 export endpoint returns a signed URL to a .sql dump file. The catch is that generating this dump is not instantaneous. Cloudflare processes it asynchronously, so the client must poll for completion.
That polling model is the thing that trips most people up the first time they implement it.
Setting Up the cloudflare-go SDK
The official Go SDK for the Cloudflare API lives at github.com/cloudflare/cloudflare-go. As of v6, the package structure changed significantly from prior versions. The D1 functionality now lives under a dedicated d1 sub-package.
One non-obvious import detail: the sub-packages require the v6 path component explicitly. Omitting it causes the compiler to resolve an incompatible older version. Always import as follows:
import (
"context"
"fmt"
"log"
"time"
cloudflare "github.com/cloudflare/cloudflare-go/v6"
"github.com/cloudflare/cloudflare-go/v6/d1"
"github.com/cloudflare/cloudflare-go/v6/option"
)Installing via modules:
go get github.com/cloudflare/cloudflare-go/v6API Token Requirements
You need an API token with D1 write permissions. Cloudflare requires write scope even for export operations. Read-only tokens are insufficient, and the error message you get back from the API does not clearly explain this. You simply receive a permissions error, which is easy to misdiagnose as a wrong account ID.
Generate the token from the Cloudflare dashboard under Account > Manage Account > API Tokens. Create a custom token, scope it to your specific account, and grant D1 Write permissions. Store it securely; it is only shown once at creation time.
You will also need two identifiers from the dashboard URL when you navigate to your D1 database:
https://dash.cloudflare.com/<ACCOUNT_ID>/workers/d1/databases/<DATABASE_ID>/metricsBoth the account ID and database ID are visible directly in that URL.
The Polling Export Pattern
The export flow involves two phases. In the first phase, you initiate the export. Cloudflare begins generating the SQL dump and returns a bookmark string. In the second phase, you poll using that bookmark until the status field reads complete, at which point a signed URL to the dump file becomes available in the response.
Here is a complete, production-ready implementation:
package main
import (
"context"
"fmt"
"log"
"time"
cloudflare "github.com/cloudflare/cloudflare-go/v6"
"github.com/cloudflare/cloudflare-go/v6/d1"
"github.com/cloudflare/cloudflare-go/v6/option"
)
const (
pollInterval = 3 * time.Second
pollMaxRetries = 30
)
func main() {
apiToken := mustEnv("CF_API_TOKEN")
accountID := mustEnv("CF_ACCOUNT_ID")
databaseID := mustEnv("CF_D1_DATABASE_ID")
client := cloudflare.NewClient(
option.WithAPIToken(apiToken),
)
ctx := context.Background()
// Phase 1: initiate the export
resp, err := client.D1.Database.Export(
ctx,
databaseID,
d1.DatabaseExportParams{
AccountID: cloudflare.F(accountID),
OutputFormat: cloudflare.F(d1.DatabaseExportParamsOutputFormatPolling),
},
)
if err != nil {
log.Fatalf("export initiation failed: %v", err)
}
bookmark := resp.AtBookmark
if bookmark == "" {
log.Fatal("received empty bookmark from export initiation")
}
fmt.Printf("export initiated, bookmark: %s\n", bookmark)
// Phase 2: poll until complete
signedURL, err := pollForCompletion(ctx, client, databaseID, accountID, bookmark)
if err != nil {
log.Fatalf("polling failed: %v", err)
}
fmt.Printf("export ready: %s\n", signedURL)
}
func pollForCompletion(
ctx context.Context,
client *cloudflare.Client,
databaseID, accountID, bookmark string,
) (string, error) {
for attempt := 0; attempt < pollMaxRetries; attempt++ {
time.Sleep(pollInterval)
resp, err := client.D1.Database.Export(
ctx,
databaseID,
d1.DatabaseExportParams{
AccountID: cloudflare.F(accountID),
CurrentBookmark: cloudflare.F(bookmark),
OutputFormat: cloudflare.F(d1.DatabaseExportParamsOutputFormatPolling),
},
)
if err != nil {
return "", fmt.Errorf("poll attempt %d failed: %w", attempt+1, err)
}
if resp.Status == "complete" {
if resp.Result.SignedURL == "" {
return "", fmt.Errorf("status is complete but SignedURL is empty")
}
return resp.Result.SignedURL, nil
}
fmt.Printf("attempt %d: status=%s\n", attempt+1, resp.Status)
}
return "", fmt.Errorf("export did not complete after %d attempts", pollMaxRetries)
}
func mustEnv(key string) string {
val := os.Getenv(key)
if val == "" {
log.Fatalf("required environment variable %s is not set", key)
}
return val
}A few things worth noting in this implementation:
- Credentials come from environment variables, not hardcoded strings. Hardcoding tokens is the single most common mistake in sample code adapted from documentation.
- The bookmark value does not change between poll requests. You do not need to re-capture it on each response.
- The `mustEnv` helper makes missing configuration a fatal startup error rather than a subtle runtime nil.
- The retry limit prevents an infinite loop if Cloudflare returns something unexpected.
Takumi's Take: The polling pattern here looks innocuous but carries a real operational risk: the signed URL has a finite TTL. Cloudflare-generated signed URLs typically expire within a few hours. If you initiate an export, let the job run in a CI pipeline, and then a downstream step stalls before downloading, you may return to find the URL has expired and the dump is gone. Always download the file immediately after receiving the signed URL. Do not store the URL itself; store the file.
Handling the Response Structure
The response from a completed export contains a nested Result object. The structure looks like this conceptually:
| Field | Type | Description |
|---|---|---|
Status | string | "complete", "active", or error states |
AtBookmark | string | Bookmark to use in subsequent poll requests |
Result.SignedURL | string | Pre-signed URL to the .sql dump file |
The Status field is the main gate. Before it reads complete, the Result.SignedURL field is empty or absent. Guard against that explicitly, as shown in the pollForCompletion function above. A nil pointer dereference here is a common failure mode in naive implementations.
Integrating with a Backup Pipeline
The signed URL gives you an HTTPS link to a .sql file. From there, you can pull it into any object storage. Here is a minimal example that downloads the dump and uploads it to R2 using the AWS-compatible S3 SDK:
func downloadAndArchive(ctx context.Context, signedURL, r2Bucket, archiveKey string) error {
// Download the SQL dump
httpResp, err := http.Get(signedURL)
if err != nil {
return fmt.Errorf("download failed: %w", err)
}
defer httpResp.Body.Close()
if httpResp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status from signed URL: %d", httpResp.StatusCode)
}
// Upload to R2 via S3-compatible API
r2Client := newR2Client() // configure with R2 endpoint and credentials
_, err = r2Client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(r2Bucket),
Key: aws.String(archiveKey),
Body: httpResp.Body,
})
if err != nil {
return fmt.Errorf("R2 upload failed: %w", err)
}
return nil
}R2 is the natural target here because it lives in the same Cloudflare account, egress is free between Cloudflare services, and you can version the dumps using timestamped keys like backups/d1/2025-01-15T03:00:00Z.sql.
Running This as a Scheduled Worker
For automated backups, you likely do not want to run this as a standalone Go binary that you trigger manually. A better approach is a Cloudflare Worker triggered by a Cron Trigger, though that means writing the polling logic in JavaScript or TypeScript instead of Go.
Alternatively, run the Go binary as a scheduled job in your existing infrastructure. A GitHub Actions workflow with a cron schedule works well for teams already using Actions:
name: D1 Backup
on:
schedule:
- cron: '0 3 * * *'
jobs:
backup:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.23'
- name: Run D1 export
env:
CF_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
CF_ACCOUNT_ID: ${{ secrets.CF_ACCOUNT_ID }}
CF_D1_DATABASE_ID: ${{ secrets.CF_D1_DATABASE_ID }}
R2_ACCESS_KEY: ${{ secrets.R2_ACCESS_KEY }}
R2_SECRET_KEY: ${{ secrets.R2_SECRET_KEY }}
run: go run ./cmd/d1backup/...Keep secrets in GitHub Actions secrets, not in workflow YAML. That should be obvious, but automated pipelines have a way of accumulating shortcuts over time.
Known Limitations and Caveats
D1 is still maturing as a product. A few constraints are relevant to the export workflow:
First, this is a full-database export, not an incremental one. There is no current API for exporting only rows modified since a given timestamp. For large databases, this adds non-trivial export time and file size.
Second, D1 has per-database size limits. At the time of writing, individual D1 databases are capped at 10 GB for paid plans. Export performance degrades noticeably near that ceiling, and poll times increase accordingly.
Third, the export API is not designed for high-frequency calls. It is a maintenance operation, not a streaming data pipeline. If you need real-time data extraction from Cloudflare's edge, Queues with a consumer Worker is the appropriate architecture, not repeated exports.
The Cloudflare-Native Stack Context
The D1 export API makes the most sense when your entire data layer lives in Cloudflare. Workers handles compute, D1 handles relational data, R2 handles object storage, and KV handles low-latency key-value reads. This stack runs entirely on Cloudflare's edge network, which means the latency profile is dramatically different from a traditional VPS or even a regional cloud deployment.
The trade-off is that you are operating entirely within one vendor's ecosystem. Cloudflare does provide self-hosting paths for some products, and D1 itself is SQLite under the hood, so migrating data out is straightforward. But the operational tooling — the Wrangler CLI, the dashboard, the APIs — are all Cloudflare-specific.
For small to mid-size teams that are comfortable with that trade-off, the productivity gains are real. The backup pipeline described in this article can be built and deployed in under an hour. That same capability on a self-managed Postgres cluster requires considerably more infrastructure work.
For teams with strict data residency requirements or who need multi-cloud portability, think carefully before committing your relational data layer to D1. The export mechanism is your escape hatch, but it is a manual one.
What to Build Next
The code in this article gives you the raw mechanics of D1 export via Go. From here, a few extensions are worth considering.
Adding schema-only exports is useful for synchronizing database structure to staging environments without copying production data. The export API supports a noData flag in the parameters that strips INSERT statements from the dump.
Building a restore path is the other half of a real backup system. A D1 import API exists as a counterpart to export. A complete backup solution tests restores on a schedule, not just backups.
Monitoring export duration over time gives you early warning that your database is growing toward the size limits. A 10-second export turning into a 90-second export is a signal worth catching before it becomes a problem.