Phase 4: Cloud storage clients + archive orchestration + GHES version checker (Go port)#1535
Draft
offbyone wants to merge 2 commits intoo1/golang-port/3from
Draft
Phase 4: Cloud storage clients + archive orchestration + GHES version checker (Go port)#1535offbyone wants to merge 2 commits intoo1/golang-port/3from
offbyone wants to merge 2 commits intoo1/golang-port/3from
Conversation
This was referenced Mar 31, 2026
offbyone
force-pushed
the
o1/golang-port/4
branch
from
eff3a82 to
981a975
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
1da1059 to
470dc87
Compare
6 tasks
offbyone
force-pushed
the
o1/golang-port/3
branch
from
470dc87 to
d53f923
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
2 times, most recently
from
65bf041 to
e18695f
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
d53f923 to
f49db2c
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
from
e18695f to
43b1aba
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
2 times, most recently
from
2301223 to
bad2e82
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
from
43b1aba to
1481bf4
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
bad2e82 to
88c89be
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
from
1481bf4 to
1bb0674
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
88c89be to
9080db5
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
from
1bb0674 to
0db6fd5
Compare
|
offbyone
force-pushed
the
o1/golang-port/3
branch
from
9080db5 to
595b8e0
Compare
offbyone
force-pushed
the
o1/golang-port/4
branch
2 times, most recently
from
4ac3d33 to
6870d1a
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
595b8e0 to
b2541a8
Compare
…-owned multipart Port three storage backends used for migration archive upload: ## Azure Blob Storage (pkg/storage/azure/) - BlobService interface for testability over Azure SDK - Upload: creates container migration-archives-<uuid>, uploads blob with 4MB block size, generates SAS URL (48h read-only expiry) - Download: HTTP GET returning bytes - Progress logging throttled to every 10 seconds - 15 tests ## AWS S3 (pkg/storage/aws/) - S3Uploader/S3Presigner interfaces for testability - Upload from stream or file path, returns 48h pre-signed URL - Session token support (SessionCredentials vs BasicCredentials) - Timeout detection wrapped as user-friendly error - Progress logging with 10-second throttle - 10 tests ## GitHub-owned storage multipart (pkg/storage/ghowned/) - Small archives (<= 100 MiB): single POST, returns URI from JSON - Large archives: 3-phase multipart (Start POST → Part PATCHes with Location header following → Complete PUT) - Configurable part size via env var (min 5 MiB) - UserError wrapping for all failure modes - 15 tests Dependencies added: azure-sdk-for-go/sdk/storage/azblob, aws-sdk-go-v2, google/uuid. All 40 tests passing, 0 lint issues.
- Archive uploader (pkg/archive/uploader.go): Priority-based routing — GitHub-owned > AWS > Azure > error. Validates credentials and dispatches to appropriate storage client. 10 tests. - GHES version checker (pkg/ghes/version.go): Determines if GHES instance requires blob storage credentials (< 3.8.0) or can use GitHub-owned storage (>= 3.8.0). Local semver parsing. 7 tests.
offbyone
force-pushed
the
o1/golang-port/4
branch
from
6870d1a to
be456cc
Compare
offbyone
force-pushed
the
o1/golang-port/3
branch
from
b2541a8 to
97d2347
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements cloud storage clients, archive upload orchestration, and GHES version checking.
Azure Blob Storage client (
pkg/storage/azure/) with connection string parsing, container creation, blob uploadAWS S3 client (
pkg/storage/aws/) with upload + presigned URL generationGitHub-owned multipart upload (
pkg/storage/ghowned/) — pure HTTP, small/large archive flows, configurable part sizeArchive upload orchestration (
pkg/archive/) with priority-based routing: GitHub-owned > AWS > Azure > errorGHES version checker (
pkg/ghes/) — determines storage requirements based on GHES versionConsumer-defined interfaces throughout for testability
57 tests across all new packages, all passing with
-raceDid you write/update appropriate tests
Release notes updated (if appropriate)— not user-facing yet (parallel Go port)Appropriate logging output
Issue linked— tracking via PR stackDocs updated (or issue created)— internal packages, covered by godocNew package licenses are added to
ThirdPartyNotices.txt(if applicable)PR Stack (Go port)