Securely Handle File Uploads and Cloud Storage
Secure file uploads and cloud storage - S3, R2, presigned URLs - covering magic-byte checks and path traversal fixes.
Why it matters
Implement robust file upload and cloud storage solutions, ensuring security and performance. This asset specializes in direct uploads via presigned URLs to services like S3 and Cloudflare R2, with advanced handling for large files and image optimization.
Outcomes
What it gets done
Implement secure file uploads using presigned URLs
Handle large file uploads efficiently without buffering
Validate file types using magic bytes, not just extensions
Optimize images post-upload
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/ag-file-uploads | bash Overview
File Uploads & Storage
Covers secure file upload and cloud storage patterns for S3 and Cloudflare R2: magic-byte type validation, size limits, path-traversal-safe filenames, and presigned URL scoping. Use whenever implementing file upload, S3/R2 storage, presigned URLs, multipart upload, or image upload handling.
What it does
File Uploads & Storage is an expert skill for handling file uploads and cloud storage - S3, Cloudflare R2, presigned URLs, multipart uploads, and image optimization - built around four core principles: never trust client file type claims, use presigned URLs for direct uploads instead of server proxying, stream large files rather than buffering them, and validate on upload while optimizing after. It documents four "Sharp Edges" security failure modes with severity, symptom, and fix. Trusting client-provided file type (CRITICAL) - a renamed malware.exe passes an extension check; the fix is checking magic bytes via the file-type library rather than the extension or Content-Type header:
import { fileTypeFromBuffer } from "file-type";
async function validateImage(buffer: Buffer) {
const type = await fileTypeFromBuffer(buffer);
const allowedTypes = ["image/jpeg", "image/png", "image/webp"];
if (!type || !allowedTypes.includes(type.mime)) {
throw new Error("Invalid file type");
}
return type;
}
No upload size restrictions (HIGH) - unbounded uploads exhaust memory/disk or run up storage bills; the fix is setting maxFileSize limits in Formidable/Multer, a client-side pre-check, and a ContentLength constraint on presigned PutObjectCommand requests. User-controlled filenames enabling path traversal (CRITICAL) - a filename like "../../../etc/passwd" used directly can overwrite system files; the fix is path.basename() plus either character sanitization or generating an entirely new crypto.randomUUID()-based name with an allow-listed extension. Presigned URLs shared or cached incorrectly (MEDIUM) - a private file's presigned URL cached by a CDN stays accessible beyond its intended scope; the fix is a short expiresIn window plus Cache-Control: no-store headers on the API response, or CloudFront signed URLs for finer control.
When to use - and when NOT to
Use this skill whenever the user mentions or implies file upload, S3, R2, presigned URL, multipart upload, image upload, or cloud storage. It delegates image-delivery/CDN optimization to performance-optimization and file-metadata database schema questions to postgres-wizard rather than handling those itself.
Inputs and outputs
Given a file-upload implementation task, the skill's validation checks flag two critical anti-patterns directly: checking only the file extension instead of magic bytes, and using a user-supplied filename directly in a file path instead of sanitizing it. Output is upload-handling code that validates real file type via magic-byte inspection, enforces size limits both client- and server-side, generates safe non-traversable filenames, and controls presigned-URL exposure with short expiry and no-cache response headers.
Integrations
Named tools and libraries: the file-type package (fileTypeFromBuffer/fileTypeFromStream) for real content-type detection, Formidable and Multer for server-side upload size limits, AWS S3's PutObjectCommand and getSignedUrl for presigned uploads and downloads, Node's path and crypto modules for filename sanitization, and CloudFront signed URLs as an alternative distribution-control mechanism.
Who it's for
Backend engineers implementing file-upload and cloud-storage features who need to avoid the specific, named security failure modes around file-type spoofing, unbounded upload size, path traversal, and presigned-URL leakage - not just a working upload flow, but a secure one.
Source README
File Uploads & Storage
Expert at handling file uploads and cloud storage. Covers S3,
Cloudflare R2, presigned URLs, multipart uploads, and image
optimization. Knows how to handle large files without blocking.
Role: File Upload Specialist
Careful about security and performance. Never trusts file
extensions. Knows that large uploads need special handling.
Prefers presigned URLs over server proxying.
Principles
- Never trust client file type claims
- Use presigned URLs for direct uploads
- Stream large files, never buffer
- Validate on upload, optimize after
Sharp Edges
Trusting client-provided file type
Severity: CRITICAL
Situation: User uploads malware.exe renamed to image.jpg. You check
extension, looks fine. Store it. Serve it. Another user
downloads and executes it.
Symptoms:
- Malware uploaded as images
- Wrong content-type served
Why this breaks:
File extensions and Content-Type headers can be faked.
Attackers rename executables to bypass filters.
Recommended fix:
CHECK MAGIC BYTES
import { fileTypeFromBuffer } from "file-type";
async function validateImage(buffer: Buffer) {
const type = await fileTypeFromBuffer(buffer);
const allowedTypes = ["image/jpeg", "image/png", "image/webp"];
if (!type || !allowedTypes.includes(type.mime)) {
throw new Error("Invalid file type");
}
return type;
}
// For streams
import { fileTypeFromStream } from "file-type";
const type = await fileTypeFromStream(readableStream);
No upload size restrictions
Severity: HIGH
Situation: No file size limit. Attacker uploads 10GB file. Server runs
out of memory or disk. Denial of service. Or massive
storage bill.
Symptoms:
- Server crashes on large uploads
- Massive storage bills
- Memory exhaustion
Why this breaks:
Without limits, attackers can exhaust resources. Even
legitimate users might accidentally upload huge files.
Recommended fix:
SET SIZE LIMITS
// Formidable
const form = formidable({
maxFileSize: 10 * 1024 * 1024, // 10MB
});
// Multer
const upload = multer({
limits: { fileSize: 10 * 1024 * 1024 },
});
// Client-side early check
if (file.size > 10 * 1024 * 1024) {
alert("File too large (max 10MB)");
return;
}
// Presigned URL with size limit
const command = new PutObjectCommand({
Bucket: BUCKET,
Key: key,
ContentLength: expectedSize, // Enforce size
});
User-controlled filename allows path traversal
Severity: CRITICAL
Situation: User uploads file named "../../../etc/passwd". You use
filename directly. File saved outside upload directory.
System files overwritten.
Symptoms:
- Files outside upload directory
- System file access
Why this breaks:
User input should never be used directly in file paths.
Path traversal sequences can escape intended directories.
Recommended fix:
SANITIZE FILENAMES
import path from "path";
import crypto from "crypto";
function safeFilename(userFilename: string): string {
// Extract just the base name
const base = path.basename(userFilename);
// Remove any remaining path chars
const sanitized = base.replace(/[^a-zA-Z0-9.-]/g, "_");
// Or better: generate new name entirely
const ext = path.extname(userFilename).toLowerCase();
const allowed = [".jpg", ".png", ".pdf"];
if (!allowed.includes(ext)) {
throw new Error("Invalid extension");
}
return crypto.randomUUID() + ext;
}
// Never do this
const path = "uploads/" + req.body.filename; // DANGER!
// Do this
const path = "uploads/" + safeFilename(req.body.filename);
Presigned URL shared or cached incorrectly
Severity: MEDIUM
Situation: Presigned URL for private file returned in API response.
Response cached by CDN. Anyone with cached URL can access
private file for hours.
Symptoms:
- Private files accessible via cached URLs
- Access after expiry
Why this breaks:
Presigned URLs grant temporary access. If cached or shared,
access extends beyond intended scope.
Recommended fix:
CONTROL PRESIGNED URL DISTRIBUTION
// Short expiry for sensitive files
const url = await getSignedUrl(s3, command, {
expiresIn: 300, // 5 minutes
});
// No-cache headers for presigned URL responses
return Response.json({ url }, {
headers: {
"Cache-Control": "no-store, max-age=0",
},
});
// Or use CloudFront signed URLs for more control
Validation Checks
Only checking file extension
Severity: CRITICAL
Message: Check magic bytes, not just extension
Fix action: Use file-type library to verify actual type
User filename used directly in path
Severity: CRITICAL
Message: Sanitize filenames to prevent path traversal
Fix action: Use path.basename() and generate safe name
Collaboration
Delegation Triggers
- image optimization CDN -> performance-optimization (Image delivery)
- storing file metadata -> postgres-wizard (Database schema)
When to Use
- User mentions or implies: file upload
- User mentions or implies: S3
- User mentions or implies: R2
- User mentions or implies: presigned URL
- User mentions or implies: multipart
- User mentions or implies: image upload
- User mentions or implies: cloud storage
Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
FAQ
Common questions
Discussion
Questions & comments ยท 0
Sign In Sign in to leave a comment.