Build production-ready Go microservices with integrated primitives
GoBricks bundles config, HTTP, AMQP messaging, database, cache, and observability primitives for Go microservices.
Maintainer of this project? Claim this page to edit the listing.
0.55.0Add to Favorites
Why it matters
Teams hire GoBricks to ship production-grade Go microservices faster by providing pre-configured, composable building blocks for HTTP servers, databases, messaging, caching, observability, and multi-tenant architecture instead of assembling these components from scratch.
Outcomes
What it gets done
Configure HTTP servers with typed handlers, standardized responses, and trace propagation
Connect to PostgreSQL, Oracle, Redis, and RabbitMQ with unified ergonomics and lifecycle management
Implement transactional outbox and inbox patterns for reliable event-driven architectures
Set up multi-tenant resource isolation with context propagation and configuration injection
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/gaborage-go-bricks | bash Overview
Go Bricks
GoBricks is a modular Go framework bundling configuration, HTTP, AMQP messaging, database, cache, and observability primitives for production microservices. Use it to bootstrap a Go microservice with type-safe database access, transactional outbox/inbox, and multi-tenant isolation built in, not assembled by hand.
What it does
GoBricks is a modular framework of production building blocks for Go microservices: configuration, HTTP server and client, AMQP messaging, database access (PostgreSQL and Oracle), Redis caching, a job scheduler, a transactional outbox/inbox, RSA key management, JOSE body encryption, multi-tenancy, and observability, all wired together through a composable module system with explicit dependency injection via ModuleDeps.
When to use - and when NOT to
Use it to bootstrap a production Go microservice without assembling these pieces yourself: an Echo-based HTTP server with typed handlers and a standard response envelope (or a raw-response escape hatch for legacy API migration), a resilient HTTP client with retries and W3C trace propagation, AMQP messaging with a validate-once/replay-many declaration pattern, a type-safe database query builder that auto-quotes Oracle reserved words and switches placeholder styles per vendor, a transactional outbox solving the dual-write problem paired with a consumer-side inbox for exactly-once handler processing, and multi-tenant resource isolation with per-tenant database/messaging connections resolved by header, subdomain, or path segment. It requires Go 1.26 and Docker for integration tests. For partner integrations needing sign-then-encrypt payloads (Visa Token Services-style), its JOSE middleware nests JWE-of-JWS on request/response bodies - though it protects bodies, not transport, so routes still need HTTPS in production.
Inputs and outputs
Input: a layered configuration (defaults, config.yaml, config.<env>.yaml, environment variables) plus registered modules implementing Name, Init, and Shutdown, optionally adding HTTP routes, AMQP declarations, or app-wide middleware. Output: a running service exposing typed HTTP endpoints, health/readiness checks, OpenTelemetry traces and metrics, structured dual-mode logs, and scheduled/queued background work.
// cmd/main.go
package main
import (
"log"
"github.com/gaborage/go-bricks/app"
"github.com/gaborage/go-bricks/config"
)
func main() {
cfg, err := config.Load()
if err != nil {
log.Fatal(err)
}
framework, _, err := app.NewWithConfig(cfg, nil)
if err != nil {
log.Fatal(err)
}
// Register modules here
if err := framework.RegisterModule(&users.Module{}); err != nil {
log.Fatal(err)
}
if err := framework.Run(); err != nil {
log.Fatal(err)
}
}
Config injection uses config: struct tags (required, default) resolved from deps.Config.InjectInto, and configuration errors surface as actionable messages in the form config_<category>: <field> <message> <action>, distinguishing missing, invalid, and optional settings.
Integrations
The database layer's Filter API (Eq, Gt, Like, Between, InSubquery, and more) works across SELECT/UPDATE/DELETE/JOIN for both PostgreSQL and Oracle, with struct-based column extraction via db:"..." tags cached after first use; Flyway integration handles schema migrations per-vendor, with a separate go-bricks-migrate CLI fanning migrations out across tenants using credentials from AWS Secrets Manager. Redis-backed caching adds CBOR type-safe serialization, GetOrSet/CompareAndSet atomic operations, and a mock-cache testing package; the job scheduler (gocron-based) supports fixed-rate, daily, weekly, hourly, and monthly schedules with panic recovery; and observability is OpenTelemetry-first, with Zerolog structured logging split into action and trace log types for cost-controlled retention in Loki or Datadog.
Who it's for
Go teams building production microservices who want mission-critical integrations (PostgreSQL, Oracle, RabbitMQ, Flyway, Redis) with unified, type-safe ergonomics and multi-tenant support built in, rather than assembling each piece separately. Open source under the MIT license.
Source README
GoBricks
Modern building blocks for Go microservices. GoBricks brings together configuration, HTTP, messaging, database, logging and observability primitives that teams need to ship production-grade services fast.
Table of Contents
- Why GoBricks?
- Developer Resources
- Feature Overview
- Quick Start
- Configuration
- Error Handling and Diagnostics
- Modules and Application Structure
- HTTP Server
- HTTP Client
- Messaging
- Database
- Cache
- Scheduler and Outbox
- KeyStore
- JOSE (Sign-then-Encrypt Bodies)
- Multi-Tenant Implementation
- Observability and Operations
- Examples
- Contributing
- License
Why GoBricks?
- Production-ready defaults for the boring-but-essential pieces (server, logging, configuration, tracing).
- Composable module system that keeps HTTP, database, and messaging concerns organized.
- Mission-critical integrations (PostgreSQL, Oracle, RabbitMQ, Flyway) with unified ergonomics.
- Modern Go practices with type safety, performance optimizations, and Go 1.26 features.
- Extensible design that works with modern Go idioms and the wider ecosystem.
Developer Resources
For Contributors & Framework Developers:
- CLAUDE.md - Comprehensive development guide with architecture, commands, testing, and workflows
- llms.txt - Quick code examples optimized for LLMs and copy-paste development
- CONTRIBUTING.md - Coding standards, tooling, and contribution workflow
For Application Developers:
- go-bricks-demo-project - Complete working examples and patterns
- MULTI_TENANT.md - Multi-tenant architecture and implementation guide
- Go Reference - Complete API documentation
Quick Commands:
make check # Pre-commit: fmt + lint + test
make test-integration # Integration tests (requires Docker)
go test -run TestName # Run specific test
Feature Overview
- Modular architecture with explicit dependencies and lifecycle hooks
- Echo-based HTTP server with typed handlers, standardized response envelopes, and raw response mode for legacy API migration
- Production-ready HTTP client with retries, exponential backoff, W3C trace propagation, and interceptor chains
- AMQP messaging with validate-once, replay-many pattern, auto-scaling consumer concurrency, and panic recovery
- Configuration loader merging defaults, YAML, and environment variables with struct-based injection (
config:tags) - Multi-database support for PostgreSQL and Oracle with struct-based column extraction and type-safe query builders
- Named databases for accessing multiple databases (including mixed vendors) in single-tenant mode
- Redis cache integration with type-safe serialization, multi-tenant isolation, and automatic lifecycle management
- Transactional outbox for reliable at-least-once event publishing (dual-write problem solved)
- Consumer-side inbox for exactly-once transactional event processing (dedup ledger written atomically with the handler's DB writes via
InboxProcessor.ProcessOnce) - Job scheduler with gocron, overlapping prevention, panic recovery, and system APIs
- Named RSA key pair and symmetric secret management from DER/raw files or base64 environment variables
- JOSE middleware for nested JWE-of-JWS protection on HTTP request/response bodies (Visa-style integrations)
- Multi-tenant architecture with complete resource isolation and context propagation
- Flyway migration integration for schema evolution
- Observability with W3C trace propagation, custom metrics, dual-mode logs, and health endpoints
- Enterprise-grade quality with comprehensive linting, 80%+ test coverage, and security scanning
Quick Start
Requirements
- Go 1.26 required
- Modern Go toolchain with module support
- Docker Desktop or Docker Engine (integration tests only)
Install
go mod init your-service
go get github.com/gaborage/go-bricks@latest
Bootstrap an application
// cmd/main.go
package main
import (
"log"
"github.com/gaborage/go-bricks/app"
"github.com/gaborage/go-bricks/config"
)
func main() {
cfg, err := config.Load()
if err != nil {
log.Fatal(err)
}
framework, _, err := app.NewWithConfig(cfg, nil)
if err != nil {
log.Fatal(err)
}
// Register modules here
if err := framework.RegisterModule(&users.Module{}); err != nil {
log.Fatal(err)
}
if err := framework.Run(); err != nil {
log.Fatal(err)
}
}
Configuration file
app:
name: "my-service"
version: "v1.0.0"
env: "development"
rate:
limit: 200
server:
port: 8080
database:
type: postgresql
host: localhost
port: 5432
database: mydb
username: postgres
password: password
cache:
enabled: true
type: redis
redis:
host: localhost
port: 6379
database: 0
poolsize: 10
log:
level: info
pretty: true
GoBricks loads defaults → config.yaml → config.<env>.yaml → environment variables. app.env controls the environment suffix and defaults to development.
Configuration
GoBricks uses Koanf for configuration management with layered loading: defaults → config.yaml → config.<env>.yaml → environment variables.
Access Patterns
cfg, err := config.Load()
if err != nil {
log.Fatal(err)
}
// Simple values with defaults
host := cfg.String("server.host", "0.0.0.0")
port := cfg.Int("server.port", 8080)
// Required values with validation
apiKey, err := cfg.RequiredString("custom.api.key")
if err != nil {
return fmt.Errorf("missing api key: %w", err)
}
// Structured configuration under custom.*
var custom struct {
FeatureFlag bool `koanf:"feature.flag"`
Endpoint string `koanf:"api.endpoint"`
}
_ = cfg.Unmarshal("custom", &custom)
Config Injection
Inject configuration directly into structs using config: tags with automatic validation:
type ServiceConfig struct {
APIKey string `config:"custom.api.key" required:"true"`
Timeout time.Duration `config:"custom.api.timeout" default:"30s"`
Retries int `config:"custom.api.retries" default:"3"`
}
func (m *Module) Init(deps *ModuleDeps) error {
var cfg ServiceConfig
if err := deps.Config.InjectInto(&cfg); err != nil {
return err // Fails fast if required keys are missing
}
m.service = NewService(cfg)
return nil
}
Supported tags: config:"key.path" (required), required:"true" (fail if missing), default:"value" (fallback).
Supported types: string, int, int64, float64, bool, time.Duration, []string (comma-separated via env/default, native sequence via YAML).
Environment Variables
Environment variables use uppercase with underscores and automatically map to dot notation:
DATABASE_HOST=prod-db.company.com # maps to database.host
DATABASE_SERVICE_NAME=FREEPDB1 # maps to database.service.name
CUSTOM_API_TIMEOUT=30s # maps to custom.api.timeout
See the config-injection example for advanced patterns.
Error Handling and Diagnostics
Clear, actionable error messages for configuration and startup failures.
Error Format
config_<category>: <field> <message> <action>
Categories: missing (required field), invalid (bad value), not_configured (optional feature), connection (resource failure)
Examples
config_missing: database.host required set DATABASE_HOST env var or add database.host to config.yaml
config_invalid: database.port invalid port; must be between 1 and 65535 must be one of: 1-65535
config_not_configured: messaging.broker.url (optional) to enable: set MESSAGING_BROKER_URL env var or add messaging.broker.url to config.yaml
Features
- Dual paths: shows both env var and YAML path
- Semantic distinction: differentiates missing, invalid, and optional
- Multi-tenant context: includes tenant ID when applicable
- No error chains: single clear message per issue
Modules and Application Structure
Module interface
type Module interface {
Name() string
Init(deps *ModuleDeps) error
Shutdown() error
}
// Optional — implement to register HTTP routes (detected via duck-typing at startup)
type RouteRegisterer interface {
RegisterRoutes(hr *server.HandlerRegistry, r server.RouteRegistrar)
}
// Optional — implement to declare AMQP exchanges, queues, and consumers
type MessagingDeclarer interface {
DeclareMessaging(decls *messaging.Declarations)
}
// Optional — implement to contribute app-wide middleware that runs once per
// request (after tenant resolution, before handlers; no per-route opt-out)
type GlobalMiddlewareRegisterer interface {
GlobalMiddleware() []server.MiddlewareFunc
}
ModuleDeps injects shared services into every module:
type ModuleDeps struct {
Logger logger.Logger // Structured logging
Config *config.Config // Configuration access
Tracer trace.Tracer // Distributed tracing (no-op if disabled)
MeterProvider metric.MeterProvider // Custom metrics (no-op if disabled)
Scheduler JobRegistrar // Job scheduling (nil if no scheduler module)
Outbox OutboxPublisher // Transactional event publishing (nil if disabled)
Inbox InboxProcessor // Durable consumer-side idempotency (nil if inbox module not registered)
KeyStore KeyStore // Named RSA key pairs (nil if not configured)
DB func(ctx context.Context) (database.Interface, error) // Tenant-aware database
DBByName func(ctx context.Context, name string) (database.Interface, error) // Named databases
Messaging func(ctx context.Context) (messaging.AMQPClient, error) // Tenant-aware messaging
Cache func(ctx context.Context) (cache.Cache, error) // Tenant-aware cache
}
Registering a module
func register(framework *app.App) error {
return framework.RegisterModule(&users.Module{})
}
Init is called once to capture dependencies and Shutdown releases resources. Route registration and messaging are opt-in: if your module implements RouteRegisterer, the framework calls RegisterRoutes to attach HTTP handlers; if it implements MessagingDeclarer, DeclareMessaging is called to declare AMQP infrastructure (validated once, replayed per-tenant). Modules that don't need HTTP or AMQP simply omit the interface. The framework ensures proper lifecycle ordering and error handling across all module hooks.
HTTP Server
Echo v5-based server with typed handlers, standardized response envelopes, and comprehensive middleware (logging, recovery, rate limiting, CORS).
Typed Handlers
func (h *Handler) createUser(req CreateReq, ctx server.HandlerContext) (server.Result[User], server.IAPIError) {
user := h.svc.Create(req)
return server.Created(user), nil
}
Request structs use tags for binding/validation (param, query, header, validate). Responses follow consistent {data:…, meta:…} envelope structure. Use server.WithRawResponse() to bypass the envelope for legacy API migration (Strangler Fig pattern).
Routing Configuration
server:
path:
base: "/api/v1" # All routes prefixed
health: "/health" # Liveness endpoint
ready: "/ready" # Readiness endpoint
Override with environment variables: SERVER_PATH_BASE, SERVER_PATH_HEALTH, SERVER_PATH_READY.
HTTP Client
Production-ready HTTP client with built-in observability and resilience.
client := httpclient.NewBuilder(logger).
WithTimeout(10 * time.Second).
WithRetries(3, 500 * time.Millisecond).
WithDefaultHeader("Accept", "application/json").
WithW3CTrace(true).
Build()
resp, err := client.Get(ctx, &httpclient.Request{
URL: "https://api.example.com/users",
})
Features: Builder pattern, W3C trace propagation, exponential backoff with full jitter, request/response interceptor chains, structured logging. Methods: Get, Post, Put, Patch, Delete, Do.
See llms.txt for more examples and go-bricks-demo-project for working demos.
Messaging
AMQP/RabbitMQ support with validate-once, replay-many declaration pattern:
- Declarative Infrastructure: Exchanges, queues, bindings, publishers, and consumers declared as data structures, validated upfront
- Multi-Tenant Isolation: Declarations replayed to tenant-specific registries for complete separation
- Auto-Reconnection: Exponential backoff for resilient operations
- Context Propagation: Tenant IDs and trace information flow automatically through messaging
- Consumer Concurrency: Auto-scaling workers (
NumCPU * 4) with configurable overrides and resource safeguards
Database
Unified interface supporting PostgreSQL and Oracle with vendor-specific optimizations and type-safe query building across all operations.
Type-Safe Filter API
GoBricks provides a composable Filter API that works across SELECT, UPDATE, DELETE, and JOIN operations. The query-builder snippets below are fragments: they assume db is the request's database.Interface, obtained via db, err := deps.DB(ctx).
// qb is a *database.QueryBuilder instance (there is no "qb" package):
qb := database.NewQueryBuilder(db.DatabaseType()) // db from deps.DB(ctx)
// Build reusable filters
f := qb.Filter()
// SELECT with filters
users := qb.Select("id", "name", "email").
From("users").
Where(f.Eq("status", "active")).
Where(f.Gt("created_at", startDate))
// UPDATE with filters
qb.Update("users").
Set("status", "inactive").
Where(f.Lt("last_login", cutoffDate))
// DELETE with filters
qb.Delete("users").
Where(f.Eq("status", "deleted")).
Where(f.Lt("deleted_at", retentionDate))
// JOIN with type-safe column comparisons
jf := qb.JoinFilter()
query := qb.Select("u.name", "p.bio").
From("users u").
LeftJoinOn("profiles p", jf.EqColumn("u.id", "p.user_id")).
Where(f.NotNull("p.bio"))
Filter Methods: Eq, NotEq, Lt, Lte, Gt, Gte, In, NotIn, Like, Null, NotNull, Between, Exists, NotExists, InSubquery, And, Or, Not, Raw
JoinFilter Methods: EqColumn, NotEqColumn, LtColumn, LteColumn, GtColumn, GteColumn, Eq, NotEq, Lt, Lte, Gt, Gte, In, NotIn, Between, Like, Null, NotNull, And, Or, Raw
All methods automatically handle:
- Vendor-specific quoting (Oracle reserved words like
NUMBER,DATE) - Placeholder formatting (
$1for PostgreSQL,:1for Oracle) - Type safety at compile time with refactor-friendly interfaces
Struct-Based Column Extraction
Eliminate column repetition using db:"column_name" struct tags. Columns are extracted once via reflection and cached forever (~26ns access):
type User struct {
ID int64 `db:"id"`
Name string `db:"name"`
Level int `db:"level"` // Oracle reserved word — auto-quoted
}
qb := database.NewQueryBuilder(db.DatabaseType()) // db from deps.DB(ctx)
cols := qb.Columns(&User{})
f := qb.Filter()
query := qb.Select(cols.Cols("ID", "Name")). // []string flattened by Select
From("users").
Where(f.Eq(cols.Col("Level"), 5))
// Oracle: SELECT id, name FROM users WHERE "level" = :1
// PostgreSQL: SELECT id, name FROM users WHERE level = $1
Benefits: DRY (define columns once), type-safe (panics on typos), vendor-aware (auto-quotes Oracle reserved words), refactor-friendly, zero overhead after first use. See CLAUDE.md for table aliases, INSERT patterns, and advanced examples.
Database Support
- PostgreSQL:
$1, $2placeholders, pgx driver, advanced features - Oracle:
:1, :2placeholders, reserved word quoting, service name/SID support, SEQUENCE objects, UDT registration
Features
- Connection pooling with production-safe defaults and health monitoring
- Named databases for multi-database single-tenant setups (
deps.DBByName(ctx, "legacy")) - Flyway integration for schema migrations (see below)
- Performance tracking via OpenTelemetry
- Type-safe query builders prevent runtime SQL errors
Schema Migrations (Flyway)
The migration package wraps the Flyway CLI for repeatable schema versioning. It auto-selects the per-vendor migration scripts and config (flyway/flyway-<vendor>.conf, migrations/<vendor>/) and surfaces Migrate, Info, and Validate commands plus RunMigrationsAtStartup for opt-in startup migrations:
m := migration.NewFlywayMigrator(cfg, logger)
_, err := m.Migrate(ctx, nil) // nil → use vendor-aware defaults
if err != nil {
log.Fatal(err)
}
Flyway must be installed and on PATH (the path is validated against a safe-path allowlist). Defaults can be overridden via a *migration.Config struct (FlywayPath, ConfigPath, MigrationPath, Timeout, DryRun).
Multi-Tenant Migrations (CI/CD)
For multi-tenant fleets, GoBricks ships a CLI (go-bricks-migrate in tools/migration/) and a library entry point (migration.MigrateAll) that fan migrations out across every tenant. Tenant IDs come from a control-plane HTTP API conforming to a pre-defined contract that uses the standard go-bricks APIResponse envelope; tenant database credentials are recovered from AWS Secrets Manager at the documented gobricks/migrate/<tenant_id> naming convention.
go-bricks-migrate migrate \
--source-url https://control-plane.example.com/api \
--aws-region us-east-1 \
--secrets-prefix gobricks/migrate/
See wiki/multi_tenant_migration.md for the full HTTP contract, IAM policy, secret payload formats (canonical and AWS RDS rotation fallback), CI/CD recipe, and library usage. Architectural rationale lives in ADR-018.
Cache
GoBricks provides Redis-based caching with type-safe serialization, multi-tenant isolation, and automatic lifecycle management through the CacheManager.
Quick Example
import (
"context"
"time"
"github.com/gaborage/go-bricks/cache"
)
type UserService struct {
getCache func(context.Context) (cache.Cache, error)
}
func (s *UserService) GetUser(ctx context.Context, id int64) (*User, error) {
// Cache(ctx) resolves the tenant from context automatically
c, err := s.getCache(ctx)
if err != nil {
return nil, err
}
cacheKey := fmt.Sprintf("user:%d", id)
// Try cache first
data, err := c.Get(ctx, cacheKey)
if err == nil {
return cache.Unmarshal[*User](data)
}
// Cache miss - query database
user, err := s.queryDatabase(ctx, id)
if err != nil {
return nil, err
}
// Store in cache with TTL
data, _ = cache.Marshal(user)
c.Set(ctx, cacheKey, data, 5*time.Minute)
return user, nil
}
Key Features
- Type-Safe Serialization: CBOR encoding with compile-time type safety
- Multi-Tenant Isolation: Automatic tenant context resolution
- Lifecycle Management: Lazy initialization, LRU eviction (max 100 tenants), idle cleanup (15m default)
- Atomic Operations:
GetOrSetfor deduplication,CompareAndSetfor distributed locking - Performance: <1ms latency for Get/Set, 100k ops/sec throughput
- Observability: OpenTelemetry spans, metrics, and health checks
Configuration
cache:
enabled: true
type: redis
manager:
maxsize: 100 # Max tenant cache instances (LRU-evicted)
idlettl: 15m # Close caches idle longer than this
cleanupinterval: 5m # How often the cleanup goroutine runs
redis:
host: localhost
port: 6379
password: "" # Set via CACHE_REDIS_PASSWORD env var (GoBricks does not expand ${VAR} in YAML)
database: 0
poolsize: 10 # Default: 10
The manager.* keys tune the per-tenant lifecycle; tune them up if you serve many tenants or want connections to live longer between bursts.
Operations
| Operation | Method | Use Case |
|---|---|---|
| Basic read | Get(ctx, key) |
Query result caching |
| Basic write | Set(ctx, key, value, ttl) |
Store computed data with TTL |
| Deduplication | GetOrSet(ctx, key, value, ttl) |
Idempotency keys, atomic SET NX |
| Distributed lock | CompareAndSet(ctx, key, expected, new, ttl) |
Cross-pod job coordination |
| Type-safe store | Marshal(v) + Set() |
Struct serialization (CBOR) |
| Type-safe retrieve | Get() + Unmarshal[T](data) |
Struct deserialization |
For comprehensive examples, see the Cache Operations section in llms.txt.
Testing with Mock Cache
GoBricks provides cache/testing package for unit tests without Redis dependencies:
import cachetest "github.com/gaborage/go-bricks/cache/testing"
func TestMyService(t *testing.T) {
mockCache := cachetest.NewMockCache()
// Configure mock behavior
mockCache.WithGetFailure(cache.ErrNotFound)
// Inject into service
deps := &app.ModuleDeps{
Cache: func(ctx context.Context) (cache.Cache, error) {
return mockCache, nil
},
}
svc := NewService(deps)
// Run tests
result, err := svc.GetUser(ctx, 123)
// Assert cache operations
cachetest.AssertOperationCount(t, mockCache, "Get", 1)
}
Features: Fluent configuration API, operation tracking, 17 assertion helpers, TTL expiration testing, multi-tenant support.
See also: database/testing for similar patterns.
Scheduler and Outbox
Job Scheduler
gocron-based job scheduling integrated with the module system. Lazy initialization, overlapping prevention (mutex per job), automatic panic recovery with stack trace logging, and system APIs (GET /_sys/job, POST /_sys/job/:jobId).
func (m *Module) Init(deps *app.ModuleDeps) error {
return deps.Scheduler.DailyAt("cleanup-job", &CleanupJob{}, scheduler.ParseTime("03:00"))
}
// Implement the Executor interface
type CleanupJob struct{}
func (j *CleanupJob) Execute(ctx scheduler.JobContext) error {
// ctx provides: JobID(), TriggerType(), Logger(), DB(), Messaging(), Config()
return nil
}
Schedule types: FixedRate(duration), DailyAt(time), WeeklyAt(weekday, time), HourlyAt(minute), MonthlyAt(dayOfMonth, time).
Transactional Outbox
Solves the dual-write problem: events are written to an outbox table in the same database transaction as business data, then reliably delivered to the message broker by a background relay. Delivery guarantee: at-least-once (consumers must be idempotent).
func (s *OrderService) CreateOrder(ctx context.Context, req CreateOrderReq) error {
db, err := s.getDB(ctx)
if err != nil { return err }
tx, err := db.Begin(ctx)
if err != nil { return err }
defer tx.Rollback(ctx)
// 1. Write business data
_, err = tx.Exec(ctx, "INSERT INTO orders (id, customer_id) VALUES ($1, $2)",
req.ID, req.CustomerID)
if err != nil { return err }
// 2. Write event to outbox (SAME transaction — atomic!)
// Publish returns the event UUID — propagated as the x-outbox-event-id
// AMQP header so downstream consumers can deduplicate.
payload, _ := json.Marshal(OrderCreatedEvent{OrderID: req.ID})
eventID, err := s.outbox.Publish(ctx, tx, &app.OutboxEvent{
EventType: "order.created",
AggregateID: fmt.Sprintf("order-%d", req.ID),
Payload: payload,
Exchange: "order.events",
})
if err != nil { return err }
s.logger.Info().Str("event_id", eventID).Msg("order.created enqueued")
return tx.Commit(ctx) // Event GUARANTEED to reach the broker eventually
}
The relay job polls for pending events every pollinterval (default 5s), publishes them to AMQP with an x-outbox-event-id header for deduplication, and a cleanup job removes published events after retentionperiod (default 72h). See CLAUDE.md for full configuration options.
Inbox (consumer side): the inbox package is the outbox's mirror - deps.Inbox.ProcessOnce(ctx, eventID, fn) records the event id in a ledger atomically with the handler's writes, giving exactly-once processing of the handler's transactional work on top of the broker's at-least-once delivery. Effects outside that DB transaction (HTTP calls, emails, broker publishes) can still repeat on redelivery - keep them idempotent. Register inbox.NewModule(); use the x-outbox-event-id header as the dedup key. See CLAUDE.md and wiki/outbox.md.
KeyStore
Named RSA key pair management for encryption and signing. Keys are loaded at startup from DER files or base64-encoded environment variables.
func (m *Module) Init(deps *app.ModuleDeps) error {
// Sign a JWT
privateKey, err := deps.KeyStore.PrivateKey("signing")
if err != nil { return err }
// Verify a signature
publicKey, err := deps.KeyStore.PublicKey("signing")
if err != nil { return err }
return nil
}
Configuration:
keystore:
keys:
signing:
public:
file: certs/signing_public.der # Local dev: path to DER file
private:
file: certs/signing_private.der
encryption:
public:
value: "" # Cloud/EKS: set via KEYSTORE_KEYS_ENCRYPTION_PUBLIC_VALUE env var (base64-encoded DER)
private:
value: "" # Set via KEYSTORE_KEYS_ENCRYPTION_PRIVATE_VALUE env var (base64-encoded DER)
Each key pair has a public and private source. For required sources, set exactly one of file: (DER path) or value: (base64-encoded DER, typically referencing an environment variable). For verification-only services, the private entry may be omitted.
See keystore/ package for full API documentation.
JOSE (Sign-then-Encrypt Bodies)
⚠️ Production requirement: JOSE protects request/response bodies, not the transport. Routes using JOSE middleware must still be served over HTTPS in production - TLS continues to protect URLs, headers, traffic patterns, and the trust handshake itself.
The jose package adds nested JWE-of-JWS protection on HTTP request and response bodies - designed for Visa Token Services-style integrations and any partner API that requires sign-then-encrypt outbound and decrypt-then-verify inbound on every payload. It's struct-tag opt-in: the framework discovers JOSE-protected routes at registration time, validates every kid against the keystore, and panics fast if anything is misconfigured rather than failing per-request.
type CreateTokenRequest struct {
_ struct{} `jose:"decrypt=our-signing,verify=visa-vts-verify"`
PAN string `json:"pan" validate:"required"`
}
type CreateTokenResponse struct {
_ struct{} `jose:"sign=our-signing,encrypt=visa-vts-encrypt"`
Token string `json:"token"`
}
Key properties:
- Bidirectional symmetry enforced - request and response must both carry tags, or neither (registration-time check).
- Strict algorithm allowlist -
RS256/PS256for signing;RSA-OAEP-256+A256GCMfor encryption.alg=none,HS*, andRSA1_5are rejected at parse time. - Hybrid error envelope - pre-trust failures (decrypt failed, signature invalid) emit a plaintext minimal
{code,message}envelope so nothing leaks to unauthenticated peers; post-trust handler errors emit the standardAPIResponseenvelope, encrypted with the route's outbound policy. - Fail-fast at startup - every
kidis resolved against the keystore atRegisterHandlertime. Missing keys, asymmetric tags, andWithRawResponse()conflicts panic at startup, never at runtime. - Observability - spans (
jose.decode_request,jose.encode_response), failure counter (jose.failures.totalby code/direction), duration histogram (jose.operation.duration).
Wire it by registering keystore.NewModule() before any module that declares JOSE-tagged routes - app/module_registry.go then auto-injects KeyStore, logger, tracer, and meter into the middleware. See CLAUDE.md for the complete failure-mode → IAPIError table and llms.txt for end-to-end examples.
Multi-Tenant Implementation
Complete resource isolation with per-tenant database and messaging connections.
Key Features
- Tenant Resolution: Header, subdomain, path-segment, or composite (first-match chain) resolvers - plus custom strategies - all validated
- Resource Isolation: Separate database/messaging connections per tenant
- Context Propagation: Tenant ID flows automatically through all operations
- Declaration Replay: Messaging infrastructure validated once, replayed per-tenant
Quick Setup
multitenant:
enabled: true
resolver:
type: "header"
header: "X-Tenant-ID"
tenants:
tenant1:
database: { ... }
messaging: { url: "..." }
Modules access tenant resources via deps.DB(ctx) and deps.Messaging(ctx).
Custom Integration
Implement app.TenantStore to integrate with AWS Secrets Manager, HashiCorp Vault, or custom backends.
See MULTI_TENANT.md for detailed architecture and multitenant-aws example for implementation patterns.
Observability and Operations
- OpenTelemetry first: native traces, metrics, and dual-mode logs.
- Structured logging via Zerolog with OTLP export for action + trace streams.
- Tracing propagates W3C
traceparentheaders. - Metrics capture HTTP/messaging/database timings plus custom application metrics via
deps.MeterProvider. - Health endpoints:
/health(liveness) and/ready(readiness with DB/messaging checks). - Graceful shutdown coordinates servers, consumers, and background workers.
Configuring OpenTelemetry (Traces + Dual-Mode Logs)
Enable observability in configuration
observability: enabled: true service: name: checkout-api version: 1.2.3 environment: production trace: enabled: true endpoint: otel-collector:4317 protocol: grpc # http for OTLP/HTTP insecure: true # disable TLS when talking to a collector inside the VPC export: timeout: 5s logs: enabled: true endpoint: otel-collector:4317 protocol: grpc insecure: true slowrequestthreshold: 750ms # requests slower than this become WARN result_code export: timeout: 5s batch: timeout: 3s max: queue: size: 2048 batch: size: 512- All application logs default to
log.type="trace"and only WARN+ severities are exported (≈95 % volume reduction). - Middleware writes synthesized request summaries with
log.type="action"for every healthy request; they keep INFO-level retention.
- All application logs default to
Initialize the OpenTelemetry provider and hook the logger
import ( "github.com/gaborage/go-bricks/config" "github.com/gaborage/go-bricks/logger" "github.com/gaborage/go-bricks/observability" "github.com/gaborage/go-bricks/server" ) // The raw HTTP engine is no longer exposed; register middleware through a // server.RouteRegistrar (e.g. a module group from RegisterRoutes, or the // runner's RootGroup()). server.LoggerWithConfig now returns server.MiddlewareFunc. func wireLogging(cfg *config.Config, r server.RouteRegistrar) (observability.Provider, logger.Logger, error) { var obsCfg observability.Config if err := cfg.Unmarshal("observability", &obsCfg); err != nil { return nil, nil, err } obsProvider, err := observability.NewProvider(&obsCfg) if err != nil { return nil, nil, err } appLogger := logger.New(cfg.Log.Level, cfg.Log.Pretty). WithOTelProvider(obsProvider) r.Use(server.LoggerWithConfig(appLogger, server.LoggerConfig{ HealthPath: "/health", ReadyPath: "/ready", SlowRequestThreshold: obsCfg.Logs.SlowRequestThreshold, })) return obsProvider, appLogger, nil }- The bridge automatically enriches Zerolog output with
trace_id,span_id, andlog.type. server.LoggerWithConfigemits action logs and registers the WARN/ERROR hook so requests with elevated severity stay in the trace stream.
- The bridge automatically enriches Zerolog output with
Route the two log classes in your backend
- Grafana Loki:
logqlquery{log.type="action"}for request summaries,{log.type="trace", trace_id="abc"}for correlated traces. - Datadog: create indexes
@log.type:action(long retention) and@log.type:trace(short retention) to control costs.
- Grafana Loki:
Local development tip: set
observability.trace.endpoint=stdoutandobservability.logs.endpoint=stdoutto pretty-print spans and logs without running a collector.
Examples
Complete Working Examples
Explore the go-bricks-demo-project repository:
http/handlers- typed HTTP handler modulehttp/client- fluent HTTP client with retries and interceptorsoracle- Oracle insert with reserved column quotingconfig-injection- custom configuration namespace demotrace-propagation- W3C tracing demonstrationopenapi-demo- OpenAPI specification generationmultitenant-aws- multi-tenant app with AWS Secrets Manager
Quick Code Reference
See llms.txt for copy-paste-ready code snippets covering:
- Module system patterns
- HTTP handlers (POST, GET, PUT, DELETE) and raw response mode
- HTTP client with retries, interceptors, and W3C traces
- Database queries (SELECT, UPDATE, DELETE, JOINs, subqueries)
- Struct-based column extraction and named databases
- Messaging (AMQP declarations, publishers, consumers)
- Transactional outbox and job scheduler
- Configuration injection with struct tags
- Custom errors and middleware
- KeyStore for RSA key management
- Multi-tenant configuration
- Observability setup and custom metrics
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.