feat: add email-server-worker with clean template architecture
Add new email-server-worker project implementing a self-scheduling background worker pattern with HTTP monitoring. Removed all business-specific code from copied source, creating a clean, reusable template. Key features: - Self-scheduling worker loop with configurable interval - Graceful shutdown support (Docker-compatible) - Prometheus metrics collection - Health check endpoints (/healthcheck, /metrics, /ping) - Example worker template for easy customization - Comprehensive architecture documentation in CLAUDE.md The worker is now ready for email server implementation with no external dependencies on Evolution/MSSQL/ElasticSearch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
133
CLAUDE.md
133
CLAUDE.md
@@ -8,6 +8,7 @@ This is a multi-project repository containing:
|
||||
- **web-app/**: Next.js 14 utility bills tracking application
|
||||
- **docker-stack/**: Docker Compose configurations and deployment scripts
|
||||
- **housekeeping/**: Database backup and maintenance scripts
|
||||
- **email-server-worker/**: Background worker service with HTTP health monitoring
|
||||
|
||||
Each project is self-contained with its own dependencies.
|
||||
|
||||
@@ -29,6 +30,13 @@ All commands should be run from within the respective project directory.
|
||||
- `./db-dump--standalone.sh` - Run standalone database dump
|
||||
- See housekeeping/README.md for more details
|
||||
|
||||
**Email Server Worker** (`cd email-server-worker`):
|
||||
- `npm install` - Install dependencies
|
||||
- `npm run start` - Start development server with nodemon
|
||||
- `npm run build` - Build TypeScript to JavaScript
|
||||
- `npm run test` - Run tests with Jest in watch mode
|
||||
- `npm run run-server` - Run built server from ./build directory
|
||||
|
||||
## Deployment Commands
|
||||
|
||||
**Building Docker Image** (`cd web-app`):
|
||||
@@ -91,4 +99,127 @@ export const actionName = withUser(async (user: AuthenticatedUser, ...args) => {
|
||||
|
||||
### Testing & Code Quality
|
||||
- ESLint with Next.js and Prettier configurations
|
||||
- No specific test framework configured - check with user before assuming testing approach
|
||||
- No specific test framework configured - check with user before assuming testing approach
|
||||
|
||||
## Email Server Worker Architecture
|
||||
|
||||
The email-server-worker is a TypeScript-based background worker service that combines periodic task execution with HTTP health monitoring and metrics collection.
|
||||
|
||||
### Tech Stack
|
||||
- **Runtime**: Node.js with TypeScript
|
||||
- **Framework**: Express for HTTP endpoints
|
||||
- **Metrics**: Prometheus (prom-client) with custom PRTG adapter
|
||||
- **Testing**: Jest with TypeScript support
|
||||
|
||||
### Core Architecture: Worker Pattern
|
||||
|
||||
The service implements a **self-contained worker pattern** that runs periodic background tasks while exposing HTTP endpoints for monitoring.
|
||||
|
||||
**Entry Point** (`email-server-worker/src/entry.ts:1`):
|
||||
- Creates Express HTTP server with graceful shutdown support (stoppable)
|
||||
- Starts the worker via `startSyncWorker()` from `email-server-worker/src/workRunner.ts:134`
|
||||
- Handles SIGTERM/SIGINT for graceful shutdown (Docker-compatible)
|
||||
- Calls `disposeSyncWorker()` on shutdown to allow pending work to complete
|
||||
|
||||
**Work Runner** (`email-server-worker/src/workRunner.ts:1`):
|
||||
The work runner implements a self-scheduling loop with the following characteristics:
|
||||
|
||||
- **Self-Scheduling Loop**: After completing work, schedules next execution via `setTimeout(workRunner, PULL_INTERVAL)` at `email-server-worker/src/workRunner.ts:113`
|
||||
- **Graceful Shutdown**: Tracks pending work via Promise, allows in-flight operations to complete before shutdown
|
||||
- **Status Tracking**: Exports `workerRunnerInfo` with `status` and `lastWorkTime` for health monitoring
|
||||
- **Error Isolation**: Worker errors don't crash the process - caught, logged, and execution continues
|
||||
- **Metrics Integration**: Automatic Prometheus metrics collection (duration, success/failure counters)
|
||||
- **Single Work Instance**: Ensures only one work cycle runs at a time via `pendingWork` Promise
|
||||
|
||||
Work Runner States (WorkerRunnerStatus enum):
|
||||
- `init` - Initial state before first run
|
||||
- `beginWork` - Work cycle started
|
||||
- `workDone` - Work completed successfully
|
||||
- `disposed` - Worker stopped, no longer scheduling
|
||||
- Other states track Prometheus stats updates
|
||||
|
||||
**Worker Implementation Pattern**:
|
||||
Workers must export a `doWork` function with signature:
|
||||
```typescript
|
||||
export const doWork = async () => {
|
||||
// Perform periodic work here
|
||||
// Throw errors to increment failedRequestCounter
|
||||
// Return normally to increment successfulRequestCounter
|
||||
};
|
||||
```
|
||||
|
||||
The work runner imports and calls this function at `email-server-worker/src/workRunner.ts:88`.
|
||||
|
||||
### Key Files & Responsibilities
|
||||
|
||||
**Core Worker Files**:
|
||||
- `email-server-worker/src/entry.ts` - HTTP server setup, signal handling, worker lifecycle management
|
||||
- `email-server-worker/src/workRunner.ts` - Self-scheduling loop, graceful shutdown, metrics integration
|
||||
- `email-server-worker/src/app.ts` - Express app configuration, route registration
|
||||
- `email-server-worker/src/lib/logger.ts` - Debug logger factory (uses 'debug' package)
|
||||
|
||||
**HTTP Routes** (`email-server-worker/src/routes/`):
|
||||
- `healthcheckRouter.ts` - Health check endpoint (checks worker status via `workerRunnerInfo`)
|
||||
- `metricsRouter.ts` - Prometheus metrics endpoint
|
||||
- `prtgMetricsRouter.ts` - PRTG-compatible metrics adapter
|
||||
- `pingRouter.ts` - Simple ping/pong endpoint
|
||||
- `errorRouter.ts` - Structured error handler for expected errors
|
||||
- `finalErrorRouter.ts` - Catch-all error handler for unexpected errors
|
||||
|
||||
**Infrastructure**:
|
||||
- `email-server-worker/src/lib/metricsCounters.ts` - Prometheus counter/histogram definitions
|
||||
- `email-server-worker/src/lib/initTools.ts` - Utility functions (coalesce, etc.)
|
||||
- `email-server-worker/src/lib/serializeError.ts` - Error serialization for logging
|
||||
- `email-server-worker/src/lib/Prometheus2Prtg.ts` - Converts Prometheus metrics to PRTG XML format
|
||||
|
||||
### Environment Variables
|
||||
|
||||
**Required**:
|
||||
- `PULL_INTERVAL` - Milliseconds between work cycles (default: "10000")
|
||||
|
||||
**Optional**:
|
||||
- `PORT` - HTTP server port (default: "3000")
|
||||
- `PROMETHEUS_APP_LABEL` - App label for Prometheus metrics (default: "evo-open-table-sync-svc")
|
||||
- `PROMETHEUS_HISTOGRAM_BUCKETS` - Histogram bucket sizes (default: "0.1, 0.5, 1, 5, 10")
|
||||
- `DEBUG` - Debug namespaces for console logging (e.g., "server:server")
|
||||
- `ENV` - Environment mode: "dev", "jest" (affects logging behavior)
|
||||
|
||||
### Creating a New Worker
|
||||
|
||||
To implement a new worker task:
|
||||
|
||||
1. **Create worker file** (e.g., `email-server-worker/src/myWorker.ts`):
|
||||
```typescript
|
||||
export const doWork = async () => {
|
||||
// Implement your periodic task here
|
||||
logger.info("Work Title", "Work completed successfully");
|
||||
|
||||
// Throw errors to mark as failed:
|
||||
// throw new Error("Something went wrong");
|
||||
};
|
||||
```
|
||||
|
||||
2. **Update `workRunner.ts`** import at line 6:
|
||||
```typescript
|
||||
import { doWork } from "./myWorker";
|
||||
```
|
||||
|
||||
3. **Add environment variables** to `email-server-worker/src/types/environment.d.ts` as needed
|
||||
|
||||
4. **Update `package.json` metadata** if the service purpose changes (name, description)
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
- Uses `stoppable` library for graceful shutdown (10-second timeout before force-close)
|
||||
- Health check endpoint at `/healthcheck` verifies worker is running and not stalled
|
||||
- Prometheus metrics at `/metrics` for monitoring
|
||||
- PRTG-compatible metrics at `/prtg` for legacy monitoring systems
|
||||
- Graceful shutdown ensures work in progress completes before container stops
|
||||
|
||||
### Testing
|
||||
|
||||
- **Framework**: Jest with esbuild-jest for TypeScript
|
||||
- **Test Location**: `email-server-worker/tests/`
|
||||
- **Mocks**: Common mocks in `email-server-worker/tests/__mocks__/` (prom-client)
|
||||
- **Test Pattern**: Co-located with source in `tests/` mirroring `src/` structure
|
||||
- **Run Tests**: `npm run test` (watch mode)
|
||||
Reference in New Issue
Block a user