feat: add email-server-worker with clean template architecture
Add new email-server-worker project implementing a self-scheduling background worker pattern with HTTP monitoring. Removed all business-specific code from copied source, creating a clean, reusable template. Key features: - Self-scheduling worker loop with configurable interval - Graceful shutdown support (Docker-compatible) - Prometheus metrics collection - Health check endpoints (/healthcheck, /metrics, /ping) - Example worker template for easy customization - Comprehensive architecture documentation in CLAUDE.md The worker is now ready for email server implementation with no external dependencies on Evolution/MSSQL/ElasticSearch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
133
CLAUDE.md
133
CLAUDE.md
@@ -8,6 +8,7 @@ This is a multi-project repository containing:
|
||||
- **web-app/**: Next.js 14 utility bills tracking application
|
||||
- **docker-stack/**: Docker Compose configurations and deployment scripts
|
||||
- **housekeeping/**: Database backup and maintenance scripts
|
||||
- **email-server-worker/**: Background worker service with HTTP health monitoring
|
||||
|
||||
Each project is self-contained with its own dependencies.
|
||||
|
||||
@@ -29,6 +30,13 @@ All commands should be run from within the respective project directory.
|
||||
- `./db-dump--standalone.sh` - Run standalone database dump
|
||||
- See housekeeping/README.md for more details
|
||||
|
||||
**Email Server Worker** (`cd email-server-worker`):
|
||||
- `npm install` - Install dependencies
|
||||
- `npm run start` - Start development server with nodemon
|
||||
- `npm run build` - Build TypeScript to JavaScript
|
||||
- `npm run test` - Run tests with Jest in watch mode
|
||||
- `npm run run-server` - Run built server from ./build directory
|
||||
|
||||
## Deployment Commands
|
||||
|
||||
**Building Docker Image** (`cd web-app`):
|
||||
@@ -91,4 +99,127 @@ export const actionName = withUser(async (user: AuthenticatedUser, ...args) => {
|
||||
|
||||
### Testing & Code Quality
|
||||
- ESLint with Next.js and Prettier configurations
|
||||
- No specific test framework configured - check with user before assuming testing approach
|
||||
- No specific test framework configured - check with user before assuming testing approach
|
||||
|
||||
## Email Server Worker Architecture
|
||||
|
||||
The email-server-worker is a TypeScript-based background worker service that combines periodic task execution with HTTP health monitoring and metrics collection.
|
||||
|
||||
### Tech Stack
|
||||
- **Runtime**: Node.js with TypeScript
|
||||
- **Framework**: Express for HTTP endpoints
|
||||
- **Metrics**: Prometheus (prom-client) with custom PRTG adapter
|
||||
- **Testing**: Jest with TypeScript support
|
||||
|
||||
### Core Architecture: Worker Pattern
|
||||
|
||||
The service implements a **self-contained worker pattern** that runs periodic background tasks while exposing HTTP endpoints for monitoring.
|
||||
|
||||
**Entry Point** (`email-server-worker/src/entry.ts:1`):
|
||||
- Creates Express HTTP server with graceful shutdown support (stoppable)
|
||||
- Starts the worker via `startSyncWorker()` from `email-server-worker/src/workRunner.ts:134`
|
||||
- Handles SIGTERM/SIGINT for graceful shutdown (Docker-compatible)
|
||||
- Calls `disposeSyncWorker()` on shutdown to allow pending work to complete
|
||||
|
||||
**Work Runner** (`email-server-worker/src/workRunner.ts:1`):
|
||||
The work runner implements a self-scheduling loop with the following characteristics:
|
||||
|
||||
- **Self-Scheduling Loop**: After completing work, schedules next execution via `setTimeout(workRunner, PULL_INTERVAL)` at `email-server-worker/src/workRunner.ts:113`
|
||||
- **Graceful Shutdown**: Tracks pending work via Promise, allows in-flight operations to complete before shutdown
|
||||
- **Status Tracking**: Exports `workerRunnerInfo` with `status` and `lastWorkTime` for health monitoring
|
||||
- **Error Isolation**: Worker errors don't crash the process - caught, logged, and execution continues
|
||||
- **Metrics Integration**: Automatic Prometheus metrics collection (duration, success/failure counters)
|
||||
- **Single Work Instance**: Ensures only one work cycle runs at a time via `pendingWork` Promise
|
||||
|
||||
Work Runner States (WorkerRunnerStatus enum):
|
||||
- `init` - Initial state before first run
|
||||
- `beginWork` - Work cycle started
|
||||
- `workDone` - Work completed successfully
|
||||
- `disposed` - Worker stopped, no longer scheduling
|
||||
- Other states track Prometheus stats updates
|
||||
|
||||
**Worker Implementation Pattern**:
|
||||
Workers must export a `doWork` function with signature:
|
||||
```typescript
|
||||
export const doWork = async () => {
|
||||
// Perform periodic work here
|
||||
// Throw errors to increment failedRequestCounter
|
||||
// Return normally to increment successfulRequestCounter
|
||||
};
|
||||
```
|
||||
|
||||
The work runner imports and calls this function at `email-server-worker/src/workRunner.ts:88`.
|
||||
|
||||
### Key Files & Responsibilities
|
||||
|
||||
**Core Worker Files**:
|
||||
- `email-server-worker/src/entry.ts` - HTTP server setup, signal handling, worker lifecycle management
|
||||
- `email-server-worker/src/workRunner.ts` - Self-scheduling loop, graceful shutdown, metrics integration
|
||||
- `email-server-worker/src/app.ts` - Express app configuration, route registration
|
||||
- `email-server-worker/src/lib/logger.ts` - Debug logger factory (uses 'debug' package)
|
||||
|
||||
**HTTP Routes** (`email-server-worker/src/routes/`):
|
||||
- `healthcheckRouter.ts` - Health check endpoint (checks worker status via `workerRunnerInfo`)
|
||||
- `metricsRouter.ts` - Prometheus metrics endpoint
|
||||
- `prtgMetricsRouter.ts` - PRTG-compatible metrics adapter
|
||||
- `pingRouter.ts` - Simple ping/pong endpoint
|
||||
- `errorRouter.ts` - Structured error handler for expected errors
|
||||
- `finalErrorRouter.ts` - Catch-all error handler for unexpected errors
|
||||
|
||||
**Infrastructure**:
|
||||
- `email-server-worker/src/lib/metricsCounters.ts` - Prometheus counter/histogram definitions
|
||||
- `email-server-worker/src/lib/initTools.ts` - Utility functions (coalesce, etc.)
|
||||
- `email-server-worker/src/lib/serializeError.ts` - Error serialization for logging
|
||||
- `email-server-worker/src/lib/Prometheus2Prtg.ts` - Converts Prometheus metrics to PRTG XML format
|
||||
|
||||
### Environment Variables
|
||||
|
||||
**Required**:
|
||||
- `PULL_INTERVAL` - Milliseconds between work cycles (default: "10000")
|
||||
|
||||
**Optional**:
|
||||
- `PORT` - HTTP server port (default: "3000")
|
||||
- `PROMETHEUS_APP_LABEL` - App label for Prometheus metrics (default: "evo-open-table-sync-svc")
|
||||
- `PROMETHEUS_HISTOGRAM_BUCKETS` - Histogram bucket sizes (default: "0.1, 0.5, 1, 5, 10")
|
||||
- `DEBUG` - Debug namespaces for console logging (e.g., "server:server")
|
||||
- `ENV` - Environment mode: "dev", "jest" (affects logging behavior)
|
||||
|
||||
### Creating a New Worker
|
||||
|
||||
To implement a new worker task:
|
||||
|
||||
1. **Create worker file** (e.g., `email-server-worker/src/myWorker.ts`):
|
||||
```typescript
|
||||
export const doWork = async () => {
|
||||
// Implement your periodic task here
|
||||
logger.info("Work Title", "Work completed successfully");
|
||||
|
||||
// Throw errors to mark as failed:
|
||||
// throw new Error("Something went wrong");
|
||||
};
|
||||
```
|
||||
|
||||
2. **Update `workRunner.ts`** import at line 6:
|
||||
```typescript
|
||||
import { doWork } from "./myWorker";
|
||||
```
|
||||
|
||||
3. **Add environment variables** to `email-server-worker/src/types/environment.d.ts` as needed
|
||||
|
||||
4. **Update `package.json` metadata** if the service purpose changes (name, description)
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
- Uses `stoppable` library for graceful shutdown (10-second timeout before force-close)
|
||||
- Health check endpoint at `/healthcheck` verifies worker is running and not stalled
|
||||
- Prometheus metrics at `/metrics` for monitoring
|
||||
- PRTG-compatible metrics at `/prtg` for legacy monitoring systems
|
||||
- Graceful shutdown ensures work in progress completes before container stops
|
||||
|
||||
### Testing
|
||||
|
||||
- **Framework**: Jest with esbuild-jest for TypeScript
|
||||
- **Test Location**: `email-server-worker/tests/`
|
||||
- **Mocks**: Common mocks in `email-server-worker/tests/__mocks__/` (prom-client)
|
||||
- **Test Pattern**: Co-located with source in `tests/` mirroring `src/` structure
|
||||
- **Run Tests**: `npm run test` (watch mode)
|
||||
7
email-server-worker/.dockerignore
Normal file
7
email-server-worker/.dockerignore
Normal file
@@ -0,0 +1,7 @@
|
||||
tests
|
||||
.git
|
||||
coverage
|
||||
node_modules
|
||||
jest.config.ts
|
||||
service-tester.sh
|
||||
build-image.sh
|
||||
2
email-server-worker/.gitignore
vendored
Normal file
2
email-server-worker/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
node_modules
|
||||
build
|
||||
75
email-server-worker/Dockerfile
Normal file
75
email-server-worker/Dockerfile
Normal file
@@ -0,0 +1,75 @@
|
||||
#-------------------------------------------------------------
|
||||
# Build command: docker build . -t pr-d-registry.ngit.hr/ngit/evo-open-table-sync-svc:1.0.0
|
||||
#-------------------------------------------------------------
|
||||
|
||||
#--------------------------------------------
|
||||
# Stage: building TypeScript
|
||||
#--------------------------------------------
|
||||
FROM node:18 as build-stage
|
||||
|
||||
ENV WORKDIR=/app
|
||||
WORKDIR /app
|
||||
|
||||
# kopiram SSH key & known_hosts koji su potrebni za `npm i`
|
||||
# zato što `package.json` pri instalacija NGIT paketa koristi
|
||||
# SSH autentikaciju
|
||||
COPY _docker_assets/.ssh /root/.ssh
|
||||
RUN chmod 700 /root/.ssh/*
|
||||
|
||||
COPY ./package*.json ./
|
||||
|
||||
# instaliram pakete
|
||||
RUN npm i && npm cache clean --force
|
||||
|
||||
COPY ./tsconfig.json ./
|
||||
COPY ./src ./src
|
||||
RUN npm run build
|
||||
|
||||
#--------------------------------------------
|
||||
# Stage: instaliram produkcijski node_modules
|
||||
#--------------------------------------------
|
||||
FROM node:18 as package-stage
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./package*.json ./
|
||||
|
||||
# instaliram SAMO produkcijske
|
||||
RUN npm i --only=production && npm cache clean --force
|
||||
|
||||
#--------------------------------------------
|
||||
# Stage: priprema finalnog image-a
|
||||
#--------------------------------------------
|
||||
FROM gcr.io/distroless/nodejs:18 as assembly-stage
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ARG PORT="3000"
|
||||
ENV PORT=${PORT}
|
||||
|
||||
# prometheus config
|
||||
ARG PROMETHEUS_APP_LABEL
|
||||
ENV PROMETHEUS_APP_LABEL=${PROMETHEUS_APP_LABEL}
|
||||
|
||||
ARG PROMETHEUS_HISTOGRAM_BUCKETS
|
||||
ENV PROMETHEUS_HISTOGRAM_BUCKETS=${PROMETHEUS_HISTOGRAM_BUCKETS}
|
||||
|
||||
# (optional) logiranje na stdout (moguće opcije: "server:server", "server:metrics", "server:healthcheck" )
|
||||
ARG DEBUG
|
||||
ENV DEBUG=${DEBUG}
|
||||
|
||||
# kopiram node-modules
|
||||
COPY --from=package-stage /app/package*.json ./
|
||||
COPY --from=package-stage /app/node_modules ./node_modules
|
||||
|
||||
# kopiram buildane datoteke
|
||||
COPY --from=build-stage /app/build/ ./server
|
||||
|
||||
# server vrtim pod ograničenim "nobody" korisnikom
|
||||
USER nobody:nobody
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s \
|
||||
CMD ["/nodejs/bin/node", "./server/healthcheck.js"]
|
||||
|
||||
# pokrećem server
|
||||
CMD ["./server/entry.js"]
|
||||
@@ -1,27 +1,115 @@
|
||||
# Email Server Worker
|
||||
|
||||
This workspace contains the email server worker service for the Evidencija Režija tenant notification system.
|
||||
Background worker service with HTTP health monitoring and metrics collection.
|
||||
|
||||
## Purpose
|
||||
## Overview
|
||||
|
||||
This service manages email operations by:
|
||||
- Polling MongoDB for email status changes
|
||||
- Detecting unverified tenant emails (EmailStatus.Unverified)
|
||||
- Sending verification emails to tenants
|
||||
- Updating email status to VerificationPending
|
||||
- Sending scheduled notifications (rent due, utility bills)
|
||||
This is a TypeScript-based background worker service that combines periodic task execution with HTTP health monitoring and metrics collection. It implements a self-scheduling worker pattern with graceful shutdown support.
|
||||
|
||||
## Architecture
|
||||
## Features
|
||||
|
||||
This is a standalone background worker service that:
|
||||
- Runs independently from the Next.js web-app
|
||||
- Communicates via the shared MongoDB database
|
||||
- Integrates with email service provider (e.g., Mailgun, SendGrid)
|
||||
- **Periodic Task Execution**: Self-scheduling worker loop with configurable interval
|
||||
- **Graceful Shutdown**: Ensures in-flight work completes before shutdown (Docker-compatible)
|
||||
- **Health Monitoring**: HTTP health check endpoint to verify worker status
|
||||
- **Metrics Collection**: Prometheus metrics with PRTG adapter
|
||||
- **Error Isolation**: Worker errors don't crash the process
|
||||
|
||||
## Setup
|
||||
## Getting Started
|
||||
|
||||
TBD
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
npm run start # Start with nodemon (auto-reload)
|
||||
```
|
||||
|
||||
### Build & Run
|
||||
|
||||
```bash
|
||||
npm run build # Compile TypeScript
|
||||
npm run run-server # Run compiled version
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
npm run test # Run Jest in watch mode
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
TBD
|
||||
### Required
|
||||
|
||||
- `PULL_INTERVAL` - Milliseconds between work cycles (default: `"10000"`)
|
||||
|
||||
### Optional
|
||||
|
||||
- `PORT` - HTTP server port (default: `"3000"`)
|
||||
- `PROMETHEUS_APP_LABEL` - App label for Prometheus metrics (default: `"email-server-worker"`)
|
||||
- `PROMETHEUS_HISTOGRAM_BUCKETS` - Histogram bucket sizes (default: `"0.1, 0.5, 1, 5, 10"`)
|
||||
- `DEBUG` - Debug namespaces for console logging (e.g., `"server:server"`)
|
||||
- `ENV` - Environment mode: `"dev"`, `"jest"` (affects logging)
|
||||
|
||||
## HTTP Endpoints
|
||||
|
||||
- `GET /healthcheck` - Health check endpoint (verifies worker is running)
|
||||
- `GET /metrics` - Prometheus metrics
|
||||
- `GET /prtg` - PRTG-compatible metrics (XML format)
|
||||
- `GET /ping` - Simple ping/pong endpoint
|
||||
|
||||
## Creating a Worker
|
||||
|
||||
See `src/exampleWorker.ts` for the worker template. The worker must export a `doWork` function:
|
||||
|
||||
```typescript
|
||||
|
||||
export const doWork = async () => {
|
||||
// Your periodic task logic here
|
||||
logger.info("Task Completed", "Work done successfully");
|
||||
|
||||
// Throw errors to mark as failed:
|
||||
// throw new Error("Something went wrong");
|
||||
};
|
||||
```
|
||||
|
||||
Update `src/workRunner.ts` line 6 to import your worker:
|
||||
|
||||
```typescript
|
||||
import { doWork } from "./yourWorker";
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
- **entry.ts** - HTTP server setup, signal handling, worker lifecycle
|
||||
- **workRunner.ts** - Self-scheduling loop, metrics, graceful shutdown
|
||||
- **app.ts** - Express app configuration, routes
|
||||
- **src/lib/** - Shared utilities (logger, metrics, etc.)
|
||||
- **src/routes/** - HTTP route handlers
|
||||
|
||||
## Deployment
|
||||
|
||||
The service uses the `stoppable` library for graceful shutdown with a 10-second timeout before force-closing connections. Docker containers will receive SIGTERM signals and shut down gracefully.
|
||||
|
||||
## Logging
|
||||
|
||||
The service supports two logging mechanisms:
|
||||
|
||||
1. **Console Logging**: Uses the `debug` package, controlled by `DEBUG` env variable
|
||||
|
||||
## Metrics
|
||||
|
||||
Prometheus metrics are automatically collected:
|
||||
|
||||
- `request_operations_total` - Total work cycles executed
|
||||
- `request_operations_ok` - Successful work cycles
|
||||
- `request_operations_failed` - Failed work cycles
|
||||
- `request_duration_seconds` - Duration histogram of work cycles
|
||||
|
||||
## Documentation
|
||||
|
||||
See `CLAUDE.md` in the repository root for complete architecture documentation and guidance.
|
||||
|
||||
30
email-server-worker/build-image.sh
Executable file
30
email-server-worker/build-image.sh
Executable file
@@ -0,0 +1,30 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$1" == "" ] ; then
|
||||
printf "\nDocker image version not set - please specify the version to build"
|
||||
printf "\n\nSyntax:\n\n build-image.sh 1.0.0\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
read -p "BUILD: Push new image to registry [y/n]? " -n 1 -r
|
||||
echo # (optional) move to a new line
|
||||
|
||||
PUSH_IMAGE_TO_REPO="$REPLY"
|
||||
|
||||
printf "\nBUILD START ...\n\n"
|
||||
|
||||
REGISTRY_URL=registry.ngit.hr
|
||||
IMAGE_NAME=evo-open-table-sync-svc
|
||||
IMAGE_VERSION=$1
|
||||
|
||||
IMAGE_TAG=$REGISTRY_URL/ngit/$IMAGE_NAME:$IMAGE_VERSION
|
||||
|
||||
docker build . -t $IMAGE_TAG
|
||||
|
||||
# if [[ "$PUSH_IMAGE_TO_REPO" =~ ^[Yy]$ ]]
|
||||
# then
|
||||
# printf "\nPushing image ...\n\n"
|
||||
# docker push $IMAGE_TAG
|
||||
# fi
|
||||
|
||||
printf "\nBUILD DONE!\n\n"
|
||||
39
email-server-worker/jest.config.ts
Normal file
39
email-server-worker/jest.config.ts
Normal file
@@ -0,0 +1,39 @@
|
||||
/** @type {import('jest/dist/types').InitialOptionsTsJest} */
|
||||
|
||||
import type { Config } from 'jest/build/index';
|
||||
|
||||
const config:Config = {
|
||||
// preset: 'ts-jest',
|
||||
transform: {
|
||||
'^.+\\.tsx?$': [
|
||||
'esbuild-jest', {
|
||||
sourcemap:true, // bez ovog VS code umjesto originala prikazuje transpilirane datoteke
|
||||
target:'es2020' // ovo je nužno kako bi BigInt funkcionirao
|
||||
}]
|
||||
},
|
||||
maxWorkers: 4,
|
||||
testEnvironment: 'node',
|
||||
// The root directory that Jest should scan for tests and modules within
|
||||
rootDir: "./",
|
||||
// A list of paths to directories that Jest should use to search for files in
|
||||
roots: [
|
||||
"<rootDir>/tests",
|
||||
],
|
||||
// The glob patterns Jest uses to detect test files
|
||||
testMatch: [
|
||||
"**/?(*.)+(spec).[tj]s?(x)",
|
||||
],
|
||||
// Automatically clear mock calls and instances between every test
|
||||
clearMocks: true,
|
||||
// An array of regexp pattern strings that are matched against all source file paths, matched files will skip transformation
|
||||
transformIgnorePatterns: ["/node_modules/"],
|
||||
// An array of regexp pattern strings that are matched against all test paths, matched tests are skipped
|
||||
testPathIgnorePatterns: ["/node_modules/", "/build/"],
|
||||
// Indicates whether each individual test should be reported during the run
|
||||
verbose: true,
|
||||
setupFiles: [
|
||||
'dotenv/config', // učitaj varijable iz .env i učini ih dostupne testiranom software-u
|
||||
]
|
||||
};
|
||||
|
||||
module.exports = config;
|
||||
49
email-server-worker/package.json
Normal file
49
email-server-worker/package.json
Normal file
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"name": "email-server-worker",
|
||||
"version": "0.1.0",
|
||||
"description": "Background worker service with HTTP health monitoring and metrics collection",
|
||||
"main": "entry.ts",
|
||||
"scripts": {
|
||||
"start": "nodemon ./src/entry.ts",
|
||||
"run-server": "DEBUG=* node --enable-source-maps ./build/entry.js",
|
||||
"build": "ttsc --project ./",
|
||||
"test": "ENV=jest jest --watch"
|
||||
},
|
||||
"author": "Nikola",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"debug": "^2.6.9",
|
||||
"express": "^4.18.2",
|
||||
"http-errors": "^1.7.2",
|
||||
"node-fetch": "^2.6.7",
|
||||
"prom-client": "^14.0.1",
|
||||
"stoppable": "^1.1.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@babel/preset-typescript": "^7.18.6",
|
||||
"@types/debug": "^4.1.7",
|
||||
"@types/express": "^4.17.13",
|
||||
"@types/http-errors": "^1.8.1",
|
||||
"@types/jest": "^29.2.5",
|
||||
"@types/node": "^16.10.2",
|
||||
"@types/node-fetch": "^2.6.2",
|
||||
"@types/stoppable": "^1.1.1",
|
||||
"@types/supertest": "^2.0.11",
|
||||
"dotenv": "^16.0.3",
|
||||
"esbuild": "^0.16.14",
|
||||
"esbuild-jest": "^0.5.0",
|
||||
"jest": "^29.3.1",
|
||||
"nodemon": "^2.0.13",
|
||||
"supertest": "^6.3.3",
|
||||
"ts-node": "^10.9.1",
|
||||
"tsconfig-paths": "^4.1.2",
|
||||
"ttypescript": "^1.5.15",
|
||||
"typescript": "^4.9.4",
|
||||
"typescript-transform-paths": "^3.4.4"
|
||||
},
|
||||
"babel": {
|
||||
"presets": [
|
||||
"@babel/preset-typescript"
|
||||
]
|
||||
}
|
||||
}
|
||||
13
email-server-worker/run-image.sh
Executable file
13
email-server-worker/run-image.sh
Executable file
@@ -0,0 +1,13 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$1" == "" ] ; then
|
||||
printf "\nNisi zadao verziju Docker image-a koji treba pokrenuti"
|
||||
printf "\n\nSintaksa:\n\n run-image.sh 1.0.0\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IMAGE_TAG=pr-d-registry.ngit.hr/ngit/evo-open-table-sync-svc:$1
|
||||
|
||||
docker run -p 3000:3000 \
|
||||
--env DEBUG=* \
|
||||
$IMAGE_TAG
|
||||
34
email-server-worker/src/app.ts
Normal file
34
email-server-worker/src/app.ts
Normal file
@@ -0,0 +1,34 @@
|
||||
import express from 'express';
|
||||
import createError from 'http-errors';
|
||||
|
||||
import { errorRouter } from './routes/errorRouter';
|
||||
import { finalErrorRouter } from './routes/finalErrorRouter';
|
||||
import { metricsRouter } from './routes/metricsRouter';
|
||||
import { pingRouter } from './routes/pingRouter';
|
||||
import { healthcheckRouter } from './routes/healthcheckRouter';
|
||||
|
||||
import { SupportedRoutes } from './types/enums/SupportedRoutes';
|
||||
|
||||
const app = express();
|
||||
|
||||
// u slučaju kada se server vrti iza proxy-a
|
||||
// ovaj flag će natjerati Express da informacije poput
|
||||
// IP adrese klijenta, protokola uzima iz X-Forward-*
|
||||
// HTTP header polja, koja postavlja proxy
|
||||
app.set('trust proxy', true);
|
||||
|
||||
// prometheus sa ove rute dohvaća zadnje važeću statistiku
|
||||
app.use(SupportedRoutes.metricsPath, metricsRouter);
|
||||
app.use(SupportedRoutes.ping, pingRouter);
|
||||
app.use(SupportedRoutes.healthcheck, healthcheckRouter);
|
||||
|
||||
// default handler
|
||||
app.use((req, res, next) => next(createError(404)));
|
||||
|
||||
// error handler za sve predviđene greške
|
||||
app.use(errorRouter);
|
||||
|
||||
// error router za nepredviđene greške
|
||||
app.use(finalErrorRouter);
|
||||
|
||||
export default app;
|
||||
122
email-server-worker/src/entry.ts
Executable file
122
email-server-worker/src/entry.ts
Executable file
@@ -0,0 +1,122 @@
|
||||
import app from './app';
|
||||
import http from 'http';
|
||||
import stoppable from 'stoppable';
|
||||
|
||||
import { createLogger } from './lib/logger';
|
||||
import { disposeSyncWorker, startSyncWorker } from './workRunner';
|
||||
const logger = createLogger("server:server");
|
||||
|
||||
/**
|
||||
* Normalize a port into a number, string, or false.
|
||||
*/
|
||||
const normalizePort = (val:string):string|number|boolean => {
|
||||
const port = parseInt(val, 10);
|
||||
|
||||
if (isNaN(port)) {
|
||||
// named pipe
|
||||
return val;
|
||||
}
|
||||
|
||||
if (port >= 0) {
|
||||
// port number
|
||||
return port;
|
||||
}
|
||||
|
||||
return false;
|
||||
};
|
||||
|
||||
/**
|
||||
* Event listener for HTTP server "error" event.
|
||||
*/
|
||||
const onError = (error:any):void => {
|
||||
if (error.syscall !== 'listen') {
|
||||
throw error;
|
||||
}
|
||||
|
||||
const bind = typeof port === 'string'
|
||||
? 'Pipe ' + port
|
||||
: 'Port ' + port;
|
||||
|
||||
// handle specific listen errors with friendly messages
|
||||
switch (error.code) {
|
||||
case 'EACCES':
|
||||
console.error(bind + ' requires elevated privileges');
|
||||
process.exit(1);
|
||||
break;
|
||||
case 'EADDRINUSE':
|
||||
console.error(bind + ' is already in use');
|
||||
process.exit(1);
|
||||
break;
|
||||
default:
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Event listener for HTTP server "listening" event.
|
||||
*/
|
||||
const onListening = ():void => {
|
||||
const addr = server.address();
|
||||
const bind = typeof addr === 'string'
|
||||
? 'pipe ' + addr
|
||||
: 'port ' + addr?.port;
|
||||
logger(`⚡️[server]: Server is running at ${bind}`);
|
||||
};
|
||||
|
||||
/**
|
||||
* Get port from environment and store in Express.
|
||||
*/
|
||||
const port:number|string|boolean = normalizePort(process.env.PORT || '3000');
|
||||
|
||||
/**
|
||||
* How long should stoppable wait before it starts force-closing connections
|
||||
* @description wait max 10 seconds - needs to be shorter than `healthcheck.timeout` (=15sec)
|
||||
*/
|
||||
const FORCE_STOP_TIMEOUT = 10000;
|
||||
|
||||
/**
|
||||
* Create HTTP server.
|
||||
*/
|
||||
const server = stoppable( http.createServer(app), FORCE_STOP_TIMEOUT );
|
||||
|
||||
// Listen on provided port, on all network interfaces.
|
||||
server.listen(port);
|
||||
server.on('error', onError);
|
||||
server.on('listening', onListening);
|
||||
|
||||
/**
|
||||
* Starting sync worker process
|
||||
*/
|
||||
startSyncWorker();
|
||||
|
||||
// quit on ctrl-c when running docker in terminal
|
||||
// (signal neće biti registriran ako je server pokrenuti via `npm` ili `nodemon` - mora biti pokrenuti izravno via Node)
|
||||
process.on('SIGINT', () => {
|
||||
logger('Got SIGINT (aka ctrl-c in docker). Graceful shutdown ', new Date().toISOString());
|
||||
shutdown();
|
||||
});
|
||||
|
||||
// quit properly on docker stop
|
||||
// (signal neće biti registriran ako je server pokrenuti via `npm` ili `nodemon` - mora biti pokrenuti izravno via Node)
|
||||
process.on('SIGTERM', () => {
|
||||
logger('Got SIGTERM (docker container stop). Graceful shutdown ', new Date().toISOString());
|
||||
shutdown();
|
||||
});
|
||||
|
||||
// shut down server
|
||||
const shutdown = async () => {
|
||||
|
||||
await disposeSyncWorker();
|
||||
|
||||
// NOTE: server.close is for express based apps
|
||||
// If using hapi, use `server.stop`
|
||||
server.close((err) => {
|
||||
if (err) {
|
||||
console.error(err);
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
logger('Exiting server process...');
|
||||
}
|
||||
process.exit();
|
||||
});
|
||||
};
|
||||
33
email-server-worker/src/exampleWorker.ts
Normal file
33
email-server-worker/src/exampleWorker.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* Example worker implementation
|
||||
*
|
||||
* This is a placeholder worker that demonstrates the worker pattern.
|
||||
* Replace this with your actual worker implementation.
|
||||
*
|
||||
* The worker is called periodically by workRunner.ts based on PULL_INTERVAL.
|
||||
*
|
||||
* @throws Error to increment failedRequestCounter in Prometheus
|
||||
* @returns Promise that resolves when work is complete (increments successfulRequestCounter)
|
||||
*/
|
||||
export const doWork = async () => {
|
||||
const startTime = Date.now();
|
||||
|
||||
// TODO: Implement your periodic worker logic here
|
||||
// Examples:
|
||||
// - Fetch data from external API
|
||||
// - Process queued tasks from database
|
||||
// - Send scheduled emails
|
||||
// - Clean up expired records
|
||||
// - Sync data between systems
|
||||
|
||||
const workDuration = Date.now() - startTime;
|
||||
|
||||
// Log success (only in non-test environments)
|
||||
if (process.env.ENV !== "jest") {
|
||||
const logMessage = `Example worker completed in ${workDuration}ms`;
|
||||
console.log(logMessage);
|
||||
}
|
||||
|
||||
// Note: Throw errors to mark work as failed:
|
||||
// throw new Error("Something went wrong");
|
||||
};
|
||||
29
email-server-worker/src/healthcheck.ts
Normal file
29
email-server-worker/src/healthcheck.ts
Normal file
@@ -0,0 +1,29 @@
|
||||
import { createLogger } from "./lib/logger";
|
||||
|
||||
import http, { IncomingMessage } from "http";
|
||||
const logger = createLogger("server:healthcheck");
|
||||
|
||||
const options = {
|
||||
host: "localhost",
|
||||
port: "3000",
|
||||
timeout: 2000,
|
||||
path: '/healthcheck/'
|
||||
};
|
||||
|
||||
const request = http.request(options, (res:IncomingMessage) => {
|
||||
|
||||
logger(`Healthcheck: STATUS ${res.statusCode}`);
|
||||
|
||||
if (res.statusCode == 200) {
|
||||
process.exit(0);
|
||||
} else {
|
||||
process.exit(1);
|
||||
}
|
||||
});
|
||||
|
||||
request.on("error", function (err:any) {
|
||||
logger("Healthcheck: ERROR");
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
request.end();
|
||||
8
email-server-worker/src/lib/initTools.ts
Normal file
8
email-server-worker/src/lib/initTools.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
|
||||
/**
|
||||
* Za neinicijaliziranu env varijablu vraća default vrijednost
|
||||
* @param value vrijednost env varijable
|
||||
* @param defaultValue default vrijednost
|
||||
* @returns
|
||||
*/
|
||||
export const coalesce = (value:string|undefined, defaultValue:string):string => value===undefined ? defaultValue : (value==="" ? defaultValue : value);
|
||||
21
email-server-worker/src/lib/logger.ts
Normal file
21
email-server-worker/src/lib/logger.ts
Normal file
@@ -0,0 +1,21 @@
|
||||
import debug from 'debug';
|
||||
|
||||
/**
|
||||
* Logs to console / stdout
|
||||
* @param namespace
|
||||
* @returns instance of Debug
|
||||
*/
|
||||
export const createLogger = (namespace:string):debug.Debugger => {
|
||||
const dbg = debug(namespace);
|
||||
|
||||
const rx = /nodemon/gi;
|
||||
|
||||
if(rx.test(process.env?.npm_lifecycle_script ?? "")) {
|
||||
// When started via nodemon:
|
||||
// forcing the use of console insted of stdout
|
||||
// -> nodemon doesn't work with stdout
|
||||
dbg.log = console.log.bind(console);
|
||||
}
|
||||
|
||||
return(dbg);
|
||||
};
|
||||
50
email-server-worker/src/lib/metricsCounters.ts
Normal file
50
email-server-worker/src/lib/metricsCounters.ts
Normal file
@@ -0,0 +1,50 @@
|
||||
import { Counter, Histogram, register } from 'prom-client';
|
||||
import { coalesce } from './initTools';
|
||||
|
||||
/** Histogram Buckets */
|
||||
const PROMETHEUS_HISTOGRAM_BUCKETS = coalesce(process.env.PROMETHEUS_HISTOGRAM_BUCKETS, "0.1, 0.5, 1, 5, 10");
|
||||
|
||||
/** Labela kojom želimo da bude označena metrika prikupljena na ovom web servisu */
|
||||
const PROMETHEUS_APP_LABEL = coalesce(process.env.PROMETHEUS_APP_LABEL, 'email-server-worker');
|
||||
|
||||
// na "app" labele ćemo razdvajanje rezultata u Grafani
|
||||
register.setDefaultLabels({ app: PROMETHEUS_APP_LABEL });
|
||||
|
||||
/**
|
||||
* Broji koliko je ukupno zahtjeva zaprimljeno za obradu
|
||||
*/
|
||||
export const totalRequestCounter = new Counter({
|
||||
name: "request_operations_total",
|
||||
help: "ukupan broj zaprimljenih zahtjeva",
|
||||
/** countere razdvajamo po vrsti zahtjeva */
|
||||
labelNames: ['path'],
|
||||
});
|
||||
|
||||
/**
|
||||
* Broji zahtjeve koji su uspješno obrađeni
|
||||
*/
|
||||
export const successfulRequestCounter = new Counter({
|
||||
name: "request_operations_ok",
|
||||
help: "broj zahtjeva koji su uspješno obrađeni",
|
||||
/** countere razdvajamo po vrsti zahtjeva */
|
||||
labelNames: ['path'],
|
||||
});
|
||||
|
||||
/**
|
||||
* Broji zahtjeve kod čije obrade je došlo do greške
|
||||
*/
|
||||
export const failedRequestCounter = new Counter({
|
||||
name: "request_operations_failed",
|
||||
help: "broj zahtjeva kod čije obrade je došlo do greške",
|
||||
/** countere razdvajamo po vrsti zahtjeva i rezultatu izvođenja */
|
||||
labelNames: ["path", "status"],
|
||||
});
|
||||
|
||||
/** Histogram mjeri koliko traje obrada pristiglog zahtjeva */
|
||||
export const requestDurationHistogram = new Histogram({
|
||||
name: "request_duration_seconds",
|
||||
help: "Trajanje request-a u sekundama",
|
||||
/** countere razdvajamo po vrsti zahtjeva i rezultatu izvođenja */
|
||||
labelNames: ["path", "status"],
|
||||
buckets: PROMETHEUS_HISTOGRAM_BUCKETS?.split(',').map((el) => parseFloat(el))
|
||||
});
|
||||
19
email-server-worker/src/lib/serializeError.ts
Normal file
19
email-server-worker/src/lib/serializeError.ts
Normal file
@@ -0,0 +1,19 @@
|
||||
|
||||
|
||||
/**
|
||||
* This function serializes an error object into a string that can be logged
|
||||
* @param ex error object
|
||||
* @returns string
|
||||
* @description SQL Server may generate more than one error for one request so you can access preceding errors with `err.precedingErrors`, while the `ex` itself is a generic error without any useful information
|
||||
*/
|
||||
export const serializeError = (ex:Error | Error & { precedingErrors?:Error[] }):string => {
|
||||
const { name, message, stack, precedingErrors } = (ex as Error & { precedingErrors?:Error[] });
|
||||
|
||||
// SQL Server may generate more than one error for one request so you can access preceding errors with `ex.precedingErrors`,
|
||||
// while the `ex` itself is a generic error without any useful information
|
||||
if(precedingErrors) {
|
||||
return(serializeError(precedingErrors[0]));
|
||||
}
|
||||
|
||||
return `${name}:${message}`;
|
||||
}
|
||||
81
email-server-worker/src/routes/errorRouter.ts
Normal file
81
email-server-worker/src/routes/errorRouter.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
import { ErrorRequestHandler, Request, Response } from "express";
|
||||
import createHttpError, { HttpError } from "http-errors";
|
||||
import { createLogger } from '../lib/logger';
|
||||
import { NgitLocals } from "../types/NgitLocals";
|
||||
import { failedRequestCounter } from "../lib/metricsCounters";
|
||||
import { SupportedRoutes } from "../types/enums/SupportedRoutes";
|
||||
|
||||
const consoleLog = createLogger("server:server");
|
||||
|
||||
/**
|
||||
* Router koji se zadnji poziva, a koji sastavlja odgovor u slučaju greške
|
||||
* @param err
|
||||
* @param req
|
||||
* @param res
|
||||
* @param next
|
||||
*/
|
||||
export const errorRouter:ErrorRequestHandler = async (err:HttpError, req, res, next) => {
|
||||
|
||||
const requestPath = req.path as SupportedRoutes;
|
||||
|
||||
// kako je ovaj error handler dosta složen, moguće je da negdje baci grešku
|
||||
// > zato je zamotan u try-catch
|
||||
// > na taj način osiguravam da neće srušiti cijeli proces
|
||||
try {
|
||||
|
||||
let { name:errorLogName, message:errorLogText } = err;
|
||||
let responseBody:string = "";
|
||||
|
||||
switch(err.status) {
|
||||
case 400:
|
||||
responseBody = 'bad request';
|
||||
break;
|
||||
case 401:
|
||||
responseBody = 'unauthorized';
|
||||
break;
|
||||
case 403:
|
||||
responseBody = 'forbidden';
|
||||
break;
|
||||
case 404:
|
||||
consoleLog(`page not found ${req.method} ${requestPath}`)
|
||||
responseBody = 'page not found';
|
||||
errorLogText = `page ${requestPath} not found`;
|
||||
break;
|
||||
case 500:
|
||||
responseBody = "internal server error";
|
||||
errorLogText = err.message;
|
||||
break;
|
||||
default:
|
||||
responseBody = err.name;
|
||||
errorLogText = `err.status=${err.status};err.name=${err.name};err.message=${err.message}`;
|
||||
}
|
||||
|
||||
consoleLog(`${errorLogName}:${errorLogText}`);
|
||||
|
||||
// `headersSent` će biti TRUE ako je router kod kojeg se dogodila greška već poslao header-e
|
||||
// > ako ih probam ponovo postaviti, to će baciti grešku ... a to ovdje mogu izbjeći
|
||||
if(!res.headersSent) {
|
||||
res.status(err.status);
|
||||
res.setHeader('Content-Type', "text/html");
|
||||
res.end(responseBody);
|
||||
} else {
|
||||
// AKO nije pozvan `end` - pozovi ga i završi obradu zahtjeva
|
||||
// ... u suprotnom će konekcija ostati otvorena do timeout-a
|
||||
if(!res.writableEnded) {
|
||||
res.end();
|
||||
}
|
||||
}
|
||||
|
||||
} catch(ex:any) {
|
||||
// ovu grešku će obraditi `finalErrorRouter`
|
||||
next(createHttpError(500, ex));
|
||||
}
|
||||
|
||||
// ne mogu dopustiti da prometheus client sruši server
|
||||
try {
|
||||
failedRequestCounter.inc({ path: requestPath, status: err.status });
|
||||
(res.locals as NgitLocals).stopPrometheusTimer({ path: req.path, status: err.status });
|
||||
} catch(ex:any) {
|
||||
console.error(ex);
|
||||
}
|
||||
};
|
||||
34
email-server-worker/src/routes/finalErrorRouter.ts
Normal file
34
email-server-worker/src/routes/finalErrorRouter.ts
Normal file
@@ -0,0 +1,34 @@
|
||||
import { ErrorRequestHandler, Request, Response } from "express";
|
||||
import { HttpError } from "http-errors";
|
||||
import { createLogger } from '../lib/logger';
|
||||
import { NgitLocals } from "../types/NgitLocals";
|
||||
|
||||
const consoleLog = createLogger("server:server");
|
||||
|
||||
/**
|
||||
* Router koji se izvršava u slučaju grube greške koja nije obrađena nigdje prije
|
||||
* @param err error objekt
|
||||
* @param req express request
|
||||
* @param res express response
|
||||
* @param next
|
||||
*/
|
||||
export const finalErrorRouter:ErrorRequestHandler = async (err:HttpError, req, res, next) => {
|
||||
|
||||
const errorLogText:string = JSON.stringify({ message:err.message, name:err.name, stack:err.stack });
|
||||
|
||||
consoleLog(`Server Error ${err.status}\n${errorLogText}`);
|
||||
|
||||
// `headersSent` će biti TRUE ako je router kod kojeg se dogodila greška već poslao header-e
|
||||
// > ako ih probam ponovo postaviti, to će baciti grešku i u ovom slučaju SRUŠITI SERVER - to ne smijemo dopustiti
|
||||
if(!res.headersSent) {
|
||||
res.status(err.status);
|
||||
res.setHeader('Content-Type', "text/html");
|
||||
res.end(`unhandled server error`);
|
||||
} else {
|
||||
// AKO nije pozvan `end` - pozovi ga i završi obradu zahtjeva
|
||||
// ... u suprotnom će konekcija ostati otvorena do timeout-a
|
||||
if(!res.writableEnded) {
|
||||
res.end();
|
||||
}
|
||||
}
|
||||
};
|
||||
35
email-server-worker/src/routes/healthcheckRouter.ts
Normal file
35
email-server-worker/src/routes/healthcheckRouter.ts
Normal file
@@ -0,0 +1,35 @@
|
||||
import { RequestHandler, Router } from "express";
|
||||
import { workerRunnerInfo } from "../workRunner";
|
||||
import { coalesce } from "../lib/initTools";
|
||||
|
||||
const PULL_INTERVAL = parseInt(coalesce(process.env.PULL_INTERVAL, "10000"));
|
||||
|
||||
/** Maximum time between two worker jobs */
|
||||
const MAX_WORKER_LATENCY = PULL_INTERVAL * 2.5;
|
||||
|
||||
/**
|
||||
* Router koji se izvršava u slučaju grube greške koja nije obrađena nigdje prije
|
||||
* @param req express request
|
||||
* @param res express response
|
||||
* @param next
|
||||
*/
|
||||
export const healthcheckRouter:RequestHandler = async (req, res, next) => {
|
||||
const workerLatency = Date.now() - workerRunnerInfo.lastWorkTime;
|
||||
|
||||
if(workerLatency > MAX_WORKER_LATENCY) {
|
||||
const msg = `No work done in ${workerLatency}ms. Last worker status = "${workerRunnerInfo.status}"`;
|
||||
|
||||
console.warn(msg)
|
||||
|
||||
res.status(500);
|
||||
res.setHeader('Content-Type', 'text/plain');
|
||||
res.end(msg);
|
||||
} else {
|
||||
res.status(200);
|
||||
res.setHeader('Content-Type', 'text/plain');
|
||||
res.end('OK');
|
||||
}
|
||||
};
|
||||
|
||||
export const pingRouter = Router();
|
||||
pingRouter.get('/', healthcheckRouter);
|
||||
19
email-server-worker/src/routes/metricsRouter.ts
Normal file
19
email-server-worker/src/routes/metricsRouter.ts
Normal file
@@ -0,0 +1,19 @@
|
||||
import { Router, NextFunction, Request, Response } from "express";
|
||||
import createError from 'http-errors';
|
||||
import { register } from 'prom-client';
|
||||
import { createLogger } from '../lib/logger';
|
||||
|
||||
const logger = createLogger("server:metrics");
|
||||
|
||||
export const metricsRouter = Router();
|
||||
|
||||
metricsRouter.get('/', async (req:Request, res:Response, next:NextFunction) => {
|
||||
// ne mogu dopustiti da prometheus client sruši server
|
||||
try {
|
||||
logger(`⚡️[server]: GET /metrics`);
|
||||
res.set('Content-Type', register.contentType);
|
||||
res.end(await register.metrics());
|
||||
} catch(ex:any) {
|
||||
next(createError(500, (ex as Error).message));
|
||||
}
|
||||
});
|
||||
16
email-server-worker/src/routes/pingRouter.ts
Normal file
16
email-server-worker/src/routes/pingRouter.ts
Normal file
@@ -0,0 +1,16 @@
|
||||
import { RequestHandler, Router } from "express";
|
||||
|
||||
/**
|
||||
* Router koji se izvršava u slučaju grube greške koja nije obrađena nigdje prije
|
||||
* @param req express request
|
||||
* @param res express response
|
||||
* @param next
|
||||
*/
|
||||
export const pingRequestHandler:RequestHandler = async (req, res, next) => {
|
||||
res.status(200);
|
||||
res.setHeader('Content-Type', 'text/plain');
|
||||
res.end('PONG');
|
||||
};
|
||||
|
||||
export const pingRouter = Router();
|
||||
pingRouter.get('/', pingRequestHandler);
|
||||
7
email-server-worker/src/types/NgitLocals.ts
Normal file
7
email-server-worker/src/types/NgitLocals.ts
Normal file
@@ -0,0 +1,7 @@
|
||||
import { LabelValues } from "prom-client";
|
||||
|
||||
/** data assignet to `express.response.locals` */
|
||||
export type NgitLocals = {
|
||||
/** Prometheus client timer */
|
||||
stopPrometheusTimer: (labels?: LabelValues<"path"|"status">) => number,
|
||||
};
|
||||
5
email-server-worker/src/types/enums/SupportedRoutes.ts
Normal file
5
email-server-worker/src/types/enums/SupportedRoutes.ts
Normal file
@@ -0,0 +1,5 @@
|
||||
export enum SupportedRoutes {
|
||||
metricsPath='/metrics',
|
||||
ping='/ping',
|
||||
healthcheck='/healthcheck',
|
||||
}
|
||||
29
email-server-worker/src/types/environment.d.ts
vendored
Normal file
29
email-server-worker/src/types/environment.d.ts
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
|
||||
declare global {
|
||||
namespace NodeJS {
|
||||
interface ProcessEnv {
|
||||
/**
|
||||
* (optional) environment u kojem se proces vrti
|
||||
* @default undefined
|
||||
* */
|
||||
ENV?:"dev"|"jest"
|
||||
/**
|
||||
* (optional) App label to be used in Prometheus (Grafana)
|
||||
* @default "email-server-worker"
|
||||
* */
|
||||
PROMETHEUS_APP_LABEL?: string
|
||||
/**
|
||||
* (optional) Prometheus histogram bucket sizes (grafana)
|
||||
* @default "0.1, 0.5, 1, 5, 10"
|
||||
* */
|
||||
PROMETHEUS_HISTOGRAM_BUCKETS?: string
|
||||
/**
|
||||
* (required) Pull interval in milliseconds - how often should worker cycle run
|
||||
* @default "10000"
|
||||
* */
|
||||
PULL_INTERVAL:string
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export {}
|
||||
165
email-server-worker/src/workRunner.ts
Normal file
165
email-server-worker/src/workRunner.ts
Normal file
@@ -0,0 +1,165 @@
|
||||
import { failedRequestCounter, requestDurationHistogram, successfulRequestCounter, totalRequestCounter } from "./lib/metricsCounters";
|
||||
import { coalesce } from "./lib/initTools";
|
||||
import { createLogger } from "./lib/logger";
|
||||
import { serializeError } from "./lib/serializeError";
|
||||
import { doWork } from "./exampleWorker";
|
||||
|
||||
/** time between two pull operations */
|
||||
const PULL_INTERVAL = parseInt(coalesce(process.env.PULL_INTERVAL, "10000"));
|
||||
const consoleLog = createLogger("server:server");
|
||||
|
||||
/** Writes entry to log */
|
||||
const logWrite = (logTitle:string, logMessage:string) => {
|
||||
consoleLog(`${logTitle}: ${logMessage}}`);
|
||||
}
|
||||
|
||||
/** Writes error to log */
|
||||
const logError = (ex: any) =>
|
||||
logWrite(serializeError(ex), "error");
|
||||
|
||||
/**
|
||||
* zastavica za zaustavljanje sinhronizacije
|
||||
*/
|
||||
let disposed:boolean = false;
|
||||
/** is worker started - prevents multiple starts */
|
||||
let workerStarted:boolean = false;
|
||||
/** Promise which is resolved once the pending work in progress is completed */
|
||||
let pendingWork:Promise<void>|undefined;
|
||||
/** Worker re-run timeout */
|
||||
let pendingTimeout:NodeJS.Timeout|undefined;
|
||||
|
||||
/** Enumeracija pojedinih statusa obrade jednog work-a */
|
||||
export enum WorkerRunnerStatus {
|
||||
init="init",
|
||||
disposed="disposed",
|
||||
beginWork="beginWork",
|
||||
updatedStats1="updatedStats1",
|
||||
updatedStats2="updatedStats2",
|
||||
stoppedStatTimer="stoppedStatTimer",
|
||||
workDone="workDone",
|
||||
newIntervalScheduled="newIntervalScheduled",
|
||||
currentWorkResolved="currentWorkResolved",
|
||||
}
|
||||
|
||||
/** Info o statusu workera */
|
||||
export type WorkerRunnerInfo = {
|
||||
/** zadnje izvršena readnja */
|
||||
status: WorkerRunnerStatus,
|
||||
/** vrijeme kada je worker zadnji puta pokrenut */
|
||||
lastWorkTime: number,
|
||||
}
|
||||
|
||||
/** Info o statusu workera, koji koristi healthcheck kako bi vidio da li stvar funkcionira */
|
||||
export const workerRunnerInfo:WorkerRunnerInfo = {
|
||||
status: WorkerRunnerStatus.init,
|
||||
lastWorkTime: Date.now()
|
||||
}
|
||||
|
||||
export const workRunner = async () => {
|
||||
|
||||
pendingTimeout = undefined;
|
||||
workerRunnerInfo.lastWorkTime = Date.now();
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.beginWork;
|
||||
|
||||
// AKO je modul zaustavljen
|
||||
// -> nemoj se pokrenuti
|
||||
if(disposed) {
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.disposed;
|
||||
return;
|
||||
}
|
||||
|
||||
// kreiram Promise koji omogućuje da dispose zna
|
||||
// pričekati da worker završi sa poslom (ako je u tom trenutku aktivan)
|
||||
pendingWork = new Promise(async (resolve) => {
|
||||
|
||||
try {
|
||||
totalRequestCounter.inc();
|
||||
|
||||
const stopPrometheusTimer = requestDurationHistogram.startTimer();
|
||||
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.updatedStats1;
|
||||
|
||||
try {
|
||||
// ne dopuštam da stvar sruši worker
|
||||
await doWork();
|
||||
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.workDone;
|
||||
|
||||
// ažuriram statistiku
|
||||
successfulRequestCounter.inc();
|
||||
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.updatedStats2;
|
||||
} catch(ex:any) {
|
||||
|
||||
// ažuriram statistiku
|
||||
failedRequestCounter.inc();
|
||||
logError(ex);
|
||||
}
|
||||
|
||||
stopPrometheusTimer();
|
||||
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.stoppedStatTimer;
|
||||
} catch(ex:any) {
|
||||
logError(ex);
|
||||
}
|
||||
|
||||
// nemoj pokrenuti timer ako je worker u međuvremenu disposed
|
||||
if(!disposed) {
|
||||
// pull again after timeout
|
||||
pendingTimeout = setTimeout(workRunner, PULL_INTERVAL);
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.newIntervalScheduled;
|
||||
} else {
|
||||
logWrite("Info", "... exiting worker loop");
|
||||
}
|
||||
|
||||
resolve();
|
||||
|
||||
workerRunnerInfo.status = WorkerRunnerStatus.currentWorkResolved;
|
||||
|
||||
pendingWork = undefined;
|
||||
});
|
||||
|
||||
// this is an async function which must return a promise
|
||||
// > so return the promise which will be resolved once the work is done
|
||||
return(pendingWork);
|
||||
};
|
||||
|
||||
/**
|
||||
* Starts the worker
|
||||
*/
|
||||
export const startSyncWorker = () => {
|
||||
if(!workerStarted && !disposed) {
|
||||
workerStarted = true;
|
||||
workRunner();
|
||||
logWrite("Info", "Worker Started");
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops and disposes the worker
|
||||
*/
|
||||
export const disposeSyncWorker = async () => {
|
||||
logWrite("Info", "Disposing worker ...");
|
||||
|
||||
disposed = true;
|
||||
|
||||
// preventing timer from trigger another work cycle
|
||||
if(pendingTimeout) {
|
||||
clearTimeout(pendingTimeout);
|
||||
}
|
||||
|
||||
// IF no work is currently in progress
|
||||
// > return a resolved promise
|
||||
if(!pendingWork) {
|
||||
return(Promise.resolve());
|
||||
}
|
||||
|
||||
await pendingWork;
|
||||
|
||||
logWrite("Info", "Worker disposed!");
|
||||
}
|
||||
|
||||
/** Ovo se koristi samo za Unit Testing */
|
||||
export const reset_dispose = () => {
|
||||
disposed = false;
|
||||
}
|
||||
29
email-server-worker/tests/__mocks__/prom-client.ts
Normal file
29
email-server-worker/tests/__mocks__/prom-client.ts
Normal file
@@ -0,0 +1,29 @@
|
||||
import { LabelValues } from "prom-client";
|
||||
|
||||
export class Counter {
|
||||
public inc() {
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
export class Histogram<T extends string> {
|
||||
startTimer(labels?: LabelValues<T>): (labels?: LabelValues<T>) => void {
|
||||
return((labels?: LabelValues<T>) => { });
|
||||
}
|
||||
}
|
||||
|
||||
class Register {
|
||||
public setDefaultLabels(labels: Object) {
|
||||
|
||||
}
|
||||
|
||||
public metrics(): Promise<string> {
|
||||
return(Promise.resolve(""));
|
||||
}
|
||||
|
||||
public get contentType() {
|
||||
return("");
|
||||
}
|
||||
}
|
||||
|
||||
export const register = new Register();
|
||||
33
email-server-worker/tests/helpers/mockHttpContext.ts
Normal file
33
email-server-worker/tests/helpers/mockHttpContext.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { Request, Response, NextFunction } from 'express';
|
||||
import { NgitLocals } from "../../src/types/NgitLocals";
|
||||
|
||||
interface IMockHttpContext {
|
||||
reqPath?:string
|
||||
headersSent?:boolean
|
||||
writableEnded?:boolean
|
||||
method?:string
|
||||
}
|
||||
|
||||
export const mockHttpContext = ({reqPath="/", headersSent=false, writableEnded=false, method="GET"}:IMockHttpContext|undefined = {}) => {
|
||||
const req = {
|
||||
path:reqPath,
|
||||
method,
|
||||
url:`https://localhost${reqPath}`,
|
||||
params: {},
|
||||
} as unknown as Request;
|
||||
|
||||
const res = {
|
||||
end: jest.fn(),
|
||||
status: jest.fn(),
|
||||
setHeader: jest.fn(),
|
||||
locals: {
|
||||
stopPrometheusTimer: jest.fn(),
|
||||
} as unknown as NgitLocals,
|
||||
headersSent,
|
||||
writableEnded,
|
||||
} as unknown as Response;
|
||||
|
||||
const next:NextFunction = jest.fn();
|
||||
|
||||
return({req,res,next})
|
||||
}
|
||||
118
email-server-worker/tests/routers/errorRouter.spec.ts
Normal file
118
email-server-worker/tests/routers/errorRouter.spec.ts
Normal file
@@ -0,0 +1,118 @@
|
||||
|
||||
import { errorRouter } from '../../src/routes/errorRouter';
|
||||
import createError from "http-errors";
|
||||
import { mockHttpContext } from "../helpers/mockHttpContext";
|
||||
|
||||
describe("errorRouter", () => {
|
||||
|
||||
beforeEach(() => {
|
||||
mockWrite.mockClear();
|
||||
});
|
||||
|
||||
test("u slučaju greške 404 mora vratiti string poruku 'page not found'", async () => {
|
||||
const err = createError(404)
|
||||
const {req,res,next} = mockHttpContext();
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(404);
|
||||
expect(res.setHeader).toHaveBeenCalledWith('Content-Type', "text/html");
|
||||
expect(res.end).toHaveBeenCalledWith("page not found");
|
||||
});
|
||||
|
||||
test("u slučaju greške 404 mora logirati request, response i tekst greške", async () => {
|
||||
const err = createError(404)
|
||||
const reqPath = "/neki-path/";
|
||||
const {req,res,next} = mockHttpContext({ reqPath });
|
||||
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.locals.logger.info).toHaveBeenCalledWith("response", "page not found");
|
||||
expect(res.locals.logger.error).toHaveBeenCalledWith(err.name, "page "+req.path+" not found");
|
||||
});
|
||||
|
||||
test("ako su header-i već poslani, ne smiju biti poslani još jednom", async () => {
|
||||
const err = createError(404)
|
||||
const {req,res,next} = mockHttpContext({ headersSent:true, writableEnded:true });
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).not.toHaveBeenCalled();
|
||||
expect(res.setHeader).not.toHaveBeenCalled();
|
||||
expect(res.end).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("ako NIJE već pozvana [end] metoda, treba je pozvati", async () => {
|
||||
const err = createError(404)
|
||||
const {req,res,next} = mockHttpContext({ headersSent:true, writableEnded:false });
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).not.toHaveBeenCalled();
|
||||
expect(res.setHeader).not.toHaveBeenCalled();
|
||||
expect(res.end).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("mora zaustaviti Prometheus Timer", async () => {
|
||||
const err = createError(404)
|
||||
const {req,res,next} = mockHttpContext({ headersSent:true, writableEnded:false });
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.locals.stopPrometheusTimer).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("u slučaju greške 500 mora vratiti string poruku 'internal server error'", async () => {
|
||||
const err = createError(500)
|
||||
const {req,res,next} = mockHttpContext();
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(500);
|
||||
expect(res.setHeader).toHaveBeenCalledWith('Content-Type', "text/html");
|
||||
expect(res.end).toHaveBeenCalledWith("internal server error");
|
||||
});
|
||||
|
||||
test("u slučaju greške 400 mora vratiti string poruku 'bad request' i logirati grešku", async () => {
|
||||
const errorMessage = "mock error text 1";
|
||||
const err = createError(400, errorMessage);
|
||||
const {req,res,next} = mockHttpContext();
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(400);
|
||||
expect(res.setHeader).toHaveBeenCalledWith('Content-Type', "text/html");
|
||||
expect(res.end).toHaveBeenCalledWith("bad request");
|
||||
|
||||
expect(res.locals.logger.errorwrite).toHaveBeenCalledWith(err.name, errorMessage);
|
||||
});
|
||||
|
||||
test("u slučaju greške 401 mora vratiti string poruku 'unauthorized' i logirati grešku", async () => {
|
||||
const errorMessage = "mock error text 2";
|
||||
const err = createError(401, errorMessage)
|
||||
const {req,res,next} = mockHttpContext();
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(401);
|
||||
expect(res.setHeader).toHaveBeenCalledWith('Content-Type', "text/html");
|
||||
expect(res.end).toHaveBeenCalledWith("unauthorized");
|
||||
|
||||
expect(res.locals.logger.error).toHaveBeenCalledWith(err.name, errorMessage);
|
||||
});
|
||||
|
||||
test("u slučaju greške 403 mora vratiti string poruku 'forbidden' i logirati grešku", async () => {
|
||||
const errorMessage = "mock error text 3";
|
||||
const err = createError(403, errorMessage);
|
||||
const {req,res,next} = mockHttpContext();
|
||||
|
||||
await errorRouter(err, req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(403);
|
||||
expect(res.setHeader).toHaveBeenCalledWith('Content-Type', "text/html");
|
||||
expect(res.end).toHaveBeenCalledWith("forbidden");
|
||||
|
||||
expect(res.locals.logger.error).toHaveBeenCalledWith(err.name, errorMessage);
|
||||
});
|
||||
});
|
||||
36
email-server-worker/tsconfig.json
Normal file
36
email-server-worker/tsconfig.json
Normal file
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2020", // https://github.com/microsoft/TypeScript/wiki/Node-Target-Mapping
|
||||
"module": "commonjs",
|
||||
"esModuleInterop": true, // solves the problem regarding the default importa vs importa *
|
||||
"strict": true,
|
||||
"sourceMap": true, // please do create source maps
|
||||
"skipLibCheck": true, // don't verify typescript of 3rd party modules
|
||||
"rootDir": "src", // root directory under which source files are located - it's subtree will be mirrored in "outDir"
|
||||
"outDir": "build", // where the build files should be stored
|
||||
// "baseUrl" ---- se NE SMIJE koristiti
|
||||
// POJAŠNJENJE: ako zadamo "baseUrl" Intellisense će početi kod autocompletion-a (Ctrl+Space)
|
||||
// umjesto relativnih insertirati apsolutni path do modula,
|
||||
// a takav path nije dobar za build niti debugging
|
||||
// "baseUrl": "./", // set a base directory to resolve non-absolute module names - This must be specified if "paths" is used
|
||||
"paths": {
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
// Slijedeće je namijenjeno BUILD projekta
|
||||
// POJAŠNJENJE: build tadi `ttypescript`
|
||||
// koji ne zna interpretirati što je podešeno pod "path"
|
||||
// > to za njega rješava "typescript-transform-paths"
|
||||
"transform": "typescript-transform-paths"
|
||||
}
|
||||
]
|
||||
},
|
||||
"include": ["src/**/*"], // location of files which need to be compiled
|
||||
// Slijedeće je namijenjeno DEBUGGING servera u VS Code-u
|
||||
// POJAŠNJENJE: kod debugginga modul se pokreće pomoću `ts-node`,
|
||||
// koji ne zna sam interpretirati što je podešeno pod "paths"
|
||||
// > to za njega rješava "tsconfig-paths/register"
|
||||
"ts-node": {
|
||||
"require": ["tsconfig-paths/register"]
|
||||
},
|
||||
}
|
||||
Reference in New Issue
Block a user