Files
evidencija-rezija/README.md
Nikola Derežić 4454ea3b7e Enhance backup system with comprehensive scripts and documentation
Database Backup Scripts:
- Add db-dump--standalone.sh for online database dumps (no downtime)
- Add db-restore-from-dump--standalone.sh for restoring from dumps
- Rename backup scripts with double-dash convention for clarity
- Add comprehensive header comments to all backup/restore scripts
- Document online vs offline, standalone vs swarm deployment types
- Add KEEP variable default (7) to dump script for rotation

Docker Configuration:
- Add mongo-backup volume mount to docker-compose-standalone.yaml
- Add mongo-backup volume mount to docker-compose-debug.yml
- Add container names to debug compose for consistency with standalone

Documentation:
- Add comprehensive Database Backup & Restore section to README.md
- Document all backup scripts with usage examples
- Document automated cron schedule (04:00 daily)
- Sunday: Full volume backup (KEEP=2, offline)
- Monday-Saturday: Database dumps (KEEP=7, online)
- Document backup directories and log file locations

Git Configuration:
- Add mongo-backup/ to .gitignore
- Fix missing newline at end of .gitignore

File Cleanup:
- Remove db-scheduled-backup.sh (superseded by cron jobs)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 11:11:17 +01:00

203 lines
8.4 KiB
Markdown

# ToDo
* Public page with instructions
# Authentication
Authentication consists of the following parts:
* `next-auth` boilerplate
* `middleware.ts` = hooks-up `next-auth` into the page processing pipeline - user session is checked before any page is rendered
* `auth.ts` = defines how the authentication is done, and how session is checked (used by middleware)
* `/app/api/[...nextauth]/route.ts` = defines route which shows an authentication form
Source:
* [How to Implement Google Authentication in a Next.js App Using NextAuth](https://www.telerik.com/blogs/how-to-implement-google-authentication-nextjs-app-using-nextauth)
* [Next Js 14 Authentication on Edge Runtime](https://www.youtube.com/watch?v=rEopVx0FKGI)
# Multi-User Support
Each location record is marked with a user ID.
All the actions user `withUser` to fetch user ID, which is then used in all the DB operations.
# Database Backup & Restore
The project includes multiple backup strategies for different deployment scenarios and requirements.
## Backup Scripts Overview
### Standalone Docker Deployments
**Online Backups (No Downtime):**
- `db-dump--standalone.sh` - Creates online backup of the 'utility-bills' database using mongodump
- Database stays running during backup
- Only backs up the database content, not the full volume
- Output: `./mongo-backup/utility-bills-dump-YYYY-MM-DD_HH-MM.tar.gz`
- Default retention: 7 backups (configurable via `KEEP` env var)
- Usage: `./db-dump--standalone.sh` or `KEEP=10 ./db-dump--standalone.sh`
- `db-restore-from-dump--standalone.sh` - Restores from mongodump archives
- Database stays running during restore
- **WARNING**: Drops existing collections before restore
- Usage: `./db-restore-from-dump--standalone.sh utility-bills-dump-2025-11-26_14-30.tar.gz`
**Offline Backups (With Downtime):**
- `db-backup--standalone.sh` - Creates offline backup of the complete mongo-volume directory
- Database container is stopped during backup for consistency
- Backs up the entire MongoDB data directory
- Output: `./backups/mongo-volume-backup-YYYY-MM-DD-HH-MM.tar.gz`
- Default retention: 7 backups (configurable via `KEEP` env var)
- Usage: `./db-backup--standalone.sh` or `KEEP=2 ./db-backup--standalone.sh`
### Docker Swarm Deployments
- `db-backup--swarm.sh` - Creates offline backup by scaling down the MongoDB service
- Service is scaled to 0 during backup
- Output: `./backups/mongo-volume-backup-YYYY-MM-DD-HH-MM.tar.gz`
- Usage: `./db-backup--swarm.sh`
- `db-restore-from-backup--swarm.sh` - Restores volume backup by scaling down the service
- Service is scaled to 0 during restore
- Optional `--pre-backup` flag for safety backup before restore
- Usage: `./db-restore-from-backup--swarm.sh mongo-volume-backup-2025-11-26-14-30.tar.gz`
## Automated Backup Schedule
Backups run automatically via cron at 04:00 every day:
```cron
# Sunday: Full volume backup (offline), keep 2 backups
0 4 * * 0 cd /home/knee-cola/web-pro/evidencija-rezija && KEEP=2 ./db-backup--standalone.sh
# Monday-Saturday: Database dump (online), keep 7 backups
0 4 * * 1-6 cd /home/knee-cola/web-pro/evidencija-rezija && KEEP=7 ./db-dump--standalone.sh
```
**Backup Strategy:**
- **Sundays**: Full volume backup with brief downtime (keeps 2 weeks of full backups)
- **Monday-Saturday**: Online database dumps without downtime (keeps 7 days of incremental backups)
All backup operations are logged with timestamps:
- Volume backups: `./backups/db-backup-standalone.log`
- Database dumps: `./mongo-backup/db-dump-db-standalone.log`
- Restore operations: `./mongo-backup/db-restore-from-dump.log`
## Backup Directories
- `./backups/` - Volume backup archives (full mongo-volume directory)
- `./mongo-backup/` - Database dump archives (utility-bills database only)
Both directories are excluded from git via `.gitignore`.
# Deploying
The deployment is done via Docker:
* build docker image
* deploy Docker service
## Building Docker image
Run the following command:
```bash
build.sh
```
The image will be stored in the local Docker instance.
## Deploying Docker service
Run the following command:
```bash
deploy.sh
```
# Implementation details
## Issues with HOSTNAME
When deplyed via docker and published via Cloudflare there's an issue with `HOSTNAME` env variable:
* if left unset, the server will use IP address assigned to container by Docker (i.e. 10.0.20.3) and **will not accept connections from outside**
```
▲ Next.js 14.0.2
- Local: http://68db6c9ebafe:80
- Network: http://10.0.20.3:80
```
* if set to "0.0.0.0" the server will serve static pages, but will *reject API calls when submitting form*
```
▲ Next.js 14.0.2
- Local: http://localhost:80
- Network: http://0.0.0.0:80
utility-bills-tracker_web-app.1.Error: Invalid Server Actions request.
`x-forwarded-host` header with value `0.0.0.0:80` does not match `origin` header with value `rezije.app` from a forwarded Server Actions request. Aborting the action.
```
* if set to "rezije.app" the server will not start since the IP address it resolves with the given FQDN does not match any of the IP addresses assigned to the container
```
▲ Next.js 14.0.2
- Local: http://localhost:80
- Network: http://0.0.0.0:80
utility-bills-tracker_web-app.1.Error: Invalid Server Actions request.
`x-forwarded-host` header with value `rezije.app:80` does not match `origin` header with value `rezije.app` from a forwarded Server Actions request. Aborting the action.
```
So there are the following issues:
* server will not accept external request - can be fixed by setting `HOSTNAME` to `0.0.0.0`
* server rejects API requests - can be fixed by adding `serverActions.allowedOrigins` option to `nextjs.config.js` file
So these are the fixes which were implemented in order to be able to run server in production.
This is a hack indicating that I don't understand how the damn thing should be configured!
Even when this hack is emplyed the server still logs the followig error:
```
failed to get redirect response TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11730:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
cause: Error: connect ECONNREFUSED 0.0.0.0:443
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:128:17) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '0.0.0.0',
port: 443
}
}
```
## Mongo DB & AVX Instructions
The MongoDB server v > 5.0 will not run on and old machine such as Acer Revo due to it's CPU.
This issue was solved by using an older Mongo DB Version 4.4.27
## Decoding Barcode
Barcode decoding is slow and can lead to locking of the main thread.
This heavy lifting could be moved to background thread.
The new solution could be based on the following code: https://github.com/pocesar/react-use-qrcode
That solution uses video to decode the QR code. This could be modified so that it uses data stored in canvas.
# OAuth verification video transcript
This is Rezije app demonstration for Google OAuth login process for the users of our app.
The video shows that our usage of OAuth scopes complies with Google API services user data policy.
Our app does not use any of the sensitive and restricted scopes.
User's can access Privacy policy and Terms of service by clicking links in footer of the page.
Let's first click on the "Privacy policy" link. Here we can see our Privacy policy.
Now let's click on the "Terms of service" link. Here We can see our Terms of service.
Links to these pages are assigned in the apropriate fields in the Google's Cloud Console and are listed in the Google's OAuth window.
From our homepage you can login by clicking on the "Sign in with Google" button.
Here in the address bar you can see that the client ID is present in the URL.
Select the account with which you would like to sign-in with and you will be logged in into the app.
We are using two scopes:
1. OAuth ID, which is used to assign ownership to the application specific data when it's is stored and retrieved from our database.
2. user's e-mail address, which is stored in our database so that we can contact our users in case such action is needed.
For any questions regarding the use of the Google API service please feel free to reach out at support@rezije.app
# Localization
Localization was done by following video: https://www.youtube.com/watch?v=uZQ5d2bRMO4