diff --git a/.changeset/sparkly-times-win.md b/.changeset/sparkly-times-win.md new file mode 100644 index 00000000..a845151c --- /dev/null +++ b/.changeset/sparkly-times-win.md @@ -0,0 +1,2 @@ +--- +--- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b923433f..beef7f37 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,6 +6,294 @@ before making a change. Please keep the conversations civil, respectful and focus on the topic being discussed. +## Issue Assignment & Fairness Policy + +To keep the project moving fairly for everyone, please follow these guidelines: + +- **Search before submitting.** Before opening a new issue or PR, search existing issues to avoid + duplicates or overlapping work. +- **All PRs must have an associated issue.** Open or find a relevant issue before starting work. + Starting a PR without a linked issue means your work may not be merged, and it does not grant + automatic assignment of an issue. +- **Avoid snowball PRs.** Keep pull requests focused. Do not mix unrelated fixes, features, or + changes in a single PR. +- **Prefer older issues first.** When choosing what to work on, prefer resolving older open issues + before newer ones, unless a blocker exists or the maintainers have explicitly agreed otherwise. +- **Abandoned assignments.** An issue assigned to a contributor and not worked on for **7 days** + (no comments or commits) is considered abandoned, unless the assignee is actively working on it + or has requested an extension from a maintainer. +- **Extensions and transfers.** Assignments can be explicitly extended or transferred to another + contributor if the original assignee is unresponsive. +- **Release your assignment.** If you are no longer interested in an issue, please comment to + request being unassigned so others can pick it up. + +## Development Environment Setup + +Install Docker Desktop following the [official guide](https://docs.docker.com/desktop/) (if you +plan to use Docker). You may have to uninstall Docker on your machine if you installed it using a +different guide. + +Clone the repository and enter the directory: + +``` +git clone git@github.com:Cameri/nostream.git +cd nostream +``` + +Install dependencies (this also sets up Husky pre-commit hooks automatically): + +``` +npm install +``` + +> **Important:** Pre-commit hooks installed by Husky run linting and formatting checks on every +> commit. Do **not** bypass them with `git commit --no-verify`. If a hook fails, fix the reported +> issues before committing. + +### Development Quick Start (Docker Compose) + +Start the relay (runs in the foreground until stopped with Ctrl+C): + +``` +./scripts/start +``` + +### Development Quick Start (Standalone) + +Set the required environment variables (or copy `.env.example` to `.env` and edit it): + +``` +DB_URI="postgresql://postgres:postgres@localhost:5432/nostr_ts_relay_test" +DB_USER=postgres +``` + +or: + +``` +DB_HOST=localhost +DB_PORT=5432 +DB_NAME=nostr_ts_relay +DB_USER=postgres +DB_PASSWORD=postgres +``` + +``` +REDIS_URI="redis://default:nostr_ts_relay@localhost:6379" + +REDIS_HOST=localhost +REDIS_PORT=6379 +REDIS_USER=default +REDIS_PASSWORD=nostr_ts_relay +``` + +Generate a long random secret and set `SECRET`: + +``` +SECRET=aaabbbccc...dddeeefff +# Secret shortened for brevity +``` + +Run migrations (at least once and after pulling new changes): + +``` +NODE_OPTIONS="-r dotenv/config" npm run db:migrate +``` + +Create the `.nostr` folder and copy the default settings file: + +``` +mkdir .nostr +cp resources/default-settings.yaml .nostr/settings.yaml +``` + +Start in development mode: + +``` +npm run dev +``` + +Or start in production mode: + +``` +npm run start +``` + +To clean up build, coverage, and test reports: + +``` +npm run clean +``` + +## Tests + +### Linting and formatting (Biome) + +Run code quality checks with Biome: + +``` +npm run lint +npm run lint:fix +npm run format +npm run format:check +``` + +### Unit tests + +Change to the project's directory: + +``` +cd /path/to/nostream +``` + +Run unit tests: + +``` +npm run test:unit +``` + +Run unit tests in watch mode: + +``` +npm run test:unit:watch +``` + +Get unit test coverage: + +``` +npm run cover:unit +``` + +Open the unit test report: + +``` +open .test-reports/unit/index.html +``` + +Open the unit test coverage report: + +``` +open .coverage/unit/lcov-report/index.html +``` + +### Integration tests (Docker Compose) + +Change to the project's directory: + +``` +cd /path/to/nostream +``` + +Run integration tests: + +``` +npm run docker:test:integration +``` + +Get integration test coverage: + +``` +npm run docker:cover:integration +``` + +### Integration tests (Standalone) + +Change to the project's directory: + +``` +cd /path/to/nostream +``` + +Set the following environment variables: + +``` +DB_URI="postgresql://postgres:postgres@localhost:5432/nostr_ts_relay_test" + +or + +DB_HOST=localhost +DB_PORT=5432 +DB_NAME=nostr_ts_relay_test +DB_USER=postgres +DB_PASSWORD=postgres +DB_MIN_POOL_SIZE=1 +DB_MAX_POOL_SIZE=2 +``` + +Run the integration tests: + +``` +npm run test:integration +``` + +Open the integration test report: + +``` +open .test-reports/integration/report.html +``` + +Get integration test coverage: + +``` +npm run cover:integration +``` + +Open the integration test coverage report: + +``` +open .coverage/integration/lcov-report/index.html +``` + +## Security & Load Testing + +Nostream includes a specialized security tester to simulate Slowloris-style connection holding and +event flood (spam) attacks. This is used to verify relay resilience and prevent memory leaks. + +### Running the Tester + +```bash +# Simulates 5,000 idle "zombie" connections + 100 events/sec spam +npm run test:load -- --zombies 5000 --spam-rate 100 +``` + +### Analyzing Memory (Heap Snapshots) + +To verify that connections are being correctly evicted and memory reclaimed: + +1. Ensure the relay is running with `--inspect` enabled (see `docker-compose.yml`). +2. Open **Chrome DevTools** (`chrome://inspect`) and connect to the relay process. +3. In the **Memory** tab, take a **Heap Snapshot** (Baseline). +4. Run the load tester. +5. Wait for the eviction cycle (default: 120s) and take a second **Heap Snapshot**. +6. Switch the view to **Comparison** and select the Baseline snapshot. +7. Verify that object counts (e.g., `WebSocketAdapter`, `SocketAddress`) return to baseline levels. + +### Server-Side Monitoring + +To observe client and subscription counts in real-time during a test, you can instrument +`src/adapters/web-socket-server-adapter.ts`: + +1. Locate the `onHeartbeat()` method. +2. Add the following logging logic: + ```typescript + private onHeartbeat() { + let totalSubs = 0; + let totalClients = 0; + this.webSocketServer.clients.forEach((webSocket) => { + totalClients++; + const webSocketAdapter = this.webSocketsAdapters.get(webSocket) as IWebSocketAdapter; + if (webSocketAdapter) { + webSocketAdapter.emit(WebSocketAdapterEvent.Heartbeat); + totalSubs += webSocketAdapter.getSubscriptions().size; + } + }); + console.log(`[HEARTBEAT] Clients: ${totalClients} | Total subscriptions: ${totalSubs} | Heap Used: ${(process.memoryUsage().heapUsed / 1024 / 1024).toFixed(1)} MB`); + } + ``` +3. View the live output via Docker logs: + ```bash + docker compose logs -f nostream + ``` + ## Local Quality Checks Run dead code and dependency analysis before opening a pull request: @@ -44,15 +332,23 @@ This interactive prompt will ask you to: The command creates a file in `.changeset/` — commit it with your PR. -### Docs-only PRs (no release) +### Empty changesets (no source code changes) -If your PR only updates documentation and should not affect versioning, add an empty changeset: +If your PR **only** updates documentation, CI/CD configuration, or test coverage — and leaves all +production source code untouched — an empty changeset is acceptable: ```bash npx changeset --empty ``` -Commit the generated `.changeset/*.md` file with your PR. This satisfies CI without producing a version bump or changelog entry. +Commit the generated `.changeset/*.md` file with your PR. This satisfies CI without producing a +version bump or changelog entry. + +This applies to PRs that exclusively contain: + +- Documentation updates (README, CONTRIBUTING, CONFIGURATION, etc.) +- CI/CD workflow changes (`.github/` files) +- Test additions or improvements (when no source code is changed) ### Release process diff --git a/README.md b/README.md index 6344b31a..41a12e19 100644 --- a/README.md +++ b/README.md @@ -472,182 +472,13 @@ To clean up the build, coverage and test reports run: ``` npm run clean ``` -## Development Quick Start (Docker Compose) -Install Docker Desktop following the [official guide](https://docs.docker.com/desktop/). -You may have to uninstall Docker on your machine if you installed it using a different guide. +## Development & Contributing -Clone repository and enter directory: - ``` - git clone git@github.com:Cameri/nostream.git - cd nostream - ``` - -Start: - ``` - ./scripts/start - ``` - - This will run in the foreground of the terminal until you stop it with Ctrl+C. - -## Tests - -### Linting and formatting (Biome) - -Run code quality checks with Biome: - - ``` - npm run lint - npm run lint:fix - npm run format - npm run format:check - ``` - -### Unit tests - -Open a terminal and change to the project's directory: - ``` - cd /path/to/nostream - ``` - -Run unit tests with: - - ``` - npm run test:unit - ``` - -Or, run unit tests in watch mode: - - ``` - npm run test:unit:watch - ``` +For development environment setup, testing, linting, load testing, and contribution guidelines +(including the issue fairness policy, husky pre-commit hooks, and changeset workflow), see +[CONTRIBUTING.md](CONTRIBUTING.md). -To get unit test coverage run: - - ``` - npm run cover:unit - ``` - -To see the unit tests report open `.test-reports/unit/index.html` with a browser: - ``` - open .test-reports/unit/index.html - ``` - -To see the unit tests coverage report open `.coverage/unit/lcov-report/index.html` with a browser: - ``` - open .coverage/unit/lcov-report/index.html - ``` - -### Integration tests (Docker Compose) - -Open a terminal and change to the project's directory: - ``` - cd /path/to/nostream - ``` - -Run integration tests with: - - ``` - npm run docker:test:integration - ``` - -And to get integration test coverage run: - - ``` - npm run docker:cover:integration - ``` - -### Integration tests (Standalone) - -Open a terminal and change to the project's directory: - ``` - cd /path/to/nostream - ``` - -Set the following environment variables: - - ``` - DB_URI="postgresql://postgres:postgres@localhost:5432/nostr_ts_relay_test" - - or - - DB_HOST=localhost - DB_PORT=5432 - DB_NAME=nostr_ts_relay_test - DB_USER=postgres - DB_PASSWORD=postgres - DB_MIN_POOL_SIZE=1 - DB_MAX_POOL_SIZE=2 - ``` - -Then run the integration tests: - - ``` - npm run test:integration - ``` - -To see the integration tests report open `.test-reports/integration/report.html` with a browser: - ``` - open .test-reports/integration/report.html - ``` - -To get the integration test coverage run: - - ``` - npm run cover:integration - ``` - -To see the integration test coverage report open `.coverage/integration/lcov-report/index.html` with a browser. - - ``` - open .coverage/integration/lcov-report/index.html - ``` - - -## Security & Load Testing - -Nostream includes a specialized security tester to simulate Slowloris-style connection holding and event flood (spam) attacks. This is used to verify relay resilience and prevent memory leaks. - -### Running the Tester - ```bash - # Simulates 5,000 idle "zombie" connections + 100 events/sec spam - npm run test:load -- --zombies 5000 --spam-rate 100 - ``` - -### Analyzing Memory (Heap Snapshots) -To verify that connections are being correctly evicted and memory reclaimed: -1. Ensure the relay is running with `--inspect` enabled (see `docker-compose.yml`). -2. Open **Chrome DevTools** (`chrome://inspect`) and connect to the relay process. -3. In the **Memory** tab, take a **Heap Snapshot** (Baseline). -4. Run the load tester. -5. Wait for the eviction cycle (default: 120s) and take a second **Heap Snapshot**. -6. Switch the view to **Comparison** and select the Baseline snapshot. -7. Verify that object counts (e.g., `WebSocketAdapter`, `SocketAddress`) return to baseline levels. - -### Server-Side Monitoring -To observe client and subscription counts in real-time during a test, you can instrument `src/adapters/web-socket-server-adapter.ts`: - -1. Locate the `onHeartbeat()` method. -2. Add the following logging logic: - ```typescript - private onHeartbeat() { - let totalSubs = 0; - let totalClients = 0; - this.webSocketServer.clients.forEach((webSocket) => { - totalClients++; - const webSocketAdapter = this.webSocketsAdapters.get(webSocket) as IWebSocketAdapter; - if (webSocketAdapter) { - webSocketAdapter.emit(WebSocketAdapterEvent.Heartbeat); - totalSubs += webSocketAdapter.getSubscriptions().size; - } - }); - console.log(`[HEARTBEAT] Clients: ${totalClients} | Total subscriptions: ${totalSubs} | Heap Used: ${(process.memoryUsage().heapUsed / 1024 / 1024).toFixed(1)} MB`); - } - ``` -3. View the live output via Docker logs: - ```bash - docker compose logs -f nostream - ``` ## Export Events Export all stored events to a [JSON Lines](https://jsonlines.org/) (`.jsonl`) file. Each line is a valid NIP-01 Nostr event JSON object. The export streams rows from the database using cursors, so it works safely on relays with millions of events without loading them into memory. @@ -754,6 +585,7 @@ Any changes made to the settings file will be read on the next start. Default settings can be found under `resources/default-settings.yaml`. Feel free to copy it to `nostream/.nostr/settings.yaml` if you would like to have a settings file before running the relay first. See [CONFIGURATION.md](CONFIGURATION.md) for a detailed explanation of each environment variable and setting. + ## Dev Channel For development discussions, please use the [Nostr Typescript Relay Dev Group](https://t.me/nostream_dev).