Bulk perf-test data seeder (sm seed) for Products, Orders, AuditLogs#118
Draft
antosubash wants to merge 2 commits intomainfrom
Draft
Bulk perf-test data seeder (sm seed) for Products, Orders, AuditLogs#118antosubash wants to merge 2 commits intomainfrom
sm seed) for Products, Orders, AuditLogs#118antosubash wants to merge 2 commits intomainfrom
Conversation
New `sm seed` CLI command (and underlying `tools/SimpleModule.PerfSeeder` console app) for populating the database with large volumes of realistic data so list endpoints, joins, and time-range queries can be benchmarked against realistic table sizes. Defaults to 1M products, 100K orders (~250K items), 500K audit entries; all overridable via --count. Uses EF Core batched inserts with AutoDetectChangesEnabled=false and ChangeTracker.Clear per batch; wraps each module's insert phase in a single transaction. Enables WAL + relaxed sync pragmas on SQLite for bulk throughput. Provider-agnostic: works with the default SQLite setup out of the box, also supports PostgreSQL and SQL Server. Smoke-tested against fresh SQLite at ~12K rows/s for Products in Release (projects to ~85s for 1M). Data is deterministic via --seed (default 42). The CLI command shells out to the PerfSeeder project so `sm` doesn't need to pull the Products/Orders/AuditLogs modules into its dependency graph. --create-schema calls EnsureCreated when migrations aren't available (e.g., a fresh dev DB).
Audit entries are emitted by the audit interceptor when entities change, so seeding them directly would produce inconsistent data (decoupled from any actual operation, Module/Path fields guessed). Drop the AuditLogs target and scope the seeder to Products + Orders. Audit coverage now accumulates naturally from real CRUD traffic post-seed. Also fix multi-module --create-schema: EnsureCreated skips table creation when any tables exist in the DB, so seeding Products then Orders into the same file failed on the second module. Switch to the IRelationalDatabaseCreator.CreateTables pattern (same as SimpleModuleWebApplicationFactory.EnsureTablesCreated), swallowing duplicate-table errors so each module context creates only its missing tables.
Deploying simplemodule-website with
|
| Latest commit: |
73044d2
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://5afefc63.simplemodule-website.pages.dev |
| Branch Preview URL: | https://claude-perf-test-data-seedin.simplemodule-website.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
sm seedCLI subcommand that populates the configured database with large volumes of realistic test data, so list endpoints, joins, and time-range queries can be benchmarked against populated tables instead of empty/near-empty ones.Defaults: 1M Products, 100K Orders (~250K items), 500K AuditLogs — all overridable via
--count. 1M is a reasonable target for the flat Products table on PostgreSQL or SQLite; keeping Orders at 100K avoids blowing up items to multi-millions (composite PK, items 1–5 per order).What it does
sm seedcommand in the existing Spectre.Console CLI.tools/SimpleModule.PerfSeeder/that references the three target modules directly. The CLI shells out to it sosmitself doesn't need module references.AutoDetectChangesEnabled=false,ChangeTracker.Clearper batch, single transaction wrapping each module phase). Provider-agnostic — works with the default SQLite config, PostgreSQL, or SQL Server.PRAGMA journal_mode=WAL,synchronous=NORMAL,temp_store=MEMORYfor the seeder's connection.--seed(default 42).--truncatepreserves the 10 migration-seed Products (Id ≤ 10); Orders truncate deletes children first.Usage
Options:
--module(products|orders|auditlogs|all),--count,--connection,--provider,--batch-size(default 5000),--seed(default 42),--truncate,--create-schema.Test plan
sm seed --helpprints the full option setsm seed --truncateagainst a developer machine — verify 1M Products, 100K Orders, 500K AuditLogs land cleanlyGET /api/productsand spot-check response times / EXPLAIN planssm seed --provider PostgreSql --connection ...against a local PG (blocked on the pre-existing migration drift for non-SQLite providers — orthogonal to this PR)Notes / scope
SimpleModule.PerfSeederconsole app can also be invoked directly (dotnet run --project tools/SimpleModule.PerfSeeder -- ...).dotnet testby convention (no Tests suffix).Generated by Claude Code