Commit Graph

26 Commits

Author SHA1 Message Date
laforetbrut
c7487196ec Phase 8: 20+ new config keys + 14 admin commands (/playersync)
Config (JdbcConfig.java completely restructured into sections):

  connection
    host, port, use_ssl, user_name, password, db_name, table_prefix, Server_id
  general
    sync_world, sync_advancements, kick_when_already_online,
    kick_message, kick_grace_period_ms, use_legacy_serialization,
    item_placeholder_title_override, item_placeholder_description_override
  save_triggers
    auto_save_interval_minutes (0-1440, default 10)
    save_on_dimension_change (default false)
    save_on_death (default true)
    save_on_respawn (default true)
  sync_toggles
    sync_inventory, sync_ender_chest, sync_xp, sync_effects,
    sync_health_food, sync_curios, sync_accessories, sync_backpacks,
    sync_cosmetic_armor, sync_refined_storage (all default true)
  performance
    heartbeat_interval_seconds (5-600, default 30)
    peer_stale_threshold_seconds (10-3600, default 60)
    join_poll_max_attempts (10-600, default 120)
    join_poll_interval_ms (100-5000, default 500)
    pool_stats_interval_minutes (0-1440, default 5)
    hikari_pool_max_size (1-200, default 15)
    hikari_leak_threshold_ms (2000-600000, default 25000)
  safety
    refuse_empty_inventory_write (default true) — enforced in writeSnapshotToDB
    max_inventory_size_bytes (default 10 MB)
    skip_saves_when_tps_below (0-20, default 0 = never)
  observability
    log_structured_json (future use)
    log_rotation_size_mb (default 10)
    log_rotation_max_files (default 5)

Wiring
  - HeartbeatService reads heartbeat_interval_seconds at start.
  - PoolStatsReporter reads pool_stats_interval_minutes (0 disables).
  - doPlayerJoin poll uses join_poll_max_attempts + join_poll_interval_ms +
    peer_stale_threshold_seconds.
  - writeSnapshotToDB: refuse_empty guard + max_inventory_size_bytes guard
    before core UPDATE. Both log via SyncLogger.dataLoss / .nbtAnomaly.
  - Restore-side toggles: applyCuriosFromData, applyAccessoriesFromData,
    applyCosmeticArmorFromData, doBackPackRestore, restoreRefinedStorageDisks
    all short-circuit when their toggle is false.

Commands — new /playersync tree (perm level 2 required):

  status             — server id + heartbeat age + exec/Hikari stats + online
  poolstats          — log current stats immediately
  flush [player]     — force save all / one
  info <player>      — DB row metadata
  dump <player>      — dump full DB row to server log
  resync <player>    — clear synced tag + kick to force re-restore
  wipe <player> confirm  — DELETE all rows (DANGER, double-keyword required)
  orphans            — list stuck online=1 rows on dead peers
  clearorphans [id]  — clear orphans (global or by server_id)
  peers              — list peer servers with ALIVE/STALE/STOPPED tag
  peerkill <id>      — force-disable a zombie peer
  cleanup            — orphans + stale peers in one shot
  reload             — note about runtime reload scope
  help               — in-chat command reference

Every command logs to SyncLogger as ADMIN_<OP> for audit trail.

Infrastructure
  - JDBCsetUp.executePreparedUpdateRet(String, Object...) returns rows-affected
    for commands that need meaningful counts.
  - VanillaSync.getExecutor() exposes the thread pool for read-only stats access
    from admin commands (replaces reflection use in PoolStatsReporter eventually).
2026-04-22 06:34:02 +02:00
laforetbrut
bd0482cb76 Phase 5: structured logging + periodic pool-stats reporter
SyncLogger additions
  - containerForceClosed(uuid, reason)
  - modCompatSkip / modCompatSaved / modCompatRestored (per-mod tracing)
  - storageSave(storageUuid, kind, detail) for backpack/SS/RS2 lines
  - poolStats(exec active/queue/idle, hikari active/idle)
  - warnPlayer / nbtAnomaly generic helpers

PoolStatsReporter.java
  - Dedicated single-thread daemon scheduler, 5-min cadence.
  - Reads VanillaSync.executorService stats via reflection.
  - Reads HikariCP MBean via new JDBCsetUp.getPoolMXBean().
  - Emits WARN logs when executor queue > 400/512 or Hikari active >= 14/15
    so admins see saturation trends before they become outages.

JDBCsetUp.getPoolMXBean()
  - Public accessor for the HikariCP pool MBean. Returns null when pool
    is uninitialised / closed.

Wire-in: PlayerSync.onServerStarting starts the reporter, onServerShutdown
stops it before pool close.

Instrumentation
  - VanillaSync.onPlayerLogout logs containerForceClosed for self + viewer
    containers.
  - ModCompatSync.snapshotAccessories logs modCompatSkip when cap==null.
2026-04-22 06:03:52 +02:00
laforetbrut
13de5b65c0 Fix backpack/curios dup, perf overhaul, drop chat+cobblemon
Root cause of backpack duplication: Sophisticated Backpacks'
setBackpackContents merges shallowly when the UUID exists, so stale
sub-tags survived every restore. doBackPackRestore now calls
removeBackpackContents before setBackpackContents for a clean replace.

Curios cosmetic stacks (getCosmeticStacks) are now snapshotted, applied,
restored and cached on all paths. Old-format rows without the "cos:"
prefix still parse unchanged, so existing DB data is preserved on upgrade.

closeContainer no longer matches by class-name substring (was closing
unrelated mod menus containing "curio"/"accessor"). Only menus whose
slots reference the disconnecting player's inventory/ender-chest are
closed.

Thread-safety: Sophisticated Storage contents are now snapshotted on the
main thread (snapshotSSData + saveSSSnapshots) instead of read from a
background thread racing with world ticks.

Event priority / defensive guards:
- onPlayerDeath is now EventPriority.LOW and skips cancelled events so
  Revive Me / Corail Tombstone's cancel runs first.
- onServerStarting short-circuits on integrated (single-player) servers
  to avoid noisy MySQL connection attempts.

Observability:
- executeBatchTransaction now returns per-statement row counts.
- writeSnapshotToDB calls SyncLogger.guardBlocked when the core UPDATE
  silently no-ops (another server claimed last_server).
- SyncLogger uses a daemon scheduler that flushes every 500 ms; shutdown
  happens after parallel saves so final save logs are no longer dropped.
- Rollback failures inside executeBatchTransaction and
  refreshInventoryForInputOutput are now logged instead of swallowed.

HikariCP retuned: maxPoolSize 25->15, connectionTimeout 30->10s,
idleTimeout 600->300s, leakDetectionThreshold 10->25s (covers worst-case
join polling without log spam).

New table_prefix config option (Tables helper) lets a user share one
MySQL database with other mods without table-name collisions. Default
is empty to preserve backward compatibility.

Reflection Methods for NeoForge AttachmentHolder are resolved once in
a static initializer and cached.

Chat sync and Cobblemon integration removed:
- Chat sync: 319 LoC of socket/thread code guarded by a config flag that
  defaulted to false; orphaned config keys are silently ignored by the
  NeoForge ModConfig loader, so no crash on upgrade.
- Cobblemon: 297 LoC of mixins that ran synchronous JDBC on the main
  thread and built SQL with raw UUID concatenation. The existing
  cobblemon table in the DB is left untouched on upgrade.

Also fixes cobblemon ALTER TABLE running blindly on every boot
(alterColumnIfNeeded helper checks INFORMATION_SCHEMA first).

Author: vyrriox
2026-04-22 02:50:26 +02:00
laforetbrut
57f7925c2f Perf: MySQL connection tuning, batch transactions, leak detection
MySQL connection string optimizations:
- rewriteBatchedStatements=true: rewrites batch INSERTs into multi-row (5-30x)
- cachePrepStmts=true + useServerPrepStmts=true: server-side prepared
  statement caching, avoids re-parsing identical queries (15-25% CPU reduction)
- prepStmtCacheSize=256: keeps 256 compiled statements warm
- useCompression=true: compresses network traffic (40-60% for large NBT blobs)
- tcpNoDelay=true: disables Nagle's algorithm for lower latency

Batch transaction for writeSnapshotToDB:
- New JDBCsetUp.executeBatchTransaction() executes multiple SQL statements
  in a SINGLE transaction on ONE connection with automatic rollback.
- writeSnapshotToDB now batches all 4-8 queries (player_data + curios +
  mod_player_data) into one connection borrow + one commit.
- Previous: 4-8 separate getConnection() + executeUpdate() + close() calls
  per player save = 4-8 network round-trips.
- Now: 1 getConnection() + N executeUpdate() + 1 commit() + 1 close()
  = 1 network round-trip for the transaction.
- With 35 players: 140-280 connection borrows → 35 connection borrows.

HikariCP leak detection:
- Added leakDetectionThreshold=10000ms to detect connections held > 10s

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:06:22 +02:00
laforetbrut
b4d863efa2 Perf: staggered auto-save, pool scaling, cached kick check
CRITICAL PERF - Staggered auto-save:
- Old: all 35 players snapshotted in ONE tick → 770-3605ms MSPT spike
  (15-36 second TPS drop every 5 minutes)
- New: queue filled every 5min, drained 1 player/tick → max 22-103ms/tick
- autoSaveQueue processes one player per server tick, imperceptible impact

CRITICAL PERF - Pool scaling for 35+ players:
- Thread pool: 2-8 → 4-16 threads, queue 256 → 512
  Prevents CallerRunsPolicy from executing DB tasks on main thread
- HikariCP: 10 → 25 max connections, 2 → 4 min idle
  Prevents connection starvation during concurrent saves

HIGH PERF - Cached kick check (eliminates main thread DB queries):
- doPlayerConnect (network thread) caches online/lastServer/serverAlive
- onPlayerLoggedInKickCheck (MAIN thread) reuses cached result
- Fast path: 1 DB query on main thread instead of 2-4
- Fallback: full DB check if cache miss (race condition safety)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 11:33:02 +02:00
laforetbrut
59bd884263 perf: zero JDBC on server thread + HikariCP + parallel shutdown + audit fixes
- Migrate connection pool from manual LinkedBlockingQueue to HikariCP
  (eliminates isValid() ping on every query visible in Spark profiler)
- Move ALL DB writes off server thread: logout uses snapshot+async+latch,
  shutdown uses snapshot+CompletableFuture.allOf for parallel saves
- Pre-read curios/accessories/cosmeticarmor/attachments on background
  thread during login (4-7 fewer DB queries on main thread per login)
- Auto-save interval increased to 5 minutes
- Fix pool shutdown ordering: shutdownPool() now runs AFTER all shutdown
  saves complete (previously could fire before, silently losing all data)
- Fix connection leak in executeQuery/executePreparedQuery when
  prepareStatement throws (leaked connections exhaust HikariCP pool)
- Fix duplication bug: saveStorageContents guard used nbt.size()<=1 which
  blocked legitimately emptied backpacks from saving to DB
- Fix stale SaveToFile overwriting logout: check playerLocks.containsKey
  before writing to prevent stale background task from regressing data
- Remove LIMIT 1000 on startup online=0 reset (could leave players stuck)
- Add executorService.shutdown() on server stop to prevent JVM hang
- Add apply methods (applyCuriosFromData, applyAccessoriesFromData, etc.)
  to separate entity writes from DB reads for thread-safe restore
- Add UUID collectors (collectBackpackUuids, collectSSUuids) and
  background save methods for snapshot+async logout/shutdown pattern
2026-03-29 18:58:27 +02:00
laforetbrut
d60b8eb01e Add connection pool - fix 10% server thread usage from MySQL connects
Spark showed PlayerSync consuming 10.16% of the server thread, almost
entirely from DriverManager.getConnection() (TCP handshake + MySQL auth
+ USE db) called for EVERY single query. With auto-save every 60s,
each player generated ~6 new connections per save cycle on main thread.

FIX: Simple connection pool (LinkedBlockingQueue, 5 connections).
- Connections are reused instead of opened/closed per query
- isValid(2) check before reuse to detect dead connections
- returnConnection() puts connections back in pool instead of closing
- QueryResult.close() also returns to pool
- autoReconnect=true in JDBC URL for resilience
- shutdownPool() for clean server stop
- Non-database connections (startup DDL) bypass the pool

Expected improvement: ~90% reduction in MySQL overhead on server thread.

Vyrriox
2026-03-26 21:13:17 +01:00
laforetbrut
484f1a8c05 Final audit: fix ghost-online, SQL injection, resource leak, NPE
CRITICAL-1/2: Remove duplicate online=1 writes from doPlayerJoin.
The synchronous onPlayerLoggedInKickCheck already sets online=1.
The background thread writes raced with logout's online=0, permanently
locking players as "online" after crash-disconnect during join.

HIGH-1: Startup SQL uses PreparedStatement for server_id (was string concat).
HIGH-2: update() method now uses try-with-resources for PreparedStatement.
HIGH-3: NPE guard in RS2 data file logging when getRS2DataFile returns null.

Vyrriox
2026-03-26 18:33:00 +01:00
laforetbrut
03b57c3e6b Fix critical sync bugs, security, and add Sophisticated Storage support
- Fix advancements disappearing: use PreparedStatements for all SQL with
  user data (advancement JSON contains chars that broke string-concat SQL),
  add null safety for advancement file
- Fix multi-server kick: run doPlayerConnect synchronously instead of async
  (players could join before the duplicate check completed)
- Fix Curios disappearing: clear slots AFTER validating data exists (not
  before), use CuriosCache for dead players on logout instead of empty API
- Fix Sophisticated Storage items: add storeSophisticatedStorageItems() and
  restoreSophisticatedStorageItems() to sync packed barrels/shulkers/chests
- Anti-duplication: clear all inventories before restoring from DB on join
- Fix tick counter: remove LevelTickEvent (fired per dimension = 3x too
  fast), merge heartbeat into ServerTickEvent
- Fix connection leaks: use try-with-resources for all QueryResult
- Fix logout order: save data BEFORE marking player offline
- Skip auto-save for dead/unsynced players to prevent saving empty data

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 11:04:00 +01:00
mlus
201e63a322 Fix SQL syntax by adding backticks around database and table names 2026-02-24 00:11:06 +08:00
mlus
29da0f28ad recently commit port 2025-11-28 21:00:25 +08:00
EoD
a510b091db add trace logs for all SQL queries
Can be enabled by starting minecraft with
-Dforge.logging.console.level=trace
2025-04-25 20:40:16 +00:00
paulm
439c7ee5bb Addeed Sophisticated Backpack Mod compatibility 2025-03-20 05:26:15 +01:00
mlus
bb45488186 fix #22 2024-09-24 21:46:12 +08:00
mlus
053758e6cc And for update 2024-08-04 17:51:07 +08:00
mlus
bd4694e44b DataBase Fixed 2024-08-04 17:46:38 +08:00
mlus
50e146d7fe Chat Sync performance enhancement 2024-08-04 14:20:25 +08:00
mlus
b3352fde51 1.3.0 update 2024-02-11 17:34:19 +08:00
mlus-Asuka
d32ae52537 update 1.2.1 2023-09-29 16:50:26 +08:00
mlus-Asuka
45e13f1199 1.2.0 release 2023-08-09 16:08:23 +08:00
mlus-Asuka
5ef817a9b7 Use HikariPool instead of original driver.Merge Fork from KK1ve. 2023-08-03 22:05:43 +08:00
mlus-Asuka
30b1690a73 fix chat and effects bugs. 2023-02-16 14:26:24 +08:00
mlus-Asuka
89e74813e2 Add chat message sync 2022-12-21 19:42:56 +08:00
mlus-Asuka
7dc5f0151b fix bug 2022-12-09 22:47:56 +08:00
mlus-Asuka
4d3d8afe3d add curios support 2022-12-08 23:02:50 +08:00
mlus-Asuka
ef153d26c3 Initial commit 2022-12-08 16:59:20 +08:00