From 60175cf714b7e6837699e8919b83b3996dc9829d Mon Sep 17 00:00:00 2001 From: Denis Angell Date: Sat, 28 Mar 2026 20:28:42 +0100 Subject: [PATCH] Add .sentinel/ project config for automated agents Commands (develop, merge, version) define the three lifecycle workflows: - develop: build, test, amendment creation when features change behavior - merge: conflict resolution rules specific to rippled's file structure - version: amendment activation lifecycle (Supported/VoteBehavior) Skills provide domain knowledge (consensus, crypto, ledger, etc.) that agents use when working on this codebase. Co-Authored-By: Claude Opus 4.6 --- .sentinel/commands/develop.md | 530 ++++++++++++++++++++++++++ .sentinel/commands/merge.md | 228 +++++++++++ .sentinel/commands/version.md | 82 ++++ .sentinel/skills/index.md | 126 ++++++ .sentinel/skills/soul/consensus.md | 86 +++++ .sentinel/skills/soul/cryptography.md | 59 +++ .sentinel/skills/soul/ledger.md | 63 +++ .sentinel/skills/soul/nodestore.md | 52 +++ .sentinel/skills/soul/peering.md | 60 +++ .sentinel/skills/soul/protocol.md | 64 ++++ .sentinel/skills/soul/rpc.md | 61 +++ .sentinel/skills/soul/shamap.md | 61 +++ .sentinel/skills/soul/sql.md | 60 +++ .sentinel/skills/soul/test.md | 75 ++++ .sentinel/skills/soul/transactors.md | 102 +++++ .sentinel/skills/soul/websockets.md | 62 +++ 16 files changed, 1771 insertions(+) create mode 100644 .sentinel/commands/develop.md create mode 100644 .sentinel/commands/merge.md create mode 100644 .sentinel/commands/version.md create mode 100644 .sentinel/skills/index.md create mode 100644 .sentinel/skills/soul/consensus.md create mode 100644 .sentinel/skills/soul/cryptography.md create mode 100644 .sentinel/skills/soul/ledger.md create mode 100644 .sentinel/skills/soul/nodestore.md create mode 100644 .sentinel/skills/soul/peering.md create mode 100644 .sentinel/skills/soul/protocol.md create mode 100644 .sentinel/skills/soul/rpc.md create mode 100644 .sentinel/skills/soul/shamap.md create mode 100644 .sentinel/skills/soul/sql.md create mode 100644 .sentinel/skills/soul/test.md create mode 100644 .sentinel/skills/soul/transactors.md create mode 100644 .sentinel/skills/soul/websockets.md diff --git a/.sentinel/commands/develop.md b/.sentinel/commands/develop.md new file mode 100644 index 00000000000..ecc1f0e85a5 --- /dev/null +++ b/.sentinel/commands/develop.md @@ -0,0 +1,530 @@ +# XRPL Amendment Creation Skill + +This skill guides you through creating XRPL amendments, whether for brand new features or fixes/extensions to existing functionality. + +## Amendment Types + +There are two main types of amendments: + +1. **New Feature Amendment** (`feature{Name}`) - Entirely new functionality +2. **Fix/Extension Amendment** (`fix{Name}`) - Modifications to existing functionality + +## Step 1: Determine Amendment Type + +Ask the user: +- Is this a **brand new feature** (new transaction type, ledger entry, or capability)? +- Or is this a **fix or extension** to existing functionality? + +--- + +## For NEW FEATURE Amendments + +### Checklist: + +#### 1. Feature Definition in features.macro +**ONLY FILE TO EDIT:** `include/xrpl/protocol/detail/features.macro` + +- [ ] Add to TOP of features.macro (reverse chronological order): +``` +XRPL_FEATURE(YourFeatureName, Supported::no, VoteBehavior::DefaultNo) +``` +- [ ] This creates the variable `featureYourFeatureName` automatically +- [ ] Follow naming convention: Use the feature name WITHOUT the "feature" prefix +- [ ] Examples: `Batch` → `featureBatch`, `LendingProtocol` → `featureLendingProtocol` + +#### 2. Support Status +- [ ] Start with `Supported::no` during development +- [ ] Change to `Supported::yes` when ready for network voting +- [ ] Use `VoteBehavior::DefaultNo` (validators must explicitly vote for it) + +#### 3. Code Implementation +- [ ] Implement new functionality (transaction type, ledger entry, etc.) +- [ ] Add feature gate check in preflight: +```cpp +if (!env.current()->rules().enabled(feature{Name})) +{ + return temDISABLED; +} +``` + +#### 4. Disable Route Handling +- [ ] Ensure transaction returns `temDISABLED` when amendment is disabled +- [ ] Implement early rejection in preflight/preclaim phase +- [ ] Add appropriate error messages + +#### 5. Test Implementation +Create comprehensive test suite with this structure: + +```cpp +class {FeatureName}_test : public beast::unit_test::suite +{ +public: + void testEnable(FeatureBitset features) + { + testcase("enabled"); + + // Test with feature DISABLED + { + auto const amendNoFeature = features - feature{Name}; + Env env{*this, amendNoFeature}; + + env(transaction, ter(temDISABLED)); + } + + // Test with feature ENABLED + { + Env env{*this, features}; + + env(transaction, ter(tesSUCCESS)); + // Validate new functionality works + } + } + + void testPreflight(FeatureBitset features) + { + testcase("preflight"); + // Test malformed transaction validation + } + + void testPreclaim(FeatureBitset features) + { + testcase("preclaim"); + // Test signature and claim phase validation + } + + void testWithFeats(FeatureBitset features) + { + testEnable(features); + testPreflight(features); + testPreclaim(features); + // Add feature-specific tests + } + + void run() override + { + using namespace test::jtx; + auto const all = supported_amendments(); + testWithFeats(all); + } +}; +``` + +#### 6. Test Coverage Requirements +- [ ] Test amendment DISABLED state (returns `temDISABLED`) +- [ ] Test amendment ENABLED state (returns `tesSUCCESS`) +- [ ] Test malformed transactions +- [ ] Test signature validation +- [ ] Test edge cases specific to feature +- [ ] Test amendment transition behavior + +#### 7. Documentation +- [ ] Create specification document (XLS_{NAME}.md) +- [ ] Document new transaction types, ledger entries, or capabilities +- [ ] Create test plan document +- [ ] Document expected behavior when enabled/disabled + +--- + +## For FIX/EXTENSION Amendments + +### Checklist: + +#### 1. Fix Definition in features.macro +**ONLY FILE TO EDIT:** `include/xrpl/protocol/detail/features.macro` + +- [ ] Add to TOP of features.macro (reverse chronological order): +``` +XRPL_FIX(YourFixName, Supported::no, VoteBehavior::DefaultNo) +``` +- [ ] This creates the variable `fixYourFixName` automatically +- [ ] Follow naming convention: Use the fix name WITHOUT the "fix" prefix (it's added automatically) +- [ ] Examples: `TokenEscrowV1` → `fixTokenEscrowV1`, `DirectoryLimit` → `fixDirectoryLimit` +- [ ] Start with `Supported::no` during development, change to `Supported::yes` when ready + +#### 2. Backward Compatibility Implementation +**Critical**: Use enable() with if/else to preserve existing functionality + +```cpp +// Check if fix is enabled +bool const fix{Name} = env.current()->rules().enabled(fix{Name}); + +// Conditional logic based on amendment state +if (fix{Name}) +{ + // NEW behavior with fix applied + // This is the corrected/improved logic +} +else +{ + // OLD behavior (legacy path) + // Preserve original functionality for backward compatibility +} +``` + +**Alternative pattern with ternary operator:** +```cpp +auto& viewToUse = sb.rules().enabled(fix{Name}) ? sb : legacyView; +``` + +#### 3. Multiple Fix Versions Pattern +For iterative fixes, use version checking: + +```cpp +bool const fixV1 = rv.rules().enabled(fixXahauV1); +bool const fixV2 = rv.rules().enabled(fixXahauV2); + +switch (transactionType) +{ + case TYPE_1: + if (fixV1) { + // Behavior with V1 fix + } else { + // Legacy behavior + } + break; + + case TYPE_2: + if (fixV2) { + // Behavior with V2 fix + } else if (fixV1) { + // Behavior with only V1 + } else { + // Legacy behavior + } + break; +} +``` + +#### 4. Test Both Paths +Always test BOTH enabled and disabled states: + +```cpp +void testFix(FeatureBitset features) +{ + testcase("fix behavior"); + + for (bool withFix : {false, true}) + { + auto const amend = withFix ? features : features - fix{Name}; + Env env{*this, amend}; + + // Setup test scenario + env.fund(XRP(1000), alice); + env.close(); + + if (!withFix) + { + // Test OLD behavior (before fix) + env(operation, ter(expectedErrorWithoutFix)); + // Verify old behavior is preserved + } + else + { + // Test NEW behavior (after fix) + env(operation, ter(expectedErrorWithFix)); + // Verify fix works correctly + } + } +} +``` + +#### 5. Security Fix Pattern +For security-critical fixes (like fixBatchInnerSigs): + +```cpp +// Test vulnerability exists WITHOUT fix +{ + auto const amendNoFix = features - fix{Name}; + Env env{*this, amendNoFix}; + + // Demonstrate vulnerability + // Expected: Validity::Valid (WRONG - vulnerable!) + BEAST_EXPECT(result == Validity::Valid); +} + +// Test vulnerability is FIXED WITH amendment +{ + Env env{*this, features}; + + // Demonstrate fix + // Expected: Validity::SigBad (CORRECT - protected!) + BEAST_EXPECT(result == Validity::SigBad); +} +``` + +#### 6. Test Coverage Requirements +- [ ] Test fix DISABLED (legacy behavior preserved) +- [ ] Test fix ENABLED (new behavior applied) +- [ ] Test amendment transition +- [ ] For security fixes: demonstrate vulnerability without fix +- [ ] For security fixes: demonstrate protection with fix +- [ ] Test edge cases that triggered the fix +- [ ] Test combinations with other amendments + +#### 7. Documentation +- [ ] Document what was broken/suboptimal +- [ ] Document the fix applied +- [ ] Document backward compatibility behavior +- [ ] Create test summary showing both paths + +--- + +## Best Practices for All Amendments + +### 1. Naming Conventions +- New features: `feature{DescriptiveName}` (e.g., `featureBatch`, `featureHooks`) +- Fixes: `fix{IssueDescription}` (e.g., `fixBatchInnerSigs`, `fixNSDelete`) +- Use CamelCase without underscores + +### 2. Feature Flag Checking +```cpp +// At the point where behavior diverges: +bool const amendmentEnabled = env.current()->rules().enabled(feature{Name}); + +// Or in view/rules context: +if (!rv.rules().enabled(feature{Name})) + return {}; // or legacy behavior +``` + +### 3. Error Codes +- New features when disabled: `temDISABLED` +- Fixes may return different validation errors based on state +- Document all error code changes + +### 4. Test Structure Template +```cpp +class Amendment_test : public beast::unit_test::suite +{ +public: + // Core tests + void testEnable(FeatureBitset features); // Enable/disable states + void testPreflight(FeatureBitset features); // Validation + void testPreclaim(FeatureBitset features); // Claim phase + + // Feature-specific tests + void test{SpecificScenario1}(FeatureBitset features); + void test{SpecificScenario2}(FeatureBitset features); + + // Master orchestrator + void testWithFeats(FeatureBitset features) + { + testEnable(features); + testPreflight(features); + testPreclaim(features); + test{SpecificScenario1}(features); + test{SpecificScenario2}(features); + } + + void run() override + { + using namespace test::jtx; + auto const all = supported_amendments(); + testWithFeats(all); + } +}; + +BEAST_DEFINE_TESTSUITE(Amendment, app, ripple); +``` + +### 5. Documentation Files +Create these files: +- **Specification**: `XLS_{FEATURE_NAME}.md` - Technical specification +- **Test Plan**: `{FEATURE}_COMPREHENSIVE_TEST_PLAN.md` - Test strategy +- **Test Summary**: `{FEATURE}_TEST_IMPLEMENTATION_SUMMARY.md` - Test results +- **Review Findings**: `{FEATURE}_REVIEW_FINDINGS.md` (if applicable) + +### 6. Amendment Transition Testing +Test the moment an amendment activates: + +```cpp +void testAmendmentTransition(FeatureBitset features) +{ + testcase("amendment transition"); + + // Start with amendment disabled + auto const amendNoFeature = features - feature{Name}; + Env env{*this, amendNoFeature}; + + // Perform operations in disabled state + env(operation1, ter(temDISABLED)); + + // Enable amendment mid-test (if testing mechanism supports it) + // Verify state transitions correctly + + // Perform operations in enabled state + env(operation2, ter(tesSUCCESS)); +} +``` + +### 7. Cache Behavior +If amendment affects caching (like fixBatchInnerSigs): +- [ ] Test cache behavior without fix +- [ ] Test cache behavior with fix +- [ ] Document cache invalidation requirements + +### 8. Multi-Amendment Combinations +Test interactions with other amendments: + +```cpp +void testMultipleAmendments(FeatureBitset features) +{ + // Test all combinations + for (bool withFeature1 : {false, true}) + for (bool withFeature2 : {false, true}) + { + auto amend = features; + if (!withFeature1) amend -= feature1; + if (!withFeature2) amend -= feature2; + + Env env{*this, amend}; + // Test interaction behavior + } +} +``` + +### 9. Performance Considerations +- [ ] Minimize runtime checks (cache `rules().enabled()` result if used multiple times) +- [ ] Avoid nested feature checks where possible +- [ ] Document performance impact + +### 10. Code Review Checklist +- [ ] Both enabled/disabled paths are tested +- [ ] Backward compatibility is preserved (for fixes) +- [ ] Error codes are appropriate +- [ ] Documentation is complete +- [ ] Security implications are considered +- [ ] Cache behavior is correct +- [ ] Edge cases are covered + +--- + +## Common Patterns Reference + +### Pattern: New Transaction Type +```cpp +// In transactor code: +TER doApply() override +{ + if (!ctx_.view().rules().enabled(feature{Name})) + return temDISABLED; + + // New transaction logic here + return tesSUCCESS; +} +``` + +### Pattern: New Ledger Entry Type +```cpp +// In ledger entry creation: +if (!view.rules().enabled(feature{Name})) + return temDISABLED; + +auto const sle = std::make_shared(ltNEW_TYPE, keylet); +view.insert(sle); +``` + +### Pattern: Behavioral Fix +```cpp +// At decision point: +bool const useFix = view.rules().enabled(fix{Name}); + +if (useFix) +{ + // Corrected behavior + return performCorrectValidation(); +} +else +{ + // Legacy behavior (preserved for compatibility) + return performLegacyValidation(); +} +``` + +### Pattern: View Selection +```cpp +// Select which view to use based on amendment: +auto& applyView = sb.rules().enabled(feature{Name}) ? newView : legacyView; +``` + +--- + +## Example Workflows + +### Workflow 1: Creating a New Feature Amendment + +1. User requests: "Add a new ClaimReward transaction type" +2. Skill asks: "What should the amendment be called? (e.g., BalanceRewards - without 'feature' prefix)" +3. Add to features.macro: +``` +XRPL_FEATURE(BalanceRewards, Supported::no, VoteBehavior::DefaultNo) +``` +4. Implement transaction with `temDISABLED` gate using `featureBalanceRewards` +5. Create test suite with testEnable, testPreflight, testPreclaim +6. Run tests with amendment enabled and disabled +7. When ready, update to `Supported::yes` in features.macro +8. Create specification document +9. Review checklist + +### Workflow 2: Creating a Fix Amendment + +1. User requests: "Fix the signature validation bug in batch transactions" +2. Skill asks: "What should the fix be called? (e.g., BatchInnerSigs - without 'fix' prefix)" +3. Add to features.macro: +``` +XRPL_FIX(BatchInnerSigs, Supported::no, VoteBehavior::DefaultNo) +``` +4. Implement fix with if/else using `fixBatchInnerSigs` to preserve old behavior +5. Create test demonstrating vulnerability without fix +6. Create test showing fix works when enabled +7. When ready, update to `Supported::yes` in features.macro +8. Document both code paths +9. Review checklist + +--- + +## Quick Reference: File Locations + +- **Amendment definitions (ONLY place to add)**: `include/xrpl/protocol/detail/features.macro` +- **Feature.h (auto-generated, DO NOT EDIT)**: `include/xrpl/protocol/Feature.h` +- **Feature.cpp (auto-generated, DO NOT EDIT)**: `src/libxrpl/protocol/Feature.cpp` +- **Test files**: `src/test/app/` or `src/test/protocol/` +- **Specifications**: Project root (e.g., `XLS_SMART_CONTRACTS.md`) +- **Test plans**: Project root (e.g., `BATCH_COMPREHENSIVE_TEST_PLAN.md`) + +## How the Macro System Works + +The amendment system uses C preprocessor macros: + +1. **features.macro** - Single source of truth (ONLY file you edit): +``` +XRPL_FEATURE(Batch, Supported::yes, VoteBehavior::DefaultNo) +XRPL_FIX(TokenEscrowV1, Supported::yes, VoteBehavior::DefaultNo) +``` + +2. **Feature.h** - Auto-generated declarations from macro: +```cpp +extern uint256 const featureBatch; +extern uint256 const fixTokenEscrowV1; +``` + +3. **Feature.cpp** - Auto-generated registrations from macro: +```cpp +uint256 const featureBatch = registerFeature("Batch", ...); +uint256 const fixTokenEscrowV1 = registerFeature("fixTokenEscrowV1", ...); +``` + +**DO NOT** modify Feature.h or Feature.cpp directly - they process features.macro automatically. + +--- + +## When to Use This Skill + +Invoke this skill when: +- Creating a new XRPL amendment +- Adding a new transaction type or ledger entry +- Fixing existing XRPL functionality +- Need guidance on amendment best practices +- Setting up amendment tests +- Reviewing amendment implementation + +The skill will guide you through the appropriate workflow based on amendment type. diff --git a/.sentinel/commands/merge.md b/.sentinel/commands/merge.md new file mode 100644 index 00000000000..a5c83422ace --- /dev/null +++ b/.sentinel/commands/merge.md @@ -0,0 +1,228 @@ +# Merge Conflict Resolution Guide + +## Description +Use when resolving merge conflicts after merging `develop` into a feature branch. Covers the key files that commonly conflict, field number allocation, namespace conventions, and the correct resolution strategy for each file type. + +## General Principles + +### Namespace +- All code uses `namespace xrpl { }` (not `ripple`). If a conflict shows `namespace ripple`, take the `xrpl` version. +- Qualified references: `xrpl::sign(...)`, `xrpl::sha256_hasher`, etc. Never `ripple::`. + +### Header Guards +- Always `#pragma once`. Never `#ifndef` include guards. If a conflict shows old-style guards, take the `#pragma once` version. + +### Copyright Headers +- The `develop` branch has removed copyright/license comment blocks from source files. If a conflict is just about the copyright header, take `develop`'s version (no header). + +### File Moves +- Many files have been moved from `src/xrpld/` to `src/libxrpl/` or from `src/xrpld/` headers to `include/xrpl/`. If git shows a file as "deleted in develop" but it exists at a new path, delete the old-path file (`git rm`) and keep the new-path version. +- Common moves: + - `src/xrpld/app/tx/detail/*.h` → `include/xrpl/tx/transactors/*.h` + - `src/xrpld/app/tx/detail/*.cpp` → `src/libxrpl/tx/*.cpp` or `src/libxrpl/tx/transactors/*.cpp` + - `src/xrpld/app/tx/detail/InvariantCheck.cpp` → `src/libxrpl/tx/invariants/InvariantCheck.cpp` + +### Transactor Migration +When merging transactors from `src/xrpld/app/tx/detail/` into `develop`, they must be fully migrated to the new libxrpl structure: + +**File locations:** +- Header: `src/xrpld/app/tx/detail/Foo.h` → `include/xrpl/tx/transactors/Foo.h` +- Source: `src/xrpld/app/tx/detail/Foo.cpp` → `src/libxrpl/tx/transactors/Foo.cpp` +- Delete the old `.cpp` (both `src/xrpld/` and `src/libxrpl/` are GLOB_RECURSE discovered — duplicates cause linker errors) +- Optionally leave a forwarding header at the old `.h` location: `#include ` + +**Header changes:** +- `#pragma once` (not `#ifndef` guards) +- `namespace xrpl` (not `ripple`) +- `#include ` (not ``) + +**Source changes:** +- `namespace xrpl` (not `ripple`) +- `#include ` (not ``) +- `#include ` (not ``) +- `#include ` for `describeOwnerDir` + +**API changes in the new Transactor:** +- `preflight1()` and `preflight2()` are **private** — transactor `preflight()` must NOT call them. The framework calls them automatically. +- Flag checking (`tfUniversalMask`) is handled by the framework via `getFlagsMask()`. Override `getFlagsMask()` only if custom flag handling is needed; otherwise the default handles `tfUniversalMask`. +- A simple `preflight()` that only did flag checks + preflight1/preflight2 should just `return tesSUCCESS;` +- `ctx_.app.journal(...)` → `ctx_.registry.journal(...)` +- `ctx_.app` does not exist in the new `ApplyContext`; use `ctx_.registry` for service access + +**transactions.macro entry:** +Every transactor must have a `#if TRANSACTION_INCLUDE` block: +```cpp +#if TRANSACTION_INCLUDE +# include +#endif +TRANSACTION(ttFOO, , Foo, ...) +``` + +### Include Paths +- `develop` uses the new macro-based transactor include system via `transactions.macro` with `TRANSACTION_INCLUDE`. Old-style explicit `#include ` lists should be replaced with the macro approach. + +## File-Specific Resolution Rules + +### `include/xrpl/protocol/detail/features.macro` + +New feature amendments go **at the top** of the active list (below the macro guard checks and `// clang-format off`), in reverse chronological order. + +``` +// Add new amendments to the top of this list. +// Keep it sorted in reverse chronological order. + +XRPL_FEATURE(MyNewFeature, Supported::yes, VoteBehavior::DefaultNo) +XRPL_FIX (ExistingFix, Supported::yes, VoteBehavior::DefaultNo) +... +``` + +Resolution: Keep both sides' new amendments. Place your feature branch's new amendments at the very top. + +### `include/xrpl/protocol/detail/sfields.macro` + +Fields are grouped by type (UINT32, UINT64, UINT128, etc.) with common and uncommon sections. Each field has a unique **type + field number** pair. + +**Resolution strategy:** +1. Keep both sides' new fields. +2. Check for field number collisions within each type. Use the next available number for your feature's fields. +3. Common fields come first, uncommon fields after (there's a comment separator). + +To find the next available field number for a type: +```bash +grep "TYPED_SFIELD.*UINT32" include/xrpl/protocol/detail/sfields.macro | sed 's/.*UINT32, *//;s/).*//' | sort -n +``` + +Similarly for OBJECT and ARRAY types (using `UNTYPED_SFIELD`): +```bash +grep "UNTYPED_SFIELD.*OBJECT" include/xrpl/protocol/detail/sfields.macro | sed 's/.*OBJECT, *//;s/).*//' | sort -n +grep "UNTYPED_SFIELD.*ARRAY" include/xrpl/protocol/detail/sfields.macro | sed 's/.*ARRAY, *//;s/).*//' | sort -n +``` + +### `include/xrpl/protocol/detail/transactions.macro` + +Transaction types have a unique **transaction type number**. New feature transactions go **at the bottom** of the active transaction list (before the system transactions starting at 100+). + +**Resolution strategy:** +1. Keep all of `develop`'s transactions. +2. Add your feature's transactions at the bottom with the next available number. +3. Use the new 7-argument `TRANSACTION` macro format: + +```cpp +/** Description of the transaction */ +#if TRANSACTION_INCLUDE +# include +#endif +TRANSACTION(ttMY_TX, , MyTx, + Delegation::delegable, + featureMyFeature, + noPriv, ({ + {sfSomeField, soeREQUIRED}, +})) +``` + +To find the next available transaction number: +```bash +grep "^TRANSACTION(" include/xrpl/protocol/detail/transactions.macro | sed 's/.*,\s*\([0-9]*\),.*/\1/' | sort -n +``` + +Note: Transaction numbers 100+ are reserved for system transactions (amendments, fees, UNL). + +### `include/xrpl/protocol/detail/ledger_entries.macro` + +Ledger entry types have a unique **ledger type number** (hex). New entries go **at the bottom** of the active list. + +**Resolution strategy:** +1. Keep all of `develop`'s entries. +2. Add your feature's entries at the bottom with the next available hex number. +3. Check for collisions: + +```bash +grep "^LEDGER_ENTRY(" include/xrpl/protocol/detail/ledger_entries.macro | grep -o '0x[0-9a-fA-F]*' | sort +``` + +### `src/libxrpl/protocol/Indexes.cpp` + +The `LedgerNameSpace` enum assigns a unique single character to each ledger entry type for index hashing. + +**Resolution strategy:** +1. Keep both sides' entries. +2. Check for character collisions. Each entry needs a unique char. +3. Find used characters: + +```bash +grep -E "^\s+[A-Z_]+ = " src/libxrpl/protocol/Indexes.cpp | sed "s/.*= '//;s/'.*//" | sort | tr -d '\n' +``` + +Also check the deprecated chars (reserved, cannot reuse): +```bash +grep "deprecated" src/libxrpl/protocol/Indexes.cpp | sed "s/.*= '//;s/'.*//" +``` + +### `src/libxrpl/protocol/InnerObjectFormats.cpp` + +Inner object formats define the fields allowed inside array elements (e.g., `sfSigners`, `sfPasskeys`). + +**Resolution:** Keep both sides' `add(...)` calls. No numbering conflicts here — just ensure no duplicate registrations. + +### `src/libxrpl/protocol/STTx.cpp` + +The `singleSignHelper` function was refactored in `develop`: +- Parameter name: `sigObject` (not `signer`) +- Signature retrieval: `sigObject.getFieldVL(sfTxnSignature)` (not `getSignature(signer)`) +- No `fullyCanonical` parameter — canonical sig checking was simplified + +**Resolution:** Take `develop`'s function signatures and patterns. Adapt any feature-specific logic to match. + +### `src/libxrpl/protocol/SecretKey.cpp` + +**Resolution:** Use `namespace xrpl`. Keep any additional `#include` directives your feature needs (e.g., OpenSSL headers for new key types). + +### `src/libxrpl/tx/Transactor.cpp` + +The `checkSign` / `checkSingleSign` / `checkMultiSign` functions were refactored in `develop`: +- `checkSingleSign` no longer takes a `Rules` parameter +- `checkSign` has a new overload taking `PreclaimContext` +- `checkBatchSign` uses the simplified `checkSingleSign` call + +**Resolution:** Take `develop`'s function signatures. Add any new authentication checks (e.g., passkey verification) into `checkSingleSign` before the final `return tefBAD_AUTH`, using `view.read(...)` to check ledger state. + +### `src/libxrpl/tx/applySteps.cpp` + +**Resolution:** Always take `develop`'s macro-based include approach: +```cpp +#include +#pragma push_macro("TRANSACTION") +#undef TRANSACTION + +#define TRANSACTION(...) +#define TRANSACTION_INCLUDE 1 + +#include + +#undef TRANSACTION +#pragma pop_macro("TRANSACTION") +``` + +Never use the old explicit `#include ` list. + +### `src/test/jtx/impl/utility.cpp` + +The `sign` function was refactored in `develop`: +- Takes `sigObject` parameter: `sign(Json::Value& jv, Account const& account, Json::Value& sigObject)` +- Has an overload: `sign(Json::Value& jv, Account const& account)` that calls `sign(jv, account, jv)` +- Uses `xrpl::sign(...)` not `ripple::sign(...)` + +**Resolution:** Take `develop`'s function signatures. Adapt feature-specific signing logic to the new pattern. + +## Conflict Resolution Checklist + +1. List all conflicted files: `git diff --name-only --diff-filter=U` +2. Check which have actual conflict markers vs just unmerged: `grep -l "<<<<<<< HEAD" ` +3. For files without markers: stage them or `git rm` if they were moved +4. For each file with markers: + - Take `develop`'s structural changes (namespace, function signatures, macro formats) + - Preserve your feature's additions (new fields, entries, transactions, logic) + - Resolve any numbering collisions by incrementing to the next available number +5. Stage resolved files: `git add ` +6. Verify no remaining markers: `grep -r "<<<<<<< " --include="*.cpp" --include="*.h" --include="*.macro"` +7. Verify no remaining unmerged: `git diff --name-only --diff-filter=U` diff --git a/.sentinel/commands/version.md b/.sentinel/commands/version.md new file mode 100644 index 00000000000..018d2ceaa91 --- /dev/null +++ b/.sentinel/commands/version.md @@ -0,0 +1,82 @@ +# Version / Release Process + +This command handles the release lifecycle for rippled amendments. + +## Rippled Release Model + +Rippled uses an amendment-based release process. There is no traditional stage/prod deploy. +Instead, features are gated behind amendments that validators vote on. The lifecycle is: + +1. **Development** (`Supported::no`) — Feature is implemented but disabled on all networks. +2. **Release candidate** (`Supported::yes, VoteBehavior::DefaultNo`) — Feature ships in a release. + Validators can vote for it, but won't by default. +3. **Default vote** (`Supported::yes, VoteBehavior::DefaultYes`) — After sufficient testing and + validator confidence, the default vote flips. Validators must explicitly opt out. +4. **Activation** — When 80%+ of validators vote yes for 2 consecutive weeks, the amendment + activates permanently on the network. There is no rollback. + +## When to Use This Command + +Run `/version` when a feature is ready to move between lifecycle stages. + +## Workflow + +### Stage 1: Development to Release Candidate + +The feature is tested and ready to ship in a binary release. + +1. Open `include/xrpl/protocol/detail/features.macro` +2. Find your amendment entry (it should be `Supported::no`) +3. Change to `Supported::yes`: +``` +XRPL_FEATURE(YourFeature, Supported::yes, VoteBehavior::DefaultNo) +``` +4. Verify the feature gate still works: + - Build the project + - Run tests with the amendment disabled: confirm `temDISABLED` + - Run tests with the amendment enabled: confirm `tesSUCCESS` +5. This change ships in the next binary release. Validators who upgrade get the code + but must explicitly vote to enable it. + +### Stage 2: Release Candidate to Default Vote + +After the feature has been running on testnet and validators are confident: + +1. Open `include/xrpl/protocol/detail/features.macro` +2. Change vote behavior: +``` +XRPL_FEATURE(YourFeature, Supported::yes, VoteBehavior::DefaultYes) +``` +3. This is a significant change — it means validators who upgrade will automatically + vote yes unless they opt out. Only do this after: + - Testnet validation period (minimum 2 weeks recommended) + - No reported issues from early-adopting validators + - Community consensus that the feature is ready + +### Fix Amendments + +Fix amendments (`XRPL_FIX`) follow the same lifecycle but often move faster: +- Security fixes may go directly to `Supported::yes, DefaultYes` +- Bug fixes typically spend less time at `DefaultNo` + +## Checklist Before Version Change + +- [ ] All tests pass with amendment enabled AND disabled +- [ ] No merge conflicts with current `develop` +- [ ] Amendment transition test exists (disabled -> enabled mid-ledger) +- [ ] Test combinations with other recent amendments +- [ ] For `DefaultYes`: testnet validation period completed +- [ ] For `DefaultYes`: no open issues against the feature +- [ ] PR description documents the version change and rationale + +## For Non-Rippled Projects + +If this project uses a conventional deploy pipeline instead of amendments: + +1. **Stage deploy**: Push to staging environment, run integration tests +2. **Staging validation**: Verify with smoke tests, check monitoring dashboards +3. **Production deploy**: Promote to production after staging sign-off +4. **Rollback plan**: Document how to revert if issues are found post-deploy + +The specific commands and environments should be defined in the project's CI/CD config +(`.github/workflows/`, `Makefile`, deployment manifests). diff --git a/.sentinel/skills/index.md b/.sentinel/skills/index.md new file mode 100644 index 00000000000..c68a81def40 --- /dev/null +++ b/.sentinel/skills/index.md @@ -0,0 +1,126 @@ +# xrpld Codebase Skills Index + +## Description +This is the top-level guide for all best-practices skills in this repository. Use this to understand the codebase organization and find the right skill for any task. + +## When to Use Skills +Reference a skill whenever you are: +- **Writing new code** in a module - check the skill first for established patterns +- **Modifying existing code** - verify your changes follow module conventions +- **Adding a new transaction type** - see `libxrpl/tx/transactors.md` for the full template +- **Debugging** - skills list key files and common pitfalls per module +- **Reviewing code** - skills document what "correct" looks like for each module + +## Codebase Architecture + +The codebase is split into two main areas: + +### `src/libxrpl/` — The Library (skills in `.claude/skills/libxrpl/`) +Reusable library code: data types, serialization, cryptography, ledger state, transaction processing, and storage. This is the **protocol layer**. + +| Module | Responsibility | +|--------|---------------| +| `basics` | Foundational types: Buffer, Slice, base_uint, Number, logging, error contracts | +| `beast` | Support layer: Journal logging, test framework, instrumentation, IP types | +| `conditions` | Crypto-conditions (RFC): fulfillment validation, DER encoding | +| `core` | Job queue, load monitoring, hash-based message dedup | +| `crypto` | CSPRNG, secure erasure, RFC1751 encoding | +| `json` | Json::Value, parsing, serialization, StaticString optimization | +| `ledger` | ReadView/ApplyView, state tables, payment sandbox, credit ops | +| `net` | HTTP/HTTPS client, SSL certs, async I/O | +| `nodestore` | Persistent node storage: RocksDB, NuDB, Memory backends | +| `protocol` | STObject hierarchy, SField, Serializer, TER codes, Features, Keylets | +| `proto` | Protocol Buffer generated headers (gRPC API definitions) | +| `rdb` | SOCI database wrapper, checkpointing | +| `resource` | Rate limiting, endpoint tracking, abuse prevention | +| `server` | Port config, SSL/TLS, WebSocket, admin networks | +| `shamap` | SHA-256 Merkle radix tree (16-way branching, COW) | +| `tx` | Transaction pipeline: Transactor base, preflight/preclaim/doApply | + +### `src/xrpld/` — The Server Application (skills in `.claude/skills/xrpld/`) +The running rippled server: application lifecycle, consensus, networking, RPC, and peer management. This is the **application layer**. + +| Module | Responsibility | +|--------|---------------| +| `app` | Application singleton, ledger management, consensus adapters, services | +| `app/main` | Application initialization and lifecycle | +| `app/ledger` | Ledger storage, retrieval, immutable state management | +| `app/consensus` | RCL consensus adapters (bridges generic algorithm to rippled) | +| `app/misc` | Fee voting, amendments, SHAMapStore, TxQ, validators, NetworkOPs | +| `app/paths` | Payment path finding algorithm, trust line caching | +| `app/rdb` | Application-level database operations | +| `app/tx` | Application-level transaction handling | +| `consensus` | Generic consensus algorithm (CRTP-based, app-independent) | +| `core` | Configuration (Config.h), time keeping, network ID | +| `overlay` | P2P networking: peer connections, protocol buffers, clustering | +| `peerfinder` | Network discovery: bootcache, livecache, slot management | +| `perflog` | Performance logging and instrumentation | +| `rpc` | RPC handler dispatch, coroutine suspension, 40+ command handlers | +| `shamap` | Application-level SHAMap operations (NodeFamily) | + +### `include/xrpl/` — Header Files +Headers live in `include/xrpl/` and mirror the `src/libxrpl/` structure. Each skill already references its corresponding headers in the "Key Files" section. + +## Cross-Cutting Conventions + +### Error Handling +- **Transaction errors**: Return `TER` enum (tesSUCCESS, tecFROZEN, temBAD_AMOUNT, etc.) +- **Logic errors**: `Throw()`, `LogicError()`, `XRPL_ASSERT()` +- **I/O errors**: `Status` enum or `boost::system::error_code` +- **RPC errors**: Inject via `context.params` + +### Assertions +```cpp +XRPL_ASSERT(condition, "ClassName::method : description"); // Debug only +XRPL_VERIFY(condition, "ClassName::method : description"); // Always enabled +``` + +### Logging +```cpp +JLOG(j_.warn()) << "Message"; // Always wrap in JLOG macro +``` + +### Memory Management +- `IntrusiveRefCounts` + `SharedIntrusive` for shared ownership in libxrpl +- `std::shared_ptr` for shared ownership in xrpld +- `std::unique_ptr` for exclusive ownership +- `CountedObject` mixin for instance tracking + +### Feature Gating +```cpp +if (ctx.rules.enabled(featureMyFeature)) { /* new behavior */ } +``` + +### Code Organization +- Headers in `include/xrpl/`, implementations in `src/libxrpl/` or `src/xrpld/` +- `#pragma once` (never `#ifndef` guards) +- `namespace xrpl { }` for all code +- `detail/` namespace for internal helpers +- Factory functions: `make_*()` returning `unique_ptr` or `shared_ptr` + +## Subsystem Skills (soul/) + +Concise invariants, bug patterns, and review checklists for each subsystem: + +| Subsystem | File | Focus | +|-----------|------|-------| +| Consensus | `soul/consensus.md` | State machine, amendments, UNL, validations, TX ordering | +| Cryptography | `soul/cryptography.md` | Key types, signing, hashing, handshake | +| Ledger | `soul/ledger.md` | Immutability, acquisition, entry types | +| NodeStore | `soul/nodestore.md` | Backends, caching, online deletion | +| Peering | `soul/peering.md` | Overlay, connections, squelching | +| Protocol | `soul/protocol.md` | Macros, serialization format, field ordering | +| RPC | `soul/rpc.md` | Handler dispatch, roles, subscriptions | +| SHAMap | `soul/shamap.md` | COW, node types, sync, proofs | +| SQL | `soul/sql.md` | Schema, config, checkpointing | +| Testing | `soul/test.md` | JTx framework, Env setup, TER expectations, amendment gating | +| Transactors | `soul/transactors.md` | Pipeline, new TX types, signing, transactor template | +| WebSockets | `soul/websockets.md` | Session lifecycle, flow control | + +### Workflow & Processes +| Skill | File | +|-------|------| +| Workflow Orchestration | `workflow.md` | +| Task Management | `task-management.md` | +| Amendment Creation | `amendment.md` | +| Merge Conflicts | `merge-conflicts.md` | diff --git a/.sentinel/skills/soul/consensus.md b/.sentinel/skills/soul/consensus.md new file mode 100644 index 00000000000..54e426cf49e --- /dev/null +++ b/.sentinel/skills/soul/consensus.md @@ -0,0 +1,86 @@ +# Consensus + +Template-based state machine in `Consensus.h` parameterized by an Adaptor (`RCLConsensus`). Three phases: open -> establish -> accepted. Modes: proposing, observing, wrongLedger, switchedLedger. + +## Key Invariants + +- A ledger cannot close until the previous ledger reaches consensus AND (has transactions OR close time reached) +- Proposals must have strictly increasing sequence numbers per peer; stale proposals are silently dropped +- The Avalanche state machine progressively lowers consensus thresholds over time (init -> mid -> late -> stuck) to prevent livelock +- `minCONSENSUS_PCT = 80` is the baseline; timing params: `ledgerMIN_CONSENSUS = 1950ms`, `ledgerMAX_CONSENSUS = 15s` +- Dead nodes (`deadNodes_`) are permanently excluded for the round once they bow out + +## Common Bug Patterns + +- Proposals referencing a stale `prevLedgerID_` after a ledger switch cause split-brain; always check `newPeerProp.prevLedger() != prevLedgerID_` before processing +- Resetting the consensus timer during `establish` phase causes re-convergence and potential split; timer must only reset on phase transitions +- `DisputedTx::updateVote` changes local vote based on peer pressure; bugs here cause determinism failures across nodes +- `createDisputes()` deduplicates via `compares` set; missing this check creates duplicate disputes that skew vote counts +- The `peerUnchangedCounter_` is reset to 0 when any vote changes; bugs in this counter cause premature consensus declaration + +## Amendments + +- 80% validator support for 2 weeks to enable; tracked via `AmendmentTable` with `amendmentMap_` +- New amendments: add to `features.macro` with `XRPL_FEATURE`/`XRPL_FIX`, increment `numFeatures` in `Feature.h` +- Unsupported enabled amendment blocks the server (`setAmendmentBlocked`); no mechanism to disable/revoke +- Voting happens each consensus round in `doVoting`; votes are persisted in `FeatureVotes` SQLite table +- `fixAmendmentMajorityCalc` changed the threshold calculation; check which calculation applies + +## UNL and Negative UNL + +- Negative UNL temporarily disables unreliable validators (max 25% of UNL: `negativeUNLMaxListed = 0.25`) +- Scoring uses `buildScoreTable` over recent ledger history; low watermark (50%) = disable candidate, high watermark (80%) = re-enable candidate +- Candidate selection is deterministic via previous ledger hash as randomizing pad +- `newValidatorDisableSkip = FLAG_LEDGER_INTERVAL * 2` prevents disabling newly joined validators prematurely + +## Validations + +- `ValidationParms` defines freshness windows: CURRENT_WALL=5min, CURRENT_LOCAL=3min, SET_EXPIRES=10min, FRESHNESS=20s +- `SeqEnforcer` rejects validations with regressed or duplicate sequence numbers (`ValStatus::badSeq`) +- Conflicting validations (same seq, different hash) are logged as byzantine behavior +- `handleNewValidation` is the entry point: checks trust, adds to `Validations` set, triggers `checkAccept` if current+trusted + +## Transaction Ordering + +- `CanonicalTXSet` orders by: salted account key (XOR with random salt) -> sequence proxy -> transaction ID +- Salt prevents manipulation of ordering by account selection +- `TxQ` uses `OrderCandidates`: higher fee level first, then `txID XOR parentHash` as tiebreaker +- Per-account limit: `maximumTxnPerAccount`; blocked transactions held until blocker resolves + +## Key Patterns + +### Proposal Validation (prevents split-brain) +```cpp +// REQUIRED: reject proposals referencing stale previous ledger +if (newPeerProp.prevLedger() != prevLedgerID_) +{ + JLOG(j_.debug()) << "Got proposal for " << newPeerProp.prevLedger() + << " but we are on " << prevLedgerID_; + return; +} +``` + +### Complete Bow-Out Handling +```cpp +// REQUIRED: all three steps — unvote, erase position, mark dead +if (newPeerProp.isBowOut()) +{ + if (result_) + for (auto& it : result_->disputes) + it.second.unVote(peerID); + if (currPeerPositions_.find(peerID) != currPeerPositions_.end()) + currPeerPositions_.erase(peerID); + deadNodes_.insert(peerID); // permanently excluded this round +} +``` + +## Key Files + +- `src/xrpld/consensus/Consensus.h` - state machine +- `src/xrpld/consensus/ConsensusParms.h` - timing/threshold params +- `src/xrpld/app/consensus/RCLConsensus.cpp` - XRPL adaptor +- `src/xrpld/consensus/DisputedTx.h` - dispute tracking +- `src/xrpld/app/misc/detail/AmendmentTable.cpp` - amendment logic +- `src/xrpld/app/misc/NegativeUNLVote.cpp` - N-UNL voting +- `src/xrpld/consensus/Validations.h` - validation tracking +- `src/xrpld/app/misc/CanonicalTXSet.h` - TX ordering diff --git a/.sentinel/skills/soul/cryptography.md b/.sentinel/skills/soul/cryptography.md new file mode 100644 index 00000000000..effda4a1b91 --- /dev/null +++ b/.sentinel/skills/soul/cryptography.md @@ -0,0 +1,59 @@ +# Cryptography + +XRPL supports secp256k1 (ECDSA) and ed25519 key types. All crypto uses OpenSSL + dedicated libs (libsecp256k1, ed25519-donna). + +## Key Invariants + +- `SecretKey` destructor calls `secure_erase` on internal buffer; any code handling secret keys must follow this pattern +- ed25519 public keys are prefixed with `0xED` (33 bytes total); secp256k1 keys are 33-byte compressed +- `sha512Half` (first 32 bytes of SHA-512) is the standard hash used throughout XRPL for node hashing, signing, etc. +- `RIPEMD-160(SHA-256(x))` is used for account ID derivation (`ripesha_hasher`) +- Base58 encoding includes a type byte prefix and 4-byte checksum (double SHA-256) + +## Common Bug Patterns + +- Mixing up key types: secp256k1 signing hashes the message with sha512Half first, ed25519 signs the raw message +- `signDigest` only works with secp256k1; calling it with ed25519 throws a logic error +- Signature canonicality: ed25519 `verify` checks signature canonicality before calling `ed25519_sign_open`; non-canonical signatures are rejected +- Overlay handshake uses `signDigest` to sign the session fingerprint (`sharedValue`); the signature binds the TLS session to the node identity + +## Review Checklist + +- New crypto code must use `crypto_prng()` singleton for randomness, never raw `rand()` +- Secret key buffers must be `secure_erase`d after use +- Verify that key type dispatch handles both secp256k1 and ed25519 (or explicitly rejects one with a clear error) + +## Key Patterns + +### Secure Erasure +```cpp +// REQUIRED: destructor must erase secret material +SecretKey::~SecretKey() +{ + secure_erase(buf_, sizeof(buf_)); +} + +// REQUIRED: erase intermediate buffers after use +beast::rngfill(buf, sizeof(buf), crypto_prng()); +SecretKey sk(Slice{buf, sizeof(buf)}); +secure_erase(buf, sizeof(buf)); // MUST erase raw buffer +``` + +### Key Type Dispatch +```cpp +// REQUIRED: handle both key types or explicitly reject +if (type == KeyType::ed25519) +{ /* ed25519 path */ } +else if (type == KeyType::secp256k1) +{ /* secp256k1 path */ } +else + LogicError("unknown key type"); // MUST NOT fall through silently +``` + +## Key Files + +- `include/xrpl/protocol/SecretKey.h` / `PublicKey.h` - key types +- `src/libxrpl/protocol/SecretKey.cpp` - signing, key generation +- `src/libxrpl/protocol/PublicKey.cpp` - verification +- `include/xrpl/protocol/digest.h` - hash functions +- `src/xrpld/overlay/detail/Handshake.cpp` - overlay handshake crypto diff --git a/.sentinel/skills/soul/ledger.md b/.sentinel/skills/soul/ledger.md new file mode 100644 index 00000000000..ce561c57cd1 --- /dev/null +++ b/.sentinel/skills/soul/ledger.md @@ -0,0 +1,63 @@ +# Ledger + +Each ledger is an immutable snapshot: header (seq, hashes, close time) + state SHAMap + transaction SHAMap. `LedgerMaster` is the central coordinator. + +## Key Invariants + +- Once `setImmutable()` is called, the ledger and its SHAMaps cannot change; only immutable ledgers can be set in `LedgerHolder` +- Every server always has an open ledger; the open ledger cannot close until previous consensus completes +- Ledger header hashes to the ledger's identity hash; includes state root, tx root, parent hash, total coins, close time +- `LedgerMaster` tracks: `mPubLedger` (last published), `mValidLedger` (last validated), `mLedgerHistory` (cache) +- Validation requires minimum trusted validations (`minVal`); filtered by Negative UNL + +## Common Bug Patterns + +- Modifying a ledger after `setImmutable()` corrupts shared state; always check `mImmutable` before mutation +- Gap detection: if ledgers 603 and 600 exist but 601-602 are missing, `LedgerMaster` requests 602 first, then backfills 601 +- `InboundLedger::gotData()` queues data for processing; calling `done()` before all data arrives creates incomplete ledgers +- `checkAccept` won't accept a ledger that isn't ahead of the last validated ledger; stale validations are silently ignored + +## Ledger Entry Types + +- Defined in `ledger_entries.macro` using `LEDGER_ENTRY(type, code, class, name, fields)` +- Each entry has an `SOTemplate` defining required/optional fields +- Key computation: `Indexes.cpp` computes unique keys (keylets) for each ledger object type +- `STLedgerEntry` wraps the serialized data with type-safe field access + +## Review Checklist + +- New ledger entry types: add to `ledger_entries.macro`, implement keylet in `Indexes.cpp` +- Verify `LedgerCleaner` can handle the new entry type for repair +- Check that acquisition code handles the entry in both `InboundLedger` and `LedgerMaster` + +## Key Patterns + +### Immutability Guard +```cpp +// After this, no mutations allowed on the ledger or its SHAMaps +inline void SHAMap::setImmutable() +{ + XRPL_ASSERT(state_ != SHAMapState::Invalid, "..."); + state_ = SHAMapState::Immutable; +} +// VERIFY: code never calls peek()/insert()/erase() after setImmutable() +``` + +### New Ledger Entry Keylet +```cpp +// REQUIRED: every new ledger entry type needs unique keylet computation +Keylet keylet::myEntry(AccountID const& id) +{ + return {ltMY_ENTRY, + sha512Half(std::uint16_t(spaceMyEntry), id)}; +} +// Also add to ledger_entries.macro and Indexes.cpp +``` + +## Key Files + +- `src/xrpld/app/ledger/Ledger.h` - ledger class +- `src/xrpld/app/ledger/detail/LedgerMaster.cpp` - central coordinator +- `src/xrpld/app/ledger/detail/InboundLedger.cpp` - ledger acquisition +- `include/xrpl/protocol/detail/ledger_entries.macro` - entry type definitions +- `src/libxrpl/protocol/Indexes.cpp` - keylet computation diff --git a/.sentinel/skills/soul/nodestore.md b/.sentinel/skills/soul/nodestore.md new file mode 100644 index 00000000000..26747199818 --- /dev/null +++ b/.sentinel/skills/soul/nodestore.md @@ -0,0 +1,52 @@ +# NodeStore + +Persistent key-value store for `NodeObject`s (ledger entries). All ledger state is stored here between launches. Keys are 256-bit hashes. + +## Key Invariants + +- `NodeObject` types: `hotLEDGER` (1), `hotACCOUNT_NODE` (3), `hotTRANSACTION_NODE` (4), `hotDUMMY` (512, cache marker for missing entries) +- Preferred backends: NuDB (append-only) and RocksDB; LevelDB/HyperLevelDB are deprecated +- `TaggedCache` evicts by both `cache_size` (max items) and `cache_age` (max minutes) +- `DatabaseRotatingImp` uses two backends (writable + archive) for online deletion; rotation moves writable to archive, creates new writable, deletes old archive +- Corrupt data triggers fatal logging; unknown/backend errors logged with appropriate severity + +## Common Bug Patterns + +- `fetchNodeObject` with `duplicate=true` copies from archive to writable backend; forgetting this in rotating mode means objects disappear after rotation +- `hotDUMMY` objects in cache mark missing entries; code that checks cache hits must distinguish real objects from dummies +- Batch write limit is 65536 objects; exceeding this silently truncates or fails depending on backend +- `fdRequired()` must be called during resource planning; running out of file descriptors causes silent backend failures + +## Review Checklist + +- Config changes: verify `[node_db]` section has valid `type`, `path`, and `compression` settings +- Online deletion: ensure `SHAMapStoreImp` coordinates rotation with the application lifecycle +- New backend types: implement the full `Backend` interface including `fdRequired()` + +## Key Patterns + +### Cache Lookup — Distinguish Real vs Dummy +```cpp +// REQUIRED: hotDUMMY marks "confirmed missing" — not a real object +auto obj = cache_.fetch(hash); +if (obj && obj->getType() == hotDUMMY) + return nullptr; // not found, just cached as missing +return obj; +``` + +### Backend File Descriptor Reporting +```cpp +// REQUIRED: every backend must accurately report FD needs +int fdRequired() const override +{ + return fdLimit_; // inaccurate values cause silent failures +} +``` + +## Key Files + +- `include/xrpl/nodestore/NodeObject.h` - object types +- `include/xrpl/nodestore/Backend.h` - backend interface +- `include/xrpl/nodestore/detail/DatabaseNodeImp.h` - standard implementation +- `src/libxrpl/nodestore/DatabaseRotatingImp.cpp` - rotating/online deletion +- `src/xrpld/app/misc/SHAMapStoreImp.cpp` - lifecycle management diff --git a/.sentinel/skills/soul/peering.md b/.sentinel/skills/soul/peering.md new file mode 100644 index 00000000000..8f7effafcd4 --- /dev/null +++ b/.sentinel/skills/soul/peering.md @@ -0,0 +1,60 @@ +# Overlay Peering + +P2P network using persistent TCP/IP connections. Messages serialized via Protocol Buffers. `OverlayImpl` manages connections; `PeerImp` handles per-peer logic. + +## Key Invariants + +- Connection preference order: Fixed Peers -> Livecache -> Bootcache +- Cluster connections do NOT count toward connection limits (unlimited) +- Protobuf message changes MUST maintain wire compatibility or risk network partitioning +- Squelching: after enough peers relay a validator's messages, a subset is "Selected" and the rest are temporarily muted to reduce bandwidth +- Handshake binds TLS session to node identity via `signDigest` of the session fingerprint + +## Common Bug Patterns + +- PeerFinder slot exhaustion: if `maxPeers` is reached, new outbound connections silently fail; check slot availability before connecting +- `HashRouter::shouldRelay` prevents duplicate relay; bypassing it causes message storms +- `ConnectAttempt::processResponse` on HTTP 503 parses "peer-ips" for alternatives; malformed responses here can crash with bad IP parsing +- `PeerImp::close` must run on the strand; calling from wrong thread causes race conditions on socket and timer state +- Destructor chain: `~PeerImp` -> `deletePeer` -> `onPeerDeactivate` -> `on_closed` -> `remove`; interrupting this chain leaks slots + +## Connection Lifecycle + +1. `OverlayImpl::connect` -> check resource limits -> allocate PeerFinder slot -> create `ConnectAttempt` +2. Async TCP connect -> TLS handshake -> HTTP upgrade with identity headers +3. `processResponse` -> verify handshake -> create `PeerImp` -> `add_active` -> `run()` +4. `doProtocolStart` -> start async message receive loop -> exchange validator lists and manifests + +## Review Checklist + +- Verify resource manager checks on both inbound and outbound connections +- New protocol messages: update protobuf definitions AND verify wire compatibility +- Squelch changes: test with high peer counts; incorrect squelch logic can silence validators + +## Key Patterns + +### Strand Execution +```cpp +// REQUIRED: socket operations must run on the strand +if (!strand_.running_in_this_thread()) + return post(strand_, std::bind( + &PeerImp::close, shared_from_this())); +// Calling socket ops from wrong thread causes races on state +``` + +### Duplicate Relay Prevention +```cpp +// REQUIRED: check HashRouter before relaying +if (!hashRouter_.shouldRelay(hash)) + return; // already relayed — suppress duplicate +overlay_.relay(message, hash); +// Bypassing this causes message storms across the network +``` + +## Key Files + +- `src/xrpld/overlay/detail/OverlayImpl.cpp` - main overlay manager +- `src/xrpld/overlay/detail/PeerImp.cpp` - per-peer logic +- `src/xrpld/overlay/detail/ConnectAttempt.cpp` - outbound connection +- `src/xrpld/overlay/Slot.h` - squelch state machine +- `src/xrpld/overlay/detail/Handshake.cpp` - handshake crypto diff --git a/.sentinel/skills/soul/protocol.md b/.sentinel/skills/soul/protocol.md new file mode 100644 index 00000000000..ff41015ffc0 --- /dev/null +++ b/.sentinel/skills/soul/protocol.md @@ -0,0 +1,64 @@ +# Protocol and Serialization + +Macro-driven system for defining features, transactions, ledger entries, and serialized fields. Canonical binary format is required for signatures and consensus. + +## Key Invariants + +- Fields are sorted by (type code, field code) for canonical serialization; sorting by Field ID bytes produces WRONG results +- Field ID encoding: 1-3 bytes depending on type/field code values (both < 16 = 1 byte) +- Signing hash prefix: `0x53545800` for single-signing, `0x534D5400` for multi-signing +- `STObject[sfFoo]` returns value or default; `STObject[~sfFoo]` returns optional (nothing if absent) +- All ST types inherit from `STBase`; `STVar` provides type-erased storage with stack/heap allocation + +## Macro System + +- `XRPL_FEATURE(name, supported, vote)` / `XRPL_FIX` / `XRPL_RETIRE` in `features.macro` +- `TRANSACTION(tag, value, class, delegation, fields)` in `transactions.macro` +- `LEDGER_ENTRY(type, code, class, name, fields)` in `ledger_entries.macro` +- `TYPED_SFIELD(name, TYPE, code)` in `sfields.macro` +- Adding any new definition requires updating the count in the corresponding header + +## Serialization Format + +- XRP Amount: 8 bytes, MSB=0, next bit=1 for positive, remaining 62 bits = value +- Token Amount: 8 bytes mantissa/exponent + 20 bytes currency + 20 bytes issuer +- AccountID: 20 bytes, length-prefixed when top-level +- STArray: elements between start (`0xf0`) and end (`0xf1`) markers +- STObject: fields in canonical order between start (`0xe0`) and end (`0xe1`) markers +- Length prefixing: 1 byte (0-192), 2 bytes (193-12480), 3 bytes (12481-918744) + +## Common Bug Patterns + +- Adding a field to `transactions.macro` without adding it to `sfields.macro` causes silent serialization failures +- Forgetting to increment `numFeatures` after adding to `features.macro` causes out-of-bounds access +- Non-canonical field ordering in hand-built binary blobs causes signature verification failures +- `soeMPTSupported` flag on amount fields enables MPT token support; omitting it silently rejects MPT payments + +## Key Patterns + +### Amendment Registration +```cpp +// In features.macro — REQUIRED format: +XRPL_FEATURE(MyNewFeature, Supported::yes, VoteBehavior::DefaultNo) +XRPL_FIX (MyBugFix, Supported::yes, VoteBehavior::DefaultNo) +// MUST also increment numFeatures in Feature.h — omitting causes OOB access +``` + +### STObject Field Access +```cpp +// Safe: optional access — returns std::optional, never throws +if (auto const val = tx[~sfAmount]) + use(*val); + +// Throws if field is absent — only safe when preflight guarantees presence +auto const amount = tx[sfAmount]; +``` + +## Key Files + +- `include/xrpl/protocol/detail/features.macro` - amendment definitions +- `include/xrpl/protocol/detail/transactions.macro` - transaction types +- `include/xrpl/protocol/detail/ledger_entries.macro` - ledger objects +- `include/xrpl/protocol/detail/sfields.macro` - field definitions +- `include/xrpl/protocol/Feature.h` - `numFeatures` constant +- `src/libxrpl/protocol/STObject.cpp` - core serialized object diff --git a/.sentinel/skills/soul/rpc.md b/.sentinel/skills/soul/rpc.md new file mode 100644 index 00000000000..665863deb53 --- /dev/null +++ b/.sentinel/skills/soul/rpc.md @@ -0,0 +1,61 @@ +# RPC + +JSON-RPC over HTTP/WebSocket and gRPC. Central handler table dispatches by method name + API version. Roles: ADMIN, USER, IDENTIFIED, PROXY, FORBID. + +## Key Invariants + +- Handler table in `Handler.cpp`: each entry = `{name, function, role, condition, minApiVer, maxApiVer}` +- `conditionMet` checks server state (e.g., `NEEDS_CURRENT_LEDGER`) before invoking handler +- API v2.0+ errors: structured objects with `status`, `code`, `message`; earlier: flat fields in response +- Sensitive fields (`passphrase`, `secret`, `seed`, `seed_hex`) are masked in error responses +- Batch requests: `"method": "batch"` with `"params"` array; each sub-request processed independently + +## Common Bug Patterns + +- New handler without entry in `Handler.cpp` static array = handler silently unreachable +- Wrong `role_` on handler: USER-level handler with admin-only data leaks; ADMIN handler accessible to users = security hole +- `conditionMet` returning false causes a generic error; ensure new conditions are documented +- Resource charging: each request gets a fee via `Resource::Consumer`; missing charge allows DoS +- `maxRequestSize` (RPC::Tuning) rejection happens before JSON parsing; oversized requests get no error detail + +## Adding New RPC Handler + +1. Declare in `Handlers.h`: `Json::Value doMyCommand(RPC::JsonContext&);` +2. Implement in new file under `src/xrpld/rpc/handlers/` +3. Register in `Handler.cpp` static array with role, condition, version range +4. For gRPC: define in `xrp_ledger.proto`, add `CallData` in `GRPCServerImpl::setupListeners()` + +## Subscriptions + +- WebSocket clients can subscribe to: `server`, `ledger`, `book_changes`, `transactions`, `validations`, `manifests`, `peer_status` (admin), `consensus` +- `WSInfoSub` delivers events via weak pointer to `WSSession`; dead sessions are automatically cleaned up +- `RPCSub` delivers to remote URL endpoints with auth and SSL support + +## Key Patterns + +### Handler Table Registration +```cpp +// In Handler.cpp handlerArray[] — REQUIRED for every new handler: +{"my_command", byRef(&doMyCommand), Role::USER, NO_CONDITION}, +// role MUST match security requirements: +// Role::ADMIN for internal-only, Role::USER for public API +// condition: NEEDS_CURRENT_LEDGER, NEEDS_NETWORK_CONNECTION, or NO_CONDITION +``` + +### Version-Ranged Handler +```cpp +// New-style handler with API version range +template <> Handler handlerFrom() +{ return {MyCommandHandler::name, &handle, + MyCommandHandler::role, MyCommandHandler::condition, + MyCommandHandler::minApiVer, MyCommandHandler::maxApiVer}; +} +``` + +## Key Files + +- `src/xrpld/rpc/handlers/Handlers.h` - authoritative handler list +- `src/xrpld/rpc/detail/Handler.cpp` - handler table and dispatch +- `src/xrpld/rpc/detail/RPCHandler.cpp` - request processing pipeline +- `src/xrpld/rpc/detail/ServerHandler.cpp` - HTTP/WS entry points +- `include/xrpl/protocol/ErrorCodes.h` - error code definitions diff --git a/.sentinel/skills/soul/shamap.md b/.sentinel/skills/soul/shamap.md new file mode 100644 index 00000000000..e8baa8060ba --- /dev/null +++ b/.sentinel/skills/soul/shamap.md @@ -0,0 +1,61 @@ +# SHAMap + +Merkle radix trie (radix 16) enabling O(1) subtree comparison via hash. Used for both state tree and transaction tree. Root is always a `SHAMapInnerNode`. + +## Key Invariants + +- Mutable SHAMaps have non-zero `cowid`; immutable have `cowid=0`. Once immutable, nodes persist for the map's lifetime with NO mechanism to remove them +- Copy-on-write: `unshareNode` must be called before mutating any node in a mutable SHAMap; failing this corrupts shared snapshots +- Inner nodes have up to 16 children; hash is computed from children's hashes. Leaf hash is computed from data + type-specific prefix +- `canonicalize` ensures only one instance per hash in the cache; prevents races between threads +- `SHAMapInnerNode` uses atomic operations + locking (`std::atomic lock_`) for concurrent child access + +## Common Bug Patterns + +- Modifying a node without calling `unshareNode` first corrupts the snapshot that shares it; this is the #1 SHAMap bug class +- `getMissingNodes` uses deferred async reads; processing completions out of order causes incorrect "full below" marking +- Inner node serialization has two formats (compressed vs full) chosen by branch count; mismatched deserializer causes corruption +- `addKnownNode` traverses toward target; if branch is empty or hash mismatches, returns "invalid" -- callers must handle this gracefully +- Proof path verification walks leaf-to-root; incorrect key at any level causes false negative + +## Serialization Formats + +- **Compressed**: only non-empty branches serialized (saves space for sparse nodes) +- **Full**: all 16 branches including empty ones (used for dense nodes) +- Choice is automatic in `serializeForWire` based on branch count + +## Leaf Node Types + +- `SHAMapAccountStateLeafNode` - account state entries +- `SHAMapTxLeafNode` - transactions +- `SHAMapTxPlusMetaLeafNode` - transactions with metadata +- Each uses a different hash prefix for domain separation + +## Key Patterns + +### State Machine +```cpp +enum class SHAMapState { + Modifying = 0, // can add/remove objects + Immutable = 1, // FROZEN — no changes allowed + Synching = 2, // hash fixed, missing nodes can be added + Invalid = 3, // corrupt — do not use +}; +// VERIFY: no peek()/insert()/erase() calls on Immutable maps +``` + +### COW Discipline (#1 Bug Class) +```cpp +// REQUIRED before mutating any shared node: +auto node = unshareNode(branch, key); // copies if shared +node->setChild(index, child); // now safe to modify +// BUG: skipping unshareNode corrupts snapshots sharing the node +``` + +## Key Files + +- `include/xrpl/shamap/SHAMap.h` - main class +- `include/xrpl/shamap/SHAMapInnerNode.h` - inner node (COW, threading) +- `include/xrpl/shamap/SHAMapLeafNode.h` - leaf node base +- `src/libxrpl/shamap/SHAMapSync.cpp` - sync, missing nodes, proofs +- `src/libxrpl/shamap/SHAMapDelta.cpp` - walkMap, parallel traversal diff --git a/.sentinel/skills/soul/sql.md b/.sentinel/skills/soul/sql.md new file mode 100644 index 00000000000..30bd2d713e9 --- /dev/null +++ b/.sentinel/skills/soul/sql.md @@ -0,0 +1,60 @@ +# SQL Database + +SQLite via SOCI for ledger/transaction history. Only SQLite is supported; Postgres has no implementation despite interface comments. + +## Key Invariants + +- Two main databases: `lgrdb_` (ledger) and `txdb_` (transactions, optional via `useTxTables` config) +- Transaction tables are optional; disabling them means no transaction history or account_tx queries +- WAL checkpointing triggers when WAL file grows beyond threshold; scheduled via job queue +- Database init failure is fatal (throws exception, prevents construction) +- Free disk space < 512MB triggers fatal error on write operations + +## Schema + +- `Ledgers` table: seq, hash, parent hash, total coins, close time, etc. Indexed by `LedgerSeq` +- `Transactions` table: TransID, TransType, FromAcct, FromSeq, LedgerSeq, Status, RawTxn, TxnMeta. Indexed by `LedgerSeq` +- `AccountTransactions` table: TransID, Account, LedgerSeq, TxnSeq. Triple-indexed for account_tx queries +- Secondary DBs: Wallet (node identity, manifests), PeerFinder (bootstrap cache), State (deletion tracking) + +## Common Bug Patterns + +- No schema migration system; `CREATE TABLE IF NOT EXISTS` means old schemas silently persist with missing columns +- PeerFinder DB is the exception -- it has schema versioning via `SchemaVersion` table +- `safety_level` config affects journal_mode and synchronous; "low" can lose data on crash +- `page_size` must be power of 2 between 512-65536; invalid values cause init failure +- Online deletion coordinates between NodeStore rotation and SQL table pruning; race conditions here lose history + +## Configuration + +| Option | Section | Values | Default | +|--------|---------|--------|---------| +| `backend` | `[relational_db]` | `sqlite` only | sqlite | +| `page_size` | `[sqlite]` | 512-65536, power of 2 | 4096 | +| `safety_level` | `[sqlite]` | high, medium, low | high | +| `journal_size_limit` | `[sqlite]` | integer >= 0 | 1582080 | + +## Key Patterns + +### Schema Evolution Caveat +```cpp +// WARNING: no migration system — old databases keep old schemas +// CREATE TABLE IF NOT EXISTS silently skips if table exists with old columns +// New columns on existing tables require manual ALTER TABLE or +// documentation that the column is optional and may be absent +``` + +### Disk Space Guard +```cpp +// REQUIRED on write paths: < 512MB triggers fatal to prevent corruption +if (freeDiskSpace < minDiskFree) + Throw("Not enough disk space for database write"); +``` + +## Key Files + +- `src/xrpld/app/rdb/backend/detail/SQLiteDatabase.cpp` - main implementation +- `src/xrpld/app/main/DBInit.h` - schema definitions +- `src/xrpld/core/detail/DatabaseCon.cpp` - connection setup and pragmas +- `src/xrpld/app/rdb/backend/detail/Node.cpp` - ledger/tx operations +- `src/xrpld/app/rdb/detail/State.cpp` - deletion state tracking diff --git a/.sentinel/skills/soul/test.md b/.sentinel/skills/soul/test.md new file mode 100644 index 00000000000..5f6de010966 --- /dev/null +++ b/.sentinel/skills/soul/test.md @@ -0,0 +1,75 @@ +# Testing + +JTx framework for in-memory ledger testing. Tests live in `src/test/`, derive from `beast::unit_test::suite`, and register with `BEAST_DEFINE_TESTSUITE`. + +## Key Patterns + +### Test File Structure +```cpp +class MyFeature_test : public beast::unit_test::suite { + void testBasic() { + testcase("basic"); + using namespace jtx; + Env env{*this}; + // ... test logic ... + } + void run() override { testBasic(); } +}; +BEAST_DEFINE_TESTSUITE(MyFeature, app, ripple); +``` + +### Amendment Gating +```cpp +// REQUIRED: test with AND without the feature amendment +Env env{*this}; // all amendments on +Env env{*this, testable_amendments() - featureMyFeature}; // feature disabled +// With custom config: +Env env{*this, envconfig([](std::unique_ptr cfg) { + /* modify config */ return cfg; +})}; +``` + +### Transaction Submission +```cpp +auto const alice = Account{"alice"}; +auto const bob = Account{"bob"}; +env.fund(XRP(10000), alice, bob); +env.close(); // REQUIRED between setup and test transactions + +env(pay(alice, bob, XRP(100))); // expects tesSUCCESS +env(pay(alice, bob, XRP(999999)), ter(tecUNFUNDED_PAYMENT)); // specific TER +env(pay(alice, bob, XRP(100)), txflags(tfPartialPayment)); // with flags +``` + +### State Verification +```cpp +env.require(balance(alice, XRP(9900))); +env.require(balance(alice, USD(1000))); +env.require(owners(alice, 2)); + +auto const sle = env.le(keylet::account(alice.id())); +BEAST_EXPECT(sle); +BEAST_EXPECT((*sle)[sfBalance] == XRP(9900)); +``` + +## Review Checklist + +- New transaction types MUST have tests with AND without the feature amendment +- Every error path (tem*, tec*, tef*) must have a test exercising it +- `env.close()` required between transactions that need ledger boundaries +- Verify state after transaction, not just TER code +- Test class naming: `FeatureName_test`, registered with `BEAST_DEFINE_TESTSUITE` + +## Common Pitfalls + +- Forgetting `*this` as first Env arg disconnects assertion reporting +- Comparing XRPAmount with raw integers instead of `XRP()` or `drops()` +- Trust lines must be established before IOU payments +- Each Env creates a full Application — keep test methods focused + +## Key Files + +- `src/test/jtx/Env.h` — test environment class +- `src/test/jtx/jtx.h` — convenience header (includes all helpers) +- `src/test/jtx/Account.h` — test accounts +- `src/test/jtx/amount.h` — XRP(), drops(), IOU helpers diff --git a/.sentinel/skills/soul/transactors.md b/.sentinel/skills/soul/transactors.md new file mode 100644 index 00000000000..dc0e5423dd3 --- /dev/null +++ b/.sentinel/skills/soul/transactors.md @@ -0,0 +1,102 @@ +# Transactors + +Transaction processing pipeline: preflight (static validation) -> preclaim (ledger state checks) -> doApply (state mutation). Base class `Transactor` in `src/libxrpl/tx/`. + +## Key Invariants + +- Pipeline is strict: preflight runs WITHOUT ledger state, preclaim runs WITH read-only view, doApply runs with mutable view +- `preflight` validates all fields exist and are well-formed; this is the ONLY place to reject malformed transactions cheaply +- Fee is always deducted even if the transaction fails (`tecCLAIM` pattern); `payFee` runs before `doApply` +- Sequence/ticket consumption happens in `consumeSeqProxy`; must succeed before any state changes +- Invariant checkers run after `doApply`; they can veto the transaction post-execution + +## Common Bug Patterns + +- New transaction type missing preflight validation for new fields = malformed transactions reach doApply and corrupt state +- Forgetting to handle `tecCLAIM` in doApply: fee is deducted but no other state changes should occur +- Batch transactions (`Batch` type) have their own signing path (`checkBatchSign`); changes to signing must cover both paths +- `calculateBaseFee` override without updating `minimumFee` causes fee calculation divergence between nodes +- Missing invariant checker update for new ledger entry types = silent constraint violations + +## Transactor Template + +### Header (`include/xrpl/tx/transactors/MyTx.h`) +```cpp +#pragma once +#include + +namespace xrpl { +class MyTransaction : public Transactor { +public: + static constexpr ConsequencesFactoryType ConsequencesFactory{Normal}; + explicit MyTransaction(ApplyContext& ctx) : Transactor(ctx) {} + + static bool checkExtraFeatures(PreflightContext const& ctx); + static std::uint32_t getFlagsMask(PreflightContext const& ctx); + static NotTEC preflight(PreflightContext const& ctx); // NO ledger + static TER preclaim(PreclaimContext const& ctx); // read-only + TER doApply() override; // read-write +}; +} +``` + +### Implementation (`src/libxrpl/tx/transactors/MyFeature/MyTx.cpp`) +```cpp +bool MyTransaction::checkExtraFeatures(PreflightContext const& ctx) +{ // REQUIRED: gate on amendment + return ctx.rules.enabled(featureMyFeature); +} + +NotTEC MyTransaction::preflight(PreflightContext const& ctx) +{ // Static validation — NO ctx.view, NO ledger access + if (ctx.tx[sfAmount] <= beast::zero) + return temBAD_AMOUNT; + return tesSUCCESS; +} + +TER MyTransaction::preclaim(PreclaimContext const& ctx) +{ // Read-only — ctx.view.read() only, NO peek/insert/erase + if (!ctx.view.exists(keylet::account(ctx.tx[sfAccount]))) + return terNO_ACCOUNT; + return tesSUCCESS; +} + +TER MyTransaction::doApply() +{ // Mutable — view().peek(), view().insert(), view().update(), view().erase() + auto sle = view().peek(keylet::account(account_)); + sle->setFieldAmount(sfBalance, newBal); + view().update(sle); // REQUIRED after mutation + return tesSUCCESS; +} +``` + +### Registration Checklist +```cpp +// ALL of these are REQUIRED for a new transaction type: +// 1. transactions.macro: TRANSACTION(ttMY_TYPE, N, MyTx, delegation, fields) +// 2. applySteps.cpp: case ttMY_TYPE: return invoke(...); +// 3. features.macro: XRPL_FEATURE(MyFeature, Supported::yes, DefaultNo) +// 4. Feature.h: increment numFeatures +// 5. InvariantCheck.cpp: update if new ledger objects created +// 6. Batch.cpp: add to disabledTxTypes if not batch-compatible +``` + +## Transaction Lifecycle + +1. `preflight` (static checks, no ledger) -> `PreflightResult` +2. `preclaim` (ledger state, read-only) -> TER +3. `operator()` orchestrates: `checkSeqProxy` -> `checkPriorTxAndLastLedger` -> `checkFee` -> `checkSign` -> `apply` +4. `apply` -> `reset` (deduct fee) -> `doApply` (state changes) -> invariant checks -> metadata generation + +## Permission System + +- `checkSign` dispatches to `checkSingleSign`, `checkMultiSign`, or `checkBatchSign` +- `checkPermission` validates delegated authority for delegatable transaction types +- Multi-sign requires M-of-N signers matching the signer list; weight threshold must be met + +## Key Files + +- `src/xrpld/app/tx/detail/Transactor.cpp` - base class and pipeline +- `include/xrpl/protocol/detail/transactions.macro` - type definitions +- `src/xrpld/app/tx/detail/` - per-type implementations (Payment.cpp, OfferCreate.cpp, etc.) +- `src/xrpld/app/tx/detail/InvariantCheck.cpp` - post-execution invariant checks diff --git a/.sentinel/skills/soul/websockets.md b/.sentinel/skills/soul/websockets.md new file mode 100644 index 00000000000..12023dd9ab3 --- /dev/null +++ b/.sentinel/skills/soul/websockets.md @@ -0,0 +1,62 @@ +# WebSockets + +Async WebSocket support for client RPC and real-time subscriptions. Both plain and SSL. Built on Boost.Beast + Boost.Asio. + +## Key Invariants + +- All async operations run on a per-session Boost.Asio strand for thread safety +- Outgoing message queue (`wq_`) has a per-session limit (`port().ws_queue_limit`); exceeding it closes the connection with policy error "client is too slow" +- `complete()` must be called after processing each message to resume reading; forgetting it stalls the session +- `WSInfoSub` holds a weak pointer to `WSSession`; dead sessions are automatically skipped during event delivery +- Message size limit: `RPC::Tuning::maxRequestSize`; oversized messages get a `jsonInvalid` error response + +## Connection Flow + +1. `Door` accepts TCP -> `Detector` probes for SSL vs plain -> creates `SSLHTTPPeer` or `PlainHTTPPeer` +2. HTTP request with WebSocket upgrade -> `ServerHandler::onHandoff` -> `session.websocketUpgrade()` -> creates `PlainWSPeer` or `SSLWSPeer` +3. `BaseWSPeer::run()` -> set permessage-deflate options -> `async_accept` handshake -> `on_ws_handshake` -> `do_read` loop +4. `on_read` -> `ServerHandler::onWSMessage` -> validate JSON -> post to job queue -> `processSession` -> send response -> `complete()` + +## Common Bug Patterns + +- `on_read` consumes the buffer AFTER calling `onWSMessage`; if the handler accesses the buffer asynchronously after `onWSMessage` returns, it reads garbage +- `close()` with pending messages defers the actual close; calling `send()` after `close()` but before actual close queues more messages +- Missing `complete()` call after sending response = session never reads again = appears hung +- Job queue shutdown: if `postCoro` returns nullptr, session must close with `going_away`; dropping this silently leaks sessions + +## Review Checklist + +- Verify strand execution for all socket operations (read, write, close, timer) +- New subscription streams: ensure `WSInfoSub::send` handles the new event type +- Flow control: test with slow clients to verify queue limit enforcement + +## Key Patterns + +### complete() Resumes Read Loop +```cpp +// REQUIRED after sending response — missing this = session hangs forever +void BaseWSPeer::complete() +{ + if (!strand_.running_in_this_thread()) + return post(strand_, std::bind( + &BaseWSPeer::complete, impl().shared_from_this())); + do_read(); // resume reading next message +} +``` + +### Queue Limit Enforcement +```cpp +// REQUIRED: close slow clients to prevent memory exhaustion +if (wq_.size() >= port().ws_queue_limit) +{ + close(boost::beast::websocket::close_code::policy_error); + return; // do NOT queue unboundedly +} +``` + +## Key Files + +- `include/xrpl/server/detail/BaseWSPeer.h` - session lifecycle and message queue +- `include/xrpl/server/detail/Door.h` - connection acceptance +- `src/xrpld/rpc/detail/ServerHandler.cpp` - `onWSMessage` and `onHandoff` +- `src/xrpld/rpc/detail/WSInfoSub.h` - subscription delivery