Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
ea1b0c4
feat: add Rust crate import support via rustdoc JSON + FFM
kdroidFilter Mar 31, 2026
977052e
feat: Phase 1 — &str/String distinction, &self/&mut self, property ex…
kdroidFilter Mar 31, 2026
8d95fdf
feat: Phase 2 — slice param support (&[u8], &[i32]) in Rust bridges
kdroidFilter Mar 31, 2026
a6df277
fix: FfmProxyGenerator handles LIST/BYTE_ARRAY input params for top-l…
kdroidFilter Mar 31, 2026
35a6350
fix: align Rust calculator UI with Kotlin/Native calculator
kdroidFilter Mar 31, 2026
a2950fd
feat: Phase 3 — data classes + build.rs bridge inclusion
kdroidFilter Mar 31, 2026
92808bb
feat: add Rust benchmark example mirroring Kotlin/Native benchmark
kdroidFilter Mar 31, 2026
38c0330
docs: update README with Rust import support, benchmarks, and archite…
kdroidFilter Mar 31, 2026
f2c4f42
feat: Phase 4 — nullable input params (Option<T>)
kdroidFilter Mar 31, 2026
f9a743c
feat: add rust-sysinfo example using sysinfo crate via FFM
kdroidFilter Mar 31, 2026
cdda218
test: add 85 end-to-end FFM tests for Rust calculator
kdroidFilter Mar 31, 2026
f7839cd
feat: suspend function support for Rust via @kne:suspend annotation
kdroidFilter Mar 31, 2026
a88b794
feat: Flow<T> support for Rust via @kne:flow(Type) annotation
kdroidFilter Mar 31, 2026
b148dfd
feat: Phase 5 — Traits → Interfaces (Describable, Resettable, Measura…
kdroidFilter Mar 31, 2026
91d828e
feat: Phase 6 — tagged enums → Kotlin sealed classes
kdroidFilter Mar 31, 2026
7d4816f
feat: Phase 7 — bidirectional callbacks (fn() params from Rust)
kdroidFilter Mar 31, 2026
96896f9
Phase 8 — Rust sysinfo example with Flow-based Compose UI
kdroidFilter Apr 1, 2026
2f4040e
Added MAP return type support with key/value serialization in Rust br…
kdroidFilter Apr 1, 2026
20341c6
Add SET and nullable DataClass return type support to Rust bridge gen…
kdroidFilter Apr 1, 2026
3b3a67f
test: expand RustBridgeGeneratorTest from 34 to 83 tests
kdroidFilter Apr 1, 2026
6c443fc
Extract cargo resolution and add nullable DataClass parameter support
kdroidFilter Apr 1, 2026
2bd4313
Fix rust-calculator: add division, error handling, and decimal result…
kdroidFilter Apr 1, 2026
9efe876
fix(rustdoc-parser): exclude non-re-exported types from bridge genera…
kdroidFilter Apr 1, 2026
4e9136e
feat(sealed-variants): add SET and MAP collection field support
kdroidFilter Apr 1, 2026
5823a02
feat(rust-sysinfo): enrich UI with sensors, disk I/O, users, and proc…
kdroidFilter Apr 1, 2026
7d31277
feat(rust-bridge): add nullable collection return type support
kdroidFilter Apr 1, 2026
865c845
feat(rust-sysinfo): add process threads, cpu time, file descriptors, …
kdroidFilter Apr 1, 2026
f477fe0
fix(native-cache): check content equality in cache invalidation logic
kdroidFilter Apr 1, 2026
b489ec0
refactor(rust-bridge): improve memory management and serialization logic
kdroidFilter Apr 1, 2026
24a2842
feat(rust): add support for Rust's Never type (!)
kdroidFilter Apr 2, 2026
72d9dd2
fix(ffm): correct MAP property getter to use StableRef pattern
kdroidFilter Apr 2, 2026
daab6e0
docs: update documentation for Never type and MAP property support
kdroidFilter Apr 3, 2026
65327d4
test(rust): add tests for Option<Vec<T>> and Option<HashMap> return t…
kdroidFilter Apr 3, 2026
7195c84
feat(tuple): add Rust tuple type support (arity 0-16) with unique nam…
kdroidFilter Apr 3, 2026
0006b69
feat(tuple): implement nested tuple marshalling for FFM
kdroidFilter Apr 3, 2026
a1a41e4
add AGENTS.md with project overview,
kdroidFilter Apr 3, 2026
02f1ac5
update AGENTS.md
kdroidFilter Apr 3, 2026
de8d1a6
feat(tuple): make nested tuple support generic and fix memory leak
kdroidFilter Apr 3, 2026
d85056b
fix(vec-u8): support Option<Vec<u8>> return types in bridge generators
kdroidFilter Apr 3, 2026
32831b8
feat(rust-import): implement impl Trait return type support
kdroidFilter Apr 3, 2026
1a110a9
feat(rust-import): support Box<dyn Trait> returns via registry
kdroidFilter Apr 4, 2026
573b068
feat(&dyn Trait): support reference params via handle + transmute
kdroidFilter Apr 4, 2026
81f37bc
feat(tuple-vec): support Vec<T> elements in tuple returns
kdroidFilter Apr 4, 2026
d92975f
feat(rust): add Vec<String>/Vec<f64>/Vec<bool> returns and harden tes…
kdroidFilter Apr 4, 2026
4c17345
fix(rust): HashMap<K,V> parameter now passes parallel arrays instead …
kdroidFilter Apr 4, 2026
6d21e9c
feat(sealed-enum): support LIST/SET/MAP collection fields in variant …
kdroidFilter Apr 4, 2026
f311a47
feat(rust): support String LIST/SET elements via *const *const c_char…
kdroidFilter Apr 4, 2026
db2680b
feat(generics): auto-monomorphisation of generic methods with custom …
kdroidFilter Apr 5, 2026
1e73c37
feat(generics): auto-monomorphisation of generic structs with trait b…
kdroidFilter Apr 5, 2026
5c6e8d5
feat(sealed-enum): verify multi-field tuple variants (N > 1)
kdroidFilter Apr 5, 2026
3cea682
feat(sealed-enum): struct variants and enum-typed fields in sealed enums
kdroidFilter Apr 5, 2026
034b3d6
feat(parser): lazy cross-crate type resolution for re-exported types
kdroidFilter Apr 5, 2026
a25bbd1
feat(parser): nested cross-crate data classes and sub-crate enums
kdroidFilter Apr 5, 2026
0fe1db6
test(rust): verify &[u8] borrowed slice return mapped to ByteArray
kdroidFilter Apr 5, 2026
1c2fd2f
feat: data class properties and &mut self + Result verification
kdroidFilter Apr 5, 2026
cdf410b
feat(rust-camera): integrate nokhwa for real camera capture
kdroidFilter Apr 5, 2026
1e02467
feat: direct crate import without writing Rust code
kdroidFilter Apr 5, 2026
e3ec49e
feat: crate() import infrastructure + camera app with live feed
kdroidFilter Apr 5, 2026
26ae5f3
feat: nokhwa camera working via crate() import — zero Rust code
kdroidFilter Apr 5, 2026
e22fcb4
fix: nokhwa camera RGB decode and handle lifecycle
kdroidFilter Apr 5, 2026
1883610
feat: add symphonia audio example and improve Rust bridge generators
kdroidFilter Apr 5, 2026
52a15c3
feat: improve FFM proxy generator and refactor examples
kdroidFilter Apr 5, 2026
803869e
refactor: split symphonia example UI into component files
kdroidFilter Apr 5, 2026
943bd1f
delete AudioPlayer.kt
kdroidFilter Apr 5, 2026
1c6dfae
feat: add callback support for sealed enum variants in FFM proxy gene…
kdroidFilter Apr 5, 2026
4dd6419
feat: add callback support for handle-backed types in Rust bridge
kdroidFilter Apr 5, 2026
88b6d4a
feat: support dyn Trait as constructor and method params
kdroidFilter Apr 6, 2026
b1dfc98
feat: support impl Trait return types for top-level functions and met…
kdroidFilter Apr 6, 2026
55af3de
feat: build complete camera app mirroring sysinfo architecture
kdroidFilter Apr 6, 2026
e819f10
feat: discover top-level functions from submodules and fix OBJECT→ENU…
kdroidFilter Apr 6, 2026
aebd7b1
feat: add rust-rfd example module for native file dialogs
kdroidFilter Apr 6, 2026
b77748e
feat: support impl Future return types and impl ToString params
kdroidFilter Apr 6, 2026
258dc6f
fix: update tests for property promotion and new type support
kdroidFilter Apr 6, 2026
678e659
fix: resolve Rust bridge compilation errors for symphonia+cpal
kdroidFilter Apr 6, 2026
09a332e
fix: sanitize Kotlin keyword param names in FFM top-level function proxy
kdroidFilter Apr 6, 2026
abd9ed5
feat: generate suspend fun for Rust async methods
kdroidFilter Apr 6, 2026
e2bd22d
docs: split README into overview, Kotlin/Native, and Rust guides
kdroidFilter Apr 6, 2026
9f38c23
docs(rust): add real-world examples section (sysinfo, rfd, nokhwa)
kdroidFilter Apr 6, 2026
12f5684
docs(rust): replace examples with Compose Desktop snippets
kdroidFilter Apr 6, 2026
a24f728
docs(rust): document object lifecycle and close() semantics
kdroidFilter Apr 6, 2026
81668b3
docs(rust): clarify close() is optional in real-world examples
kdroidFilter Apr 6, 2026
8ae57b8
feat: add rust-tray-icon example with plugin fixes
kdroidFilter Apr 6, 2026
dc7aee6
feat: support nullable closures (Option<impl Fn(...)>) in Rust bridge
kdroidFilter Apr 6, 2026
0dff63f
feat: support opaque type parameters in Rust bridge
kdroidFilter Apr 6, 2026
d17a14d
docs(rust): update tray-icon limitations after opaque type support
kdroidFilter Apr 6, 2026
ea16ed7
feat: bridge MenuItem as opaque type in tray-icon example
kdroidFilter Apr 6, 2026
0480df3
docs(rust): clarify TrayIconBuilder.with_menu() resolved by wrapper p…
kdroidFilter Apr 6, 2026
e6ae647
feat: bridge impl Trait params and Box<dyn Trait> params for external…
kdroidFilter Apr 6, 2026
68b7a9d
docs(rust): remove resolved limitations from tray-icon table
kdroidFilter Apr 6, 2026
bd78a9e
test: add load and concurrency tests for impl Trait param bridging
kdroidFilter Apr 6, 2026
a4974a4
refactor(rust-tray-icon): minimize wrapper, remove make_icon_internal…
kdroidFilter Apr 6, 2026
6d1d238
fix: skip transitive dependency types in bridge generation
kdroidFilter Apr 6, 2026
d5f8e12
feat: support nested collections in Map values (Map<K, Vec<T>>)
kdroidFilter Apr 6, 2026
5f403aa
docs(rust): update HashMap entry in supported types table
kdroidFilter Apr 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,9 @@
.idea/
.kotlin/
build

# Rust / Cargo
target/
**/rust/src/kne_bridges.rs
**/rust/build.rs
Cargo.lock
297 changes: 297 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,297 @@
# AGENTS.md — Nucleus Native Access

## Project Overview

**Nucleus Native Access** is a Gradle plugin that bridges native code (Kotlin/Native or Rust) to the JVM via FFM (Foreign Function & Memory API). No JNI, no annotations, no boilerplate.

```
Native code (Kotlin/Native or Rust) → FFM bridges → JVM proxy classes
```

## Key Directories

```
├── plugin-build/plugin/src/main/kotlin/io/github/kdroidfilter/nucleusnativeaccess/plugin/
│ ├── ir/KneIR.kt # Shared IR: types, signatures, typeId generation
│ ├── analysis/
│ │ ├── RustdocJsonParser.kt # Parses rustdoc JSON → KneIR
│ │ └── RustWorkAction.kt # Orchestrates Rust pipeline
│ ├── codegen/
│ │ ├── FfmProxyGenerator.kt # JVM proxies with FFM MethodHandle (shared)
│ │ ├── RustBridgeGenerator.kt # Rust #[no_mangle] extern "C" bridges
│ │ └── NativeBridgeGenerator.kt # Kotlin/Native @CName bridges
│ └── tasks/
│ ├── GenerateRustBindingsTask.kt # Bridge generation task
│ └── CargoBuildTask.kt # cargo build invocation
├── examples/
│ └── rust-calculator/ # Rust example + tests
│ ├── rust/src/lib.rs # Rust source
│ └── build/generated/kne/ # Generated code (bridges, proxies)
│ ├── rustBridges/kne_bridges.rs
│ └── jvmProxies/com/example/rustcalculator/
└── tests/ # Integration tests
```

## Rust Bridge Pipeline

```
1. cargo rustdoc --output-format json → Extract public API
2. RustdocJsonParser.kt → Parse to KneIR
3. RustBridgeGenerator.kt → Generate #[no_mangle] pub extern "C" fn
4. FfmProxyGenerator.kt → Generate JVM proxies with FFM
5. cargo build --release → Compile Rust
6. Bundle .dylib into JAR → kne/native/{os}-{arch}/
```

## KneType.TUPLE and typeId

Tuples are represented as `KneType.TUPLE(elementTypes: List<KneType>)` with a `typeId` property that generates unique signatures:

```kotlin
// Format: T=Tuple, U=Unit, Z=Bool, B=Byte, S=Short, I=Int, J=Long, F=Float, D=Double, R=String
(i32, (String, bool)) → typeId = "TITRZ"
(i32, i32) → typeId = "TII"
(String, bool) → typeId = "TRZ"
```

## Rust Bridge Generation (RustBridgeGenerator.kt)

### Tuple Return Handling

For tuple returns, the bridge generates out-param code that writes each element to a buffer:

```rust
// For (i32, String, bool):
unsafe { *out_t_0 = result.0 as i32; }
// String handling: copy to buffer
let _e_bytes = result.1.as_bytes();
unsafe { std::ptr::copy_nonoverlapping(_e_bytes.as_ptr(), out_t_1, _e_bytes.len()); }
unsafe { *out_t_1.add(_e_bytes.len()) = 0; }
unsafe { out_t_2.write(if result.2 { 1 } else { 0 }); }
```

### Nested Tuple Handling

Nested tuples are handled specially — for `isOutParam=true`, a nested tuple gets a single `ADDRESS` layout (pointer to boxed tuple), not recursively expanded.

### IMPORTANT: Memory Layout Compatibility Issue

**Rust `(String, bool)` has DIFFERENT memory layout than Kotlin expects!**

Rust layout (32 bytes):
```
Offset 0-7: bool (1 byte) + padding 7 bytes
Offset 8-15: String.ptr (8 bytes)
Offset 16-23: String.len (8 bytes)
Offset 24-31: String.cap (8 bytes)
```

Kotlin `readTuple2_TRZ` expects:
```
Offset 0-7: String.ptr (8 bytes)
Offset 8-15: String.len (8 bytes)
Offset 16-23: String.cap (8 bytes)
Offset 24-31: bool (1 byte) + padding 7 bytes
```

**See NESTED_TUPLE_PROBLEM.md for full details and current status.**

## FFM Conventions

### ValueLayout mappings

| KneType | FFM Layout |
|---------|------------|
| INT | `JAVA_INT` (4 bytes) |
| LONG | `JAVA_LONG` (8 bytes) |
| BOOLEAN | `JAVA_INT` (0/1) |
| STRING | `ADDRESS` (pointer) |
| TUPLE | `ADDRESS` (pointer to heap buffer) |

### Function Descriptors

Tuple returns use `FunctionDescriptor.ofVoid(...)` with out-param layouts:
```kotlin
// (i32, i32) return:
FunctionDescriptor.ofVoid(JAVA_LONG, JAVA_INT, JAVA_INT)
// handle out_t_0 out_t_1

// (i32, String, bool) return:
FunctionDescriptor.ofVoid(JAVA_LONG, JAVA_INT, ADDRESS, JAVA_INT, JAVA_INT)
// handle out_t_0 out_t_1 out_t_1_len out_t_2
```

## Important Code Patterns

### When modifying RustBridgeGenerator.kt

1. **String returns**: Use buffer pattern with null-termination
2. **Nested tuples**: Allocate a `Box::into_raw(Box::new([0u8; 32]))` buffer and write fields in Kotlin's expected layout
3. **Empty strings**: Check `if _s_len > 0` before using pointer (Rust empty strings have ptr=1, len=0)

### When modifying FfmProxyGenerator.kt

1. **expandTupleParamLayouts**: For out-params, nested tuples use `ADDRESS` (pointer to boxed). Primitives use their actual `ffmLayout`, not `ADDRESS`.
2. **buildMethodDescriptor**: Tuple returns become `UNIT` with out-param expansion.

### Generated Files

Files in `build/generated/kne/` are **regenerated on every build** from source:
- `rustBridges/kne_bridges.rs` — from Rustdoc JSON
- `jvmProxies/` — from KneIR

**Do not edit generated files manually.** They are overwritten on next build.

To force regeneration:
```bash
rm -rf examples/rust-calculator/build/generated/kne/
./gradlew :examples:rust-calculator:assemble --no-build-cache
```

## Running Tests

```bash
# All tuple tests
./gradlew :examples:rust-calculator:jvmTest --tests "com.example.rustcalculator.TupleTest"

# Single test
./gradlew :examples:rust-calculator:jvmTest --tests "com.example.rustcalculator.TupleTest.get_nested_tuple*"

# With debug output
./gradlew :examples:rust-calculator:jvmTest --tests "com.example.rustcalculator.TupleTest.get_nested_tuple*" --info

# Force full rebuild (clears cache)
./gradlew :examples:rust-calculator:assemble --no-build-cache
./gradlew :examples:rust-calculator:jvmTest
```

## Debugging Tips

1. **SIGSEGV crashes**: Usually indicates invalid memory address being dereferenced. Check:
- Is the pointer 0 or very small (like 6 = string length)?
- Is the segment size set correctly with `.reinterpret(size)`?
- Is Rust writing to the correct memory location?

2. **WrongMethodTypeException**: FFM descriptor mismatch. Check:
- Does the Kotlin `FunctionDescriptor` match the Rust `extern "C"` signature?
- Did you change `expandTupleParamLayouts` without rebuilding native?

3. **String issues**: Empty strings in Rust have `ptr=1, len=0`. Always check `if _s_len > 0` before using pointer.

4. **Debug output**: Both Rust (`eprintln!`) and Kotlin (`System.err.println`) can be used for tracing. Rust debug goes to stderr captured by Gradle.

## Debugging Log Practice (IMPORTANT)

When fixing a complex bug, **maintain a debugging log** to avoid repeating the same mistakes. Before starting:

```
## Bug: [short description]

### Symptom
- What the user sees
- Error type, logs

### Root Cause (if known)
- What's actually happening

### Attempts

| # | Date | Change | Result | Notes |
|---|------|--------|--------|-------|
| 1 | ... | Modified X | Failed | SIGSEGV at ... |
| 2 | ... | Modified Y | WrongMethodTypeException | Descriptor mismatch |
| 3 | ... | Reverted Y, Modified Z | Partial | 7/8 tests pass |

### Key Findings
- Finding 1
- Finding 2
```

**Why this matters**: Complex bugs (like nested tuple memory layouts) require many iterations. Without logging, you risk:
- Forgetting what you already tried
- Repeating failed approaches
- Losing track of which combination of fixes worked

**Always document**:
- Every change attempted
- The exact error/log output after each attempt
- Why something failed
- What you learned about the problem

**Update NESTED_TUPLE_PROBLEM.md** with findings as you progress.

## Testing Philosophy

This project bridges native code to JVM via FFM — every call crosses the native boundary. **Unit tests at the bridge level are useless.** Only end-to-end tests that actually cross the FFM boundary are meaningful.

### Reference Implementation: Kotlin/Native Tests

The `examples/calculator` module has **1700+ end-to-end tests** covering every feature. Use these as inspiration for testing patterns.

**Key test categories** (from `EdgeCaseTest.kt`, `LoadAndConcurrencyTest.kt`):

1. **Boundary values**: `Int.MAX_VALUE`, `Int.MIN_VALUE`, overflow behavior
2. **Empty/non-empty variants**: empty strings, empty collections, zero values
3. **Unicode**: emoji, international characters, null chars
4. **All enum values**: test every enum variant explicitly
5. **Nullable transitions**: null ↔ non-null for all nullable types
6. **Lifecycle**: create/use/close cycles, multiple objects
7. **Exception recovery**: verify object still works after exception
8. **Integration**: combinations of features in sequence
9. **Load tests**: 100K+ FFM calls on single instance
10. **Concurrency**: multiple threads hitting native lib simultaneously

### Rust Tests: Current Status

The `examples/rust-calculator` tests are **less comprehensive** than Kotlin/Native. They cover basic functionality but lack:
- Load tests (100K+ calls)
- Concurrency tests (multi-threaded)
- Full edge case coverage for all types
- Exception recovery tests

### Test Naming Convention

```kotlin
@Test fun `edge prim - Int MAX_VALUE`() = Calculator(Int.MAX_VALUE).use { ... }
@Test fun `load - 100K add calls single instance`() = Calculator(0).use { ... }
@Test fun `concurrent - 10 threads x 10K adds on separate instances`() = ...
```

Format: `[category] - [description]`

Categories: `edge`, `str`, `enum`, `null`, `bytes`, `dc`, `obj`, `exc`, `cb`, `integration`, `load`, `concurrent`

### When to Add Tests

- **New type support**: Add edge cases for all boundary values
- **New return type**: Add empty/non-empty, null/non-null variants
- **Bug fix**: Add regression test that reproduces the bug
- **Memory issue**: Add load test that exercises the code path heavily

### Test Infrastructure

```kotlin
// AutoCloseable pattern for all native objects
class Calculator(autoCloseable) { ... }

// Extension for automatic cleanup
fun <T : AutoCloseable, R> T.use(block: (T) -> R): R {
return try { block(this) } finally { close() }
}

// Usage
Calculator(0).use { calc ->
calc.add(5)
assertEquals(5, calc.current)
} // automatically closed
```

## Conventions

- **Comments**: Write in English
- **Kotlin style**: Follow Kotlin conventions and idiomatic style
- **No Co-Authored-By**: Never add AI attribution in commits
- **Je ne cheat pas**: Implement fully regardless of complexity
- **Tests first**: Always run tests after changes, use `--no-build-cache`
- **Battle test edge cases**: Every new feature needs edge case tests, load tests, and concurrency tests
- **Update README**: After implementing a feature or fixing a bug, update the README.md to document the change (new supported type, limitation fixed, etc.)
Loading
Loading