Skip to content

Expand LSP capabilities, add full Go ASN.1 frontend, and broaden CI coverage#775

Closed
rafael2knokia wants to merge 5 commits into
masterfrom
vanadium-improvements
Closed

Expand LSP capabilities, add full Go ASN.1 frontend, and broaden CI coverage#775
rafael2knokia wants to merge 5 commits into
masterfrom
vanadium-improvements

Conversation

@rafael2knokia
Copy link
Copy Markdown
Collaborator

@rafael2knokia rafael2knokia commented May 22, 2026

Acknowledgement

This PR was inspired in significant part by
Vanadium by Mikhail Krylov
(BSD-3-licensed). The pure-Go ASN.1 frontend, the Wadler-style
formatter combinator layer, the schema-driven TTCN-3 AST generator
and the lint Rule lifecycle all follow Vanadium's design. The
attribution that should have shipped with the original commit is
now in THIRD_PARTY_NOTICES.md, in per-package
doc comments, and in the README's new Acknowledgements section.

No verbatim Vanadium source code is included; the ports are
reimplementations in Go.

Summary

This PR is a broad pass over the LSP, formatter, analysis, project,
and CI layers. It is split into self-contained commits so they can
be reviewed (or reverted) independently.

LSP

  • Wire the existing CLI lint engine and a new semantic analyzer into
    textDocument/publishDiagnostics.
  • Add a textDocument/codeAction handler that consumes autofix payloads
    produced by the linter, and emit a source.organizeImports action
    that sorts, dedupes, and respects visibility per module
    (issue Manage imports automatically #592).
  • Implement the standard methods that were previously stubs:
    signatureHelp, rename / prepareRename, documentHighlight,
    typeDefinition, workspaceSymbol, foldingRange, callHierarchy,
    and completionItem/resolve.
  • Cross-language go-to-definition: jumping from a TTCN-3 identifier
    into an ASN.1 module now works.
  • References and rename resolve through symbol definitions instead of
    raw name-text matching, eliminating cross-module false positives.
  • Switch textDocument/sync from Full to Incremental and splice
    changes into cached buffers per keystroke.
  • Flip diagnostics / format / semanticTokens / inlayHint defaults to
    on while keeping the legacy ttcn3.experimental.*.enabled keys as
    opt-out fallbacks.
  • Advertise the kinds we actually emit via CodeActionOptions so
    clients can pre-filter via the Only field.
  • Cache line-offset lookups in ttcn3/syntax.Root.searchLines with
    correctness + sequential/random benchmarks (issue Add cache for binary search in lines-map #640).

Formatter

  • Replace the tabwriter-based CanonicalPrinter with a Wadler-style
    document combinator layer plus a width-aware wrapping formatter,
    configurable via a new [tools.fmt] section in the manifest.
    (Design inspired by Vanadium's PrintDirective vocabulary.)

Lint / analysis

  • Mirror the formatter wiring with [tools.lint] for disabling
    individual rules.
  • Arena allocator with sync.Pool for hot lint paths.
  • Rule lifecycle inspired by Vanadium's Rule / Context.

ASN.1 frontend (internal/asn1)

A pragmatic but full-featured pure-Go frontend covering X.680, X.681,
X.682, and X.683. The cross-module Basket, the
WithSyntaxParser/ObjectSetResolver class driver and the
TTCN-3 lowering pass are direct Go reimplementations of Vanadium's
Asn1ModuleBasket, ClassObjectParser/ClassSetResolver and
Asn1AstTransformer.

  • New lexer, AST, and recursive-descent parser with a fully buffered
    token stream for reliable backtracking.
  • Semantic resolver with cross-module basket and EXPORTS/IMPORTS
    validation.
  • Parameterisation engine with cross-module chained substitution and
    caching.
  • WITH SYNTAX / object-set / component-relation class driver.
  • Lowering pass that emits parseable TTCN-3 source.
  • Hooked into ttcn3/db (ParseFileFull, ASN1Location) and the
    LSP definition handler.
  • Legacy asn1.Parse / asn1.ParseFile preserved via an adapter
    layer so existing callers don't break.

Project layer

Protocol / runtime

  • LSP 3.17 additions: pull-diagnostics, type hierarchy, position
    encoding.
  • Schema-driven AST generator under ttcn3/v2/syntax/nodes.
    (Schema layout adopted from Vanadium's src/ast/nodes.yml.)

CI

  • Add a 3-OS matrix (ubuntu-latest, macos-latest, windows-latest)
    to the Tests job (issue Fix tests for windows #629).
  • Enable core.longpaths=true on Windows before checkout so the
    ETSI conformance suite (paths up to 274 chars) can be cloned.
  • Normalise path-sensitive tests across internal/cache,
    internal/fs, internal/lsp, internal/lsp/span, and project
    so they pass on Windows without altering production behaviour.

Docs

  • New end-to-end Getting Started walkthrough at
    docs/getting-started.md, linked from README.md (issue Getting started is unfinished #572).
  • New THIRD_PARTY_NOTICES.md documenting third-party inspirations
    and licenses.

Linked issues

Closes #592, #629, #640, #650, #572.

Test plan

  • go test ./... green on Linux
  • go test ./... green on macOS (via CI matrix)
  • go test ./... green on Windows (via CI matrix)
  • golangci-lint run passes (lint job stays green)
  • New ASN.1 fixture + integration suite exercises parser ->
    resolver -> param -> class -> transform end-to-end
  • organize_imports action verified against unsorted, sorted,
    single-import, and multi-module inputs
  • .tpd loader unit-tested for source/folder/reference resolution
    and recursive ReferencedProjects expansion

Wire the existing CLI lint engine and a new semantic analyzer into LSP
diagnostics, add a code-action handler that consumes autofix payloads,
and implement the standard LSP methods that were previously missing:
signatureHelp, rename/prepareRename, documentHighlight, typeDefinition,
workspaceSymbol, foldingRange, callHierarchy, and completionItem/resolve.

References and rename now resolve through symbol definitions instead of
raw name-text matching, eliminating cross-module false positives.

Replace the tabwriter-based CanonicalPrinter with a Wadler-style
document combinator layer plus a width-aware wrapping formatter, both
configurable via a new [tools.fmt] section in the project manifest.
Mirror that with [tools.lint] for disabling individual rules.

Add a pragmatic ASN.1 frontend (internal/asn1) so .asn/.asn1 modules
contribute names to the database, a schema-driven AST generator under
ttcn3/v2/syntax/nodes, an arena allocator with sync.Pool for hot lint
paths, and LSP 3.17 protocol additions (pull-diagnostics, type
hierarchy, position encoding).

Switch text document sync from Full to Incremental and splice edits
into cached buffers per keystroke. Flip the diagnostics/format/
semanticTokens/inlayHint defaults to on while keeping the legacy
ttcn3.experimental.*.enabled keys as opt-out fallbacks.
ASN.1 frontend (X.680/X.681/X.682/X.683):
  * lexer, AST, recursive-descent parser with fully buffered token
    stream for reliable backtracking
  * semantic resolver with cross-module basket and EXPORTS/IMPORTS
    validation
  * parameterisation engine with cross-module chained substitution
    and caching
  * WITH SYNTAX / object set / component-relation class driver
  * lowering pass that emits parseable TTCN-3 source
  * wired into ttcn3/db (ParseFileFull, ASN1Location) and LSP
    definition handler for cross-language go-to-definition
  * legacy asn1.Parse / asn1.ParseFile preserved via adapter

Companion fixes for open github.com/nokia/ntt issues:
  * #629 Windows CI matrix + path-separator-safe fs tests
  * #572 end-to-end getting-started guide
  * #640 line-offset cache in ttcn3/syntax.Root.searchLines with
    correctness + sequential/random benchmarks
  * #592 source.organizeImports LSP code action that sorts, dedupes
    and respects visibility per module
  * #650 Titan .tpd loader with recursive ReferencedProjects support
    and project.Open / Discover integration
testdata/ttcn3-conformance-tests/ ships ~5800 ETSI files whose
relative paths exceed Windows' 260-character MAX_PATH limit (the
longest are 274 chars). Without core.longpaths=true the very first
actions/checkout step fails with "Filename too long" for every long
file, before any Go test ever runs.

Setting `git config --system core.longpaths true` before checkout
is the canonical fix - the action picks it up and the clone
succeeds.
The new Windows CI job surfaced a cluster of tests that hard-coded
forward slashes or the C: drive letter. Each fix normalises the
expectation rather than altering production behaviour:

  * internal/cache: filepath.Join the cache dir so the comparison
    uses native separators.
  * internal/fs: build the file:// URL via span.URIFromPath so
    Windows drive letters end up percent-encoded correctly, and
    feed JoinPath expectations through filepath.FromSlash for the
    plain-path cases (URL cases stay slash-only).
  * internal/lsp/hover: render the expected file:line marker with
    filepath.FromSlash so the leading separator matches the OS.
  * internal/lsp/organize_imports: query the WorkspaceEdit changes
    map using the same key construction as the production code,
    which round-trips the URI through Filename() (and therefore
    prepends a drive letter on Windows).
  * internal/lsp/span: derive the expected drive letter from the
    current working directory so the suite works on both C: and
    D: runners.
  * project: use filepath.FromSlash for relative hooks_file
    assertions, and special-case the "/file" case under
    runtime.GOOS == "windows" where it is genuinely not absolute.
@rafael2knokia rafael2knokia changed the title Close vanadium feature gap across LSP, formatter, and analysis Expand LSP capabilities, add full Go ASN.1 frontend, and broaden CI coverage May 22, 2026
@makekryl
Copy link
Copy Markdown

Well, despite the fact that I was not notified about this and noticed this by accident, I, as the Vanadium author, believe I have to say that plagiarizing a very massive, handmade work (which is still in progress and hasn't been abandoned!) with the help of AI agents (I'm not sure if it's possible to port this amount of complex logic otherwise in the in a record-breaking couple of hours, while I spent almost a year on it) - especially without leaving any credit to the original work! - is not a good move (rather, a very bad move) for the future of open-source in general.

If you like the tools I make, you could always contact me with a proposal to work on this.

@rafael2knokia rafael2knokia requested a review from moosq May 22, 2026 10:54
@rafael2knokia rafael2knokia marked this pull request as draft May 22, 2026 11:00
@rafael2knokia
Copy link
Copy Markdown
Collaborator Author

Well, despite the fact that I was not notified about this and noticed this by accident, I, as the Vanadium author, believe I have to say that plagiarizing a very massive, handmade work (which is still in progress and hasn't been abandoned!) with the help of AI agents (I'm not sure if it's possible to port this amount of complex logic otherwise in the in a record-breaking couple of hours, while I spent almost a year on it) - especially without leaving any credit to the original work! - is not a good move (rather, a very bad move) for the future of open-source in general.

If you like the tools I make, you could always contact me with a proposal to work on this.

Hi, this was not the intent, mostly an experiment. Your software is pretty complete, wonderful work.
I'll add the appropriate credits and keep this as draft.

Several pieces of this branch (the pure-Go ASN.1 frontend, the
Wadler-style formatter combinator layer, the schema-driven TTCN-3
AST generator and the lint Rule lifecycle) were modeled on the
design of Vanadium (https://github.com/makekryl/vanadium) by
Mikhail Krylov. The original PR landed without that attribution
and the upstream author rightfully flagged it.

Vanadium is BSD-3-licensed and so is ntt, so the BSD-3 attribution
requirement is satisfied by:

  * a new THIRD_PARTY_NOTICES.md that reproduces the Vanadium
    copyright notice + license text and lists the borrowed concepts
    (Asn1ModuleBasket -> resolver.Basket, ClassObjectParser/
    ClassSetResolver -> WithSyntaxParser/ObjectSetResolver,
    Asn1AstTransformer -> transform.LowerModule, nodes.yml schema
    -> ttcn3/v2/syntax/nodes, PrintDirective vocabulary -> ttcn3/
    format combinators, Rule lifecycle -> ttcn3/lint Rule);
  * per-package doc-comment pointers to Vanadium and the notices
    file in the most clearly derivative packages;
  * an Acknowledgements section in README.md.

No verbatim Vanadium source was copied; the ports are
reimplementations in Go.
@rafael2knokia
Copy link
Copy Markdown
Collaborator Author

rafael2knokia commented May 22, 2026

@makekryl You're right, and I'm sorry. Vanadium directly inspired multiple subsystems in this PR — most obviously the ASN.1 Basket / ObjectSetResolver / WithSyntaxParser architecture, the YAML-driven AST schema, the Wadler-style formatter combinators, and the lint Rule lifecycle. The branch name and the first commit title (Close vanadium feature gap…) make that clear internally, but I failed to carry that acknowledgement into the PR description, source files, or a NOTICE entry, and you found out about it by accident. That's not OK and I want to fix it.

I've just pushed 92dbe17 which:

  1. Adds a THIRD_PARTY_NOTICES.md crediting Vanadium and you as the author of the original work, with the full BSD-3 copyright/license notice and a list of the borrowed concepts (Asn1ModuleBasketresolver.Basket, ClassObjectParser / ClassSetResolverWithSyntaxParser / ObjectSetResolver, Asn1AstTransformertransform.LowerModule, nodes.yml schema → ttcn3/v2/syntax/nodes, PrintDirective vocabulary → ttcn3/format combinators, Rule lifecycle → ttcn3/lint).
  2. Adds attribution in the package-level doc comments of the most clearly derivative files (internal/asn1/resolver, internal/asn1/class, internal/asn1/transform, ttcn3/format/doc.go, ttcn3/v2/syntax/nodes/doc.go, ttcn3/lint/lint.go).
  3. Adds an Acknowledgements section to README.md pointing at the notices file.
  4. Rewrites the PR description so Vanadium's role is the very first thing a reviewer sees.

I started this as an experiment and apologize any inconvenient.

@rafael2knokia rafael2knokia deleted the vanadium-improvements branch May 22, 2026 11:04
@makekryl
Copy link
Copy Markdown

My sincere gratitude for refraining from creating diverging same-origin implementations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Manage imports automatically

2 participants