Expand LSP capabilities, add full Go ASN.1 frontend, and broaden CI coverage#775
Expand LSP capabilities, add full Go ASN.1 frontend, and broaden CI coverage#775rafael2knokia wants to merge 5 commits into
Conversation
Wire the existing CLI lint engine and a new semantic analyzer into LSP diagnostics, add a code-action handler that consumes autofix payloads, and implement the standard LSP methods that were previously missing: signatureHelp, rename/prepareRename, documentHighlight, typeDefinition, workspaceSymbol, foldingRange, callHierarchy, and completionItem/resolve. References and rename now resolve through symbol definitions instead of raw name-text matching, eliminating cross-module false positives. Replace the tabwriter-based CanonicalPrinter with a Wadler-style document combinator layer plus a width-aware wrapping formatter, both configurable via a new [tools.fmt] section in the project manifest. Mirror that with [tools.lint] for disabling individual rules. Add a pragmatic ASN.1 frontend (internal/asn1) so .asn/.asn1 modules contribute names to the database, a schema-driven AST generator under ttcn3/v2/syntax/nodes, an arena allocator with sync.Pool for hot lint paths, and LSP 3.17 protocol additions (pull-diagnostics, type hierarchy, position encoding). Switch text document sync from Full to Incremental and splice edits into cached buffers per keystroke. Flip the diagnostics/format/ semanticTokens/inlayHint defaults to on while keeping the legacy ttcn3.experimental.*.enabled keys as opt-out fallbacks.
ASN.1 frontend (X.680/X.681/X.682/X.683):
* lexer, AST, recursive-descent parser with fully buffered token
stream for reliable backtracking
* semantic resolver with cross-module basket and EXPORTS/IMPORTS
validation
* parameterisation engine with cross-module chained substitution
and caching
* WITH SYNTAX / object set / component-relation class driver
* lowering pass that emits parseable TTCN-3 source
* wired into ttcn3/db (ParseFileFull, ASN1Location) and LSP
definition handler for cross-language go-to-definition
* legacy asn1.Parse / asn1.ParseFile preserved via adapter
Companion fixes for open github.com/nokia/ntt issues:
* #629 Windows CI matrix + path-separator-safe fs tests
* #572 end-to-end getting-started guide
* #640 line-offset cache in ttcn3/syntax.Root.searchLines with
correctness + sequential/random benchmarks
* #592 source.organizeImports LSP code action that sorts, dedupes
and respects visibility per module
* #650 Titan .tpd loader with recursive ReferencedProjects support
and project.Open / Discover integration
testdata/ttcn3-conformance-tests/ ships ~5800 ETSI files whose relative paths exceed Windows' 260-character MAX_PATH limit (the longest are 274 chars). Without core.longpaths=true the very first actions/checkout step fails with "Filename too long" for every long file, before any Go test ever runs. Setting `git config --system core.longpaths true` before checkout is the canonical fix - the action picks it up and the clone succeeds.
The new Windows CI job surfaced a cluster of tests that hard-coded
forward slashes or the C: drive letter. Each fix normalises the
expectation rather than altering production behaviour:
* internal/cache: filepath.Join the cache dir so the comparison
uses native separators.
* internal/fs: build the file:// URL via span.URIFromPath so
Windows drive letters end up percent-encoded correctly, and
feed JoinPath expectations through filepath.FromSlash for the
plain-path cases (URL cases stay slash-only).
* internal/lsp/hover: render the expected file:line marker with
filepath.FromSlash so the leading separator matches the OS.
* internal/lsp/organize_imports: query the WorkspaceEdit changes
map using the same key construction as the production code,
which round-trips the URI through Filename() (and therefore
prepends a drive letter on Windows).
* internal/lsp/span: derive the expected drive letter from the
current working directory so the suite works on both C: and
D: runners.
* project: use filepath.FromSlash for relative hooks_file
assertions, and special-case the "/file" case under
runtime.GOOS == "windows" where it is genuinely not absolute.
|
Well, despite the fact that I was not notified about this and noticed this by accident, I, as the Vanadium author, believe I have to say that plagiarizing a very massive, handmade work (which is still in progress and hasn't been abandoned!) with the help of AI agents (I'm not sure if it's possible to port this amount of complex logic otherwise in the in a record-breaking couple of hours, while I spent almost a year on it) - especially without leaving any credit to the original work! - is not a good move (rather, a very bad move) for the future of open-source in general. If you like the tools I make, you could always contact me with a proposal to work on this. |
Hi, this was not the intent, mostly an experiment. Your software is pretty complete, wonderful work. |
Several pieces of this branch (the pure-Go ASN.1 frontend, the Wadler-style formatter combinator layer, the schema-driven TTCN-3 AST generator and the lint Rule lifecycle) were modeled on the design of Vanadium (https://github.com/makekryl/vanadium) by Mikhail Krylov. The original PR landed without that attribution and the upstream author rightfully flagged it. Vanadium is BSD-3-licensed and so is ntt, so the BSD-3 attribution requirement is satisfied by: * a new THIRD_PARTY_NOTICES.md that reproduces the Vanadium copyright notice + license text and lists the borrowed concepts (Asn1ModuleBasket -> resolver.Basket, ClassObjectParser/ ClassSetResolver -> WithSyntaxParser/ObjectSetResolver, Asn1AstTransformer -> transform.LowerModule, nodes.yml schema -> ttcn3/v2/syntax/nodes, PrintDirective vocabulary -> ttcn3/ format combinators, Rule lifecycle -> ttcn3/lint Rule); * per-package doc-comment pointers to Vanadium and the notices file in the most clearly derivative packages; * an Acknowledgements section in README.md. No verbatim Vanadium source was copied; the ports are reimplementations in Go.
|
@makekryl You're right, and I'm sorry. Vanadium directly inspired multiple subsystems in this PR — most obviously the ASN.1 I've just pushed
I started this as an experiment and apologize any inconvenient. |
|
My sincere gratitude for refraining from creating diverging same-origin implementations |
Acknowledgement
This PR was inspired in significant part by
Vanadium by Mikhail Krylov
(BSD-3-licensed). The pure-Go ASN.1 frontend, the Wadler-style
formatter combinator layer, the schema-driven TTCN-3 AST generator
and the lint Rule lifecycle all follow Vanadium's design. The
attribution that should have shipped with the original commit is
now in
THIRD_PARTY_NOTICES.md, in per-packagedoc comments, and in the README's new Acknowledgements section.
No verbatim Vanadium source code is included; the ports are
reimplementations in Go.
Summary
This PR is a broad pass over the LSP, formatter, analysis, project,
and CI layers. It is split into self-contained commits so they can
be reviewed (or reverted) independently.
LSP
textDocument/publishDiagnostics.textDocument/codeActionhandler that consumes autofix payloadsproduced by the linter, and emit a
source.organizeImportsactionthat sorts, dedupes, and respects visibility per module
(issue Manage imports automatically #592).
signatureHelp,rename/prepareRename,documentHighlight,typeDefinition,workspaceSymbol,foldingRange,callHierarchy,and
completionItem/resolve.into an ASN.1 module now works.
raw name-text matching, eliminating cross-module false positives.
textDocument/syncfrom Full to Incremental and splicechanges into cached buffers per keystroke.
on while keeping the legacy
ttcn3.experimental.*.enabledkeys asopt-out fallbacks.
CodeActionOptionssoclients can pre-filter via the
Onlyfield.ttcn3/syntax.Root.searchLineswithcorrectness + sequential/random benchmarks (issue Add cache for binary search in lines-map #640).
Formatter
tabwriter-basedCanonicalPrinterwith a Wadler-styledocument combinator layer plus a width-aware wrapping formatter,
configurable via a new
[tools.fmt]section in the manifest.(Design inspired by Vanadium's PrintDirective vocabulary.)
Lint / analysis
[tools.lint]for disablingindividual rules.
sync.Poolfor hot lint paths.Rule/Context.ASN.1 frontend (
internal/asn1)A pragmatic but full-featured pure-Go frontend covering X.680, X.681,
X.682, and X.683. The cross-module
Basket, theWithSyntaxParser/ObjectSetResolverclass driver and theTTCN-3 lowering pass are direct Go reimplementations of Vanadium's
Asn1ModuleBasket,ClassObjectParser/ClassSetResolverandAsn1AstTransformer.token stream for reliable backtracking.
validation.
caching.
WITH SYNTAX/ object-set / component-relation class driver.ttcn3/db(ParseFileFull,ASN1Location) and theLSP definition handler.
asn1.Parse/asn1.ParseFilepreserved via an adapterlayer so existing callers don't break.
Project layer
.tpd) loader with recursive<ReferencedProjects>support, wired intoproject.Open,project.AutomaticRoot, andproject.Discover(issue Support Titan project descriptor files #650).Protocol / runtime
encoding.
ttcn3/v2/syntax/nodes.(Schema layout adopted from Vanadium's
src/ast/nodes.yml.)CI
ubuntu-latest,macos-latest,windows-latest)to the Tests job (issue Fix tests for windows #629).
core.longpaths=trueon Windows before checkout so theETSI conformance suite (paths up to 274 chars) can be cloned.
internal/cache,internal/fs,internal/lsp,internal/lsp/span, andprojectso they pass on Windows without altering production behaviour.
Docs
docs/getting-started.md, linked fromREADME.md(issue Getting started is unfinished #572).THIRD_PARTY_NOTICES.mddocumenting third-party inspirationsand licenses.
Linked issues
Closes #592, #629, #640, #650, #572.
Test plan
go test ./...green on Linuxgo test ./...green on macOS (via CI matrix)go test ./...green on Windows (via CI matrix)golangci-lint runpasses (lint job stays green)resolver -> param -> class -> transform end-to-end
organize_importsaction verified against unsorted, sorted,single-import, and multi-module inputs
.tpdloader unit-tested for source/folder/reference resolutionand recursive
ReferencedProjectsexpansion