Skip to content

Basic syntax highlighting and file recognition#5

Open
VishalRaut2106 wants to merge 6 commits into
toon-format:mainfrom
VishalRaut2106:main
Open

Basic syntax highlighting and file recognition#5
VishalRaut2106 wants to merge 6 commits into
toon-format:mainfrom
VishalRaut2106:main

Conversation

@VishalRaut2106
Copy link
Copy Markdown
Member

🎨 Overview

Implements v0.1.0 with complete syntax highlighting and file recognition for TOON files.

✅ What's Added

  • Syntax highlighting - Complete TextMate grammar for TOON syntax
  • File recognition - .toon extension support with proper language ID
  • Language config - Indentation rules, brackets, auto-closing pairs
  • Example file - Basic TOON syntax demonstration

🌈 Syntax Support

  • Comments (#)
  • Strings (quoted and unquoted)
  • Numbers, booleans, null values
  • Object keys and nested structures
  • Array syntax with brackets

📋 Roadmap Progress

  • v0.0.x - Initial project setup ✅
  • v0.1.x - Basic syntax highlighting ✅ (this PR)
  • v0.2.x - Format validation (next)
  • v0.3.x - Code formatting and auto-completion (planned)
  • v1.0.0 - First stable release (planned)

🧪 Testing

  • Builds successfully (pnpm build)
  • Passes all ESLint rules (pnpm lint)
  • Syntax highlighting works for all TOON elements
  • File recognition with .toon extension
  • Proper indentation and bracket matching

🚀 Ready For

  • User testing with TOON files
  • Next phase: v0.2.x validation implementation
  • Community feedback on syntax highlighting

📝 Files Changed

  • package.json - Added language and grammar contributions
  • language-configuration.json - Language settings and indentation rules
  • syntaxes/toon.tmLanguage.json - Complete TextMate grammar
  • examples/basic.toon - Example TOON file for testing
  • README.md & CHANGELOG.md - Updated documentation

- TypeScript setup with strict type checking
- ESLint + formatting configured
- Build pipeline (pnpm build, pnpm dev)
- Basic extension scaffolding with hello command
- Ready for collaborative development

Follows roadmap:
- v0.0.x - Initial project setup ✅
- v0.1.x - Basic syntax highlighting (planned)
- v0.2.x - Format validation (planned)
- v0.3.x - Code formatting and auto-completion (planned)
- v1.0.0 - First stable release (planned)
- Add TextMate grammar for TOON syntax highlighting with support for comments, strings, numbers, booleans, null values, keys, and arrays
- Add language configuration with line comments, bracket matching, auto-closing pairs, and indentation rules
- Register `.toon` file extension with VS Code language system
- Add basic example file demonstrating TOON syntax features
- Update package.json to version 0.1.0 and include language and grammar contributions
- Update README.md with features section, example code, and roadmap progress
- Update CHANGELOG.md with v0.1.0 release notes
Copy link
Copy Markdown
Contributor

@johannschopplich johannschopplich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together! The extension scaffolding, package.json wiring, CI, and release workflow are all solid and reusable. The grammar needs some significant work before we can merge, though.

Spec issue: no comment syntax in TOON
The spec explicitly states that TOON has no comment syntax (called out in the intro, the YAML comparison, and the strict mode error list). The #.*$ pattern and "lineComment": "#" should be removed. A # in a TOON document is a string value, not a comment.

Example file uses YAML syntax, not TOON
hobbies: [reading, coding, hiking] is YAML-style. The TOON equivalent is hobbies[3]: reading,coding,hiking.

Missing: array header syntax
The most distinctive TOON construct (key[N]: and key[N]{f1,f2}:) isn't matched by the current key regex, which breaks when [N] or {fields} appears between the key and the colon. Array headers, tabular headers, and their value rows all end up unstyled.

Other gaps worth addressing:

  • Dotted keys (user.name) don't match (missing . in the character class)
  • List item markers (- ) are unstyled
  • ( and ) in language-configuration have no basis in the spec
  • #comments is ordered before #strings, so # inside a quoted string incorrectly gets comment scope

I'd suggest going through the spec (§5, §6, §7, §9) before reworking the grammar, especially the array header syntax. Happy to answer any questions about the spec along the way!

@VishalRaut2106
Copy link
Copy Markdown
Member Author

@johannschopplich Please review the requested updates.

@VishalRaut2106
Copy link
Copy Markdown
Member Author

@johannschopplich any update?

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initial TOON language support to the VS Code extension by registering the .toon language, shipping a TextMate grammar, and updating docs/examples to reflect the v0.1.0 milestone.

Changes:

  • Register TOON as a contributed language (.toon extension) with a TextMate grammar and language configuration.
  • Implement a first-pass TextMate grammar for keys/headers plus basic literals (quoted strings, numbers, booleans, null).
  • Update the command ID/message and refresh README/docs/changelog plus an example TOON file.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
package.json Bumps version to 0.1.0, contributes language + grammar, renames command.
syntaxes/toon.tmLanguage.json Adds initial TextMate grammar for TOON headers/keys and basic literal tokenization.
language-configuration.json Defines brackets, auto-closing pairs, and indentation rules for TOON.
src/extension.ts Renames the contributed command and updates the message text.
README.md Updates public-facing docs and examples for v0.1.0 language support.
docs/README.md Adds a quick reference and example for the TOON format.
examples/basic.toon Provides a sample .toon file for manual verification.
CHANGELOG.md Introduces a changelog entry for 0.1.0.
Comments suppressed due to low confidence (1)

README.md:57

  • README has two roadmap/status sections. The later “Project Status & Roadmap” still labels v0.0.x as “(current), which conflicts with the earlier v0.1.0status and the version bump inpackage.json`. Please consolidate/update so the current version and roadmap are consistent.
## Project Status & Roadmap

Following semantic versioning towards 1.0.0:

- **v0.0.x** - Initial project setup, bare minimum structure (current)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread package.json
Comment thread syntaxes/toon.tmLanguage.json
Comment thread language-configuration.json Outdated
Comment thread CHANGELOG.md
Copy link
Copy Markdown
Member Author

@VishalRaut2106 VishalRaut2106 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality: Structure follows VS Code extension best practices. Scaffolding, packaging configuration, and TypeScript setup are sound.
Test Coverage: Manual build/test steps described, but no automated tests for grammar or language features.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

README.md:61

  • README has two roadmap sections (## Roadmap and ## Project Status & Roadmap) that partially duplicate each other, and the latter still labels v0.0.x as “current” even though the file also states v0.1.0 is complete. Consider consolidating into a single roadmap/status section to avoid conflicting guidance.
## Project Status & Roadmap

Following semantic versioning towards 1.0.0:

- **v0.0.x** - Initial project setup, bare minimum structure (current)
- **v0.1.x** - Basic syntax highlighting and file recognition
- **v0.2.x** - Format validation and error detection
- **v0.3.x** - Code formatting and auto-completion
- **v1.0.0** - First stable release with full TOON format support

Comment on lines +50 to +56
"values": {
"patterns": [
{ "include": "#string-quoted" },
{ "include": "#number" },
{ "include": "#boolean" },
{ "include": "#null" }
]
Comment thread language-configuration.json Outdated
Comment thread CHANGELOG.md
Comment on lines +15 to +18
- List item markers: `- value`
- Strings (quoted), numbers, booleans, null values
- No comment syntax (TOON has none per spec)

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants