What about client-server architecture?

[andrejlevkovitch]

At first I want to sorry about english. So this tool (color_coded) are greate! But... here is a several problems with it:

1. Speed. Lexical and semantic analizing is slow operations and it need many resources (cpu and memory), so, sometimes vim works very slow. Usage of memory by vim can be more then 1.8Gb!

2. Hard for debugging. color_coded can work only as integrated in vim plugin, so I don't know how debug it at real time or other way...

3. Stability. Sometimes color_coded chrashs. And it crash vim also. This hapens not offen, but this is very unpleasant.

4. Integrating with other editors

So, I think client-server architecture will fix all this problems. I write some prototype of server and client. Client fully writed by vimscript (but uses extarnal program md5sum for caching server results) and not required lua support.

What you think about it?

[andrejlevkovitch]

@jeaye I see that you start active development of this tool, so I want to you check out my prototype. I think this will very interesting for you. This prototype is ready to use, so it don't take a long time

[jeaye]

Hey @andrejlevkovitch. Thanks for the ticket and for pinging me to look at it again. You're right that I've been actively developing some big improvements to color_coded. Among them is actually a client/server separation, along the lines of what you developed! The big differences are that it's using NeoVim's RPC model and that it's been rewritten in Rust, to help with stability.

I think you're definitely on the right track with where things need to go and I appreciate you bringing it all up here. These are the current goals I have, working on the NeoVim + Rust version of color_coded:

goals
  ux
    easier to install (precompiled binaries)
      update llvm more easily (avoid portability issues)
    easier to configure (toml)
    match tracing (no more incorrect highlighting when typing or scrolling)
  features
    support for neovim
    lsp highlight support
    better logging (:CCLog to open log file in new buffer)
    syntax highlighting supported in multiple windows at once
    better tests
    easier to extend (multiple languages)
      clojure
      rust
  performance
    entirely async, using tokio
    minimize allocations
    keep track of compilations per hash, to avoid duplicate compilations

Stay tuned!

[andrejlevkovitch]

I see that some of your goals, like: match tracing, asynchronous, highlight in multiple windows - I already realize in my plugin. But I used new features of vim8 for it, like channels, textproperties and jobs. Neovim has no one from they (at this time). But I see that neovim has own similar features, so, it is passible to write client for hl-server by using standard neovim features. Unfortunately I don't use neovim and don't know its api

[jeaye]

Yeah, my goal is to ultimately support both Vim8 and Neovim, but I'm starting with neovim since it has a much better API for these sorts of things and I want to see how good the experience can actually be. From there, the plan is to try to bring the Vim8 version up to parity.

I appreciate your work on the hl-server and vim-hl-client! Going forward, my goal is for color_coded to be the gold standard for semantic highlighting. For that, I think the move to Rust is going to be important for maintaining code safety as the complexity grows. Getting people to contribute to color_coded has been tough and I think that's primarily because of the C++, multi-threading, templates, etc. I'm hoping that having a Rust code base, with good test coverage, will help encourage people to jump in and feel confident about making changes without breaking things.

[andrejlevkovitch]

So, good luck! Unfortunately I can't help with neovim and rust - I don`t work with first and don't know the second. But if you will need any other help - just contact me.