[BUG] GB18030 Detection Fails for Rare Characters

[EnthusiasticTech]

Project

cortex

Description

Chinese GB18030 4-byte sequences rejected.

Error Message

Debug Logs

System Information

$ bounty info
Bounty Challenge v0.1.0
OS: Linux 6.8.0-87-generic (Ubuntu 24.04.3 LTS)
CPU: Intel Core i7
RAM: 16 GB
Cortex Version: 0.1.1

Screenshots

No response

Steps to Reproduce

1. cortex import gb18030_session.json

Expected Behavior

Support full GB18030 character set.

Actual Behavior

Decode error on 4-byte sequences.

Additional Context

No response

[echobt]

Thank you for reporting this issue! We've confirmed the problem and submitted a fix.

The issue you described with Chinese GB18030 4-byte character sequences has been addressed. The import feature now properly handles files with a wider range of character encodings, including the full GB18030 character set with support for rare Chinese characters.

The fix is now under review: CortexLM/cortex-cli#152

Once the fix is merged, it will be available in the next release. Thanks for helping us improve Cortex!