Skip to content

[TASK] Add a new NLI ball for non-linear operators #34

@shirohasuki

Description

@shirohasuki

Deliverables

  • Add a Nonlinear-Operators‘ RTL implementation in the prototype lib (under the arch path).
  • A Pull Request (PR) containing a test written in C for the Common Non-linear operation and a README to introduce your design.
  • Report the performance results in this issue.

Task Description

  • LLMs rely heavily on nonlinear operators like SiLU, RMSNorm, and Softmax. While linear layers are highly optimised, these nonlinear functions often require expensive high-precision floating-point arithmetic. Traditional lookup table (LUT) methods often fail to handle the extreme outliers common in modern LLMs, leading to significant accuracy degradation.
  • NLI is a calibration-free, hardware-friendly framework designed to efficiently approximate different nonlinear functions. Offline find optimal cutpoints, NLI maintains high accuracy even for extreme outliers in LLM inference, where traditional LUT methods usually collapse.
  • You can learn this methodology from the paper, "NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference". Design a ball for Buckyball based on this methodology.
  • Please refer to the previous Pull Request (Completed the development of ReluBall and further improved the operation manual #6) for the detailed ball implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions