Nvidia Orin Deployment

[iMbaGong]

Hi, thank for great work!
When I try to build and run the yolo-world tensorrt model on nvidia orin nx(with cuda11.4 and tensorrt8.5.2), i meet a mistake. It's output score all became 100%. When i test on host machine, when docker build from nvcr.io/nvidia/tensorrt:24.02-py3, it is ok. But when it build from nvcr.io/nvidia/tensorrt:23.01-py3, the same problem happened.

[PrinceP]

@iMbaGong Can you provide examples for the output scores of 100%. Is confidence only getting wrong or the boxes are also issue?

Also, The operator einsum might cause issues with older 23.01
https://github.com/AILab-CVC/YOLO-World/blob/master/docs/deploy.md#faq

[iMbaGong]

just like this

And the operator einsum didn't cause a error in 23.01

[iMbaGong]

@PrinceP You can reproduce the problem by docker below

FROM nvcr.io/nvidia/cuda:11.4.3-devel-ubuntu20.04

# Install required packages for building and running the sample project
RUN apt-get -y update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata \
&& apt-get update && apt-get install -y \
    build-essential \
    cmake \
    git \
    pkg-config \
    libavcodec-dev \
    libhdf5-dev \
    libavformat-dev \
    libavdevice-dev \
    libcurl4-openssl-dev \
    libssl-dev \
    wget

RUN apt-get install -y \
    libopencv-dev

ENV cuda_version cuda11.8
ENV cudnn_version 8.6.0.163
ENV trt_version 8.5.2

RUN apt-get update \
&& apt-get install libcudnn8=${cudnn_version}-1+${cuda_version} \
    libcudnn8-dev=${cudnn_version}-1+${cuda_version} \
    libnvinfer8=${trt_version}-1+${cuda_version} \
    libnvinfer-plugin8=${trt_version}-1+${cuda_version} \
    libnvonnxparsers8=${trt_version}-1+${cuda_version} \
    libnvparsers8=${trt_version}-1+${cuda_version} \
    libnvinfer-dev=${trt_version}-1+${cuda_version} \
    libnvinfer-plugin-dev=${trt_version}-1+${cuda_version} \
    libnvonnxparsers-dev=${trt_version}-1+${cuda_version} \
    libnvparsers-dev=${trt_version}-1+${cuda_version}


ENV SPDLOGGER_VERSION 1.12.0
RUN wget -P /tmp https://github.com/gabime/spdlog/archive/v${SPDLOGGER_VERSION}.tar.gz
RUN tar -C /tmp -xzf /tmp/v${SPDLOGGER_VERSION}.tar.gz
RUN cp -R /tmp/spdlog-${SPDLOGGER_VERSION}/include/spdlog /usr/include/
RUN rm /tmp/v${SPDLOGGER_VERSION}.tar.gz

RUN rm -rf /workspace
WORKDIR /app

ENV NVIDIA_VISIBLE_DEVICES=all
ENV OPENCV_FFMPEG_LOGLEVEL=0

CMD [ "sleep", "infinity"]