Can't reproduce MSMT17 performance

[RRoundTable]

Based on README, the performace of ViT-B/16+ICS is 75.1%. But I got 71.2% with MSMT17_V2 dataset. ViT-B/16+ICS is evaluated on MSMT17_V2?

``` 023-07-05 13:54:03 transreid INFO: Namespace(config_file='configs/msmt17/vit_base_ics_384.yml', opts=['MODEL.DEVICE_ID', "('0')"]) 2023-07-05 13:54:03 transreid INFO: Loaded configuration file configs/msmt17/vit_base_ics_384.yml 2023-07-05 13:54:03 transreid INFO: MODEL: PRETRAIN_PATH: '/home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/vit_base_ics_cfs_lup.pth' PRETRAIN_HW_RATIO: 2 METRIC_LOSS_TYPE: 'triplet' IF_LABELSMOOTH: 'off' IF_WITH_CENTER: 'no' NAME: 'transformer' NO_MARGIN: True DEVICE_ID: ('2') TRANSFORMER_TYPE: 'vit_base_patch16_224_TransReID' STRIDE_SIZE: [16, 16] STEM_CONV: True # False for vanilla ViT-S # DIST_TRAIN: True

INPUT:
SIZE_TRAIN: [384, 128]
SIZE_TEST: [384, 128]
PROB: 0.5 # random horizontal flip
RE_PROB: 0.5 # random erasing
PADDING: 10
PIXEL_MEAN: [0.5, 0.5, 0.5]
PIXEL_STD: [0.5, 0.5, 0.5]

DATASETS:
NAMES: ('MSMT17_V2')
ROOT_DIR: ('/home/hpds/Repositories/ml-models/dataset')

DATALOADER:
SAMPLER: 'softmax_triplet'
NUM_INSTANCE: 4
NUM_WORKERS: 8

SOLVER:
OPTIMIZER_NAME: 'SGD'
MAX_EPOCHS: 120
BASE_LR: 0.0004
WARMUP_EPOCHS: 20
IMS_PER_BATCH: 64
WARMUP_METHOD: 'cosine'
LARGE_FC_LR: False
CHECKPOINT_PERIOD: 120
LOG_PERIOD: 20
EVAL_PERIOD: 120
WEIGHT_DECAY: 1e-4
WEIGHT_DECAY_BIAS: 1e-4
BIAS_LR_FACTOR: 2

TEST:
EVAL: True
IMS_PER_BATCH: 256
RE_RANKING: False
WEIGHT: '/home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/transformer_120.pth'
NECK_FEAT: 'before'
FEAT_NORM: 'yes'

OUTPUT_DIR: '../../log/transreid/msmt17/vit_base_ics_cfs_lup_384'

2023-07-05 13:54:03 transreid INFO: Running with config:
DATALOADER:
NUM_INSTANCE: 4
NUM_WORKERS: 8
REMOVE_TAIL: 0
SAMPLER: softmax_triplet
DATASETS:
NAMES: MSMT17_V2
ROOT_DIR: /home/hpds/Repositories/ml-models/dataset
ROOT_TRAIN_DIR: ../data
ROOT_VAL_DIR: ../data
INPUT:
PADDING: 10
PIXEL_MEAN: [0.5, 0.5, 0.5]
PIXEL_STD: [0.5, 0.5, 0.5]
PROB: 0.5
RE_PROB: 0.5
SIZE_TEST: [384, 128]
SIZE_TRAIN: [384, 128]
MODEL:
ATT_DROP_RATE: 0.0
COS_LAYER: False
DEVICE: cuda
DEVICE_ID: 0
DEVIDE_LENGTH: 4
DIST_TRAIN: False
DROPOUT_RATE: 0.0
DROP_OUT: 0.0
DROP_PATH: 0.1
FEAT_DIM: 512
GEM_POOLING: False
ID_LOSS_TYPE: softmax
ID_LOSS_WEIGHT: 1.0
IF_LABELSMOOTH: off
IF_WITH_CENTER: no
JPM: False
LAST_STRIDE: 1
METRIC_LOSS_TYPE: triplet
NAME: transformer
NECK: bnneck
NO_MARGIN: True
PRETRAIN_CHOICE: imagenet
PRETRAIN_HW_RATIO: 2
PRETRAIN_PATH: /home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/vit_base_ics_cfs_lup.pth
REDUCE_FEAT_DIM: False
RE_ARRANGE: True
SHIFT_NUM: 5
SHUFFLE_GROUP: 2
SIE_CAMERA: False
SIE_COE: 3.0
SIE_VIEW: False
STEM_CONV: True
STRIDE_SIZE: [16, 16]
TRANSFORMER_TYPE: vit_base_patch16_224_TransReID
TRIPLET_LOSS_WEIGHT: 1.0
OUTPUT_DIR: ../../log/transreid/msmt17/vit_base_ics_cfs_lup_384
SOLVER:
BASE_LR: 0.0004
BIAS_LR_FACTOR: 2
CENTER_LOSS_WEIGHT: 0.0005
CENTER_LR: 0.5
CHECKPOINT_PERIOD: 120
COSINE_MARGIN: 0.5
COSINE_SCALE: 30
EVAL_PERIOD: 120
GAMMA: 0.1
IMS_PER_BATCH: 64
LARGE_FC_LR: False
LOG_PERIOD: 20
MARGIN: 0.3
MAX_EPOCHS: 120
MOMENTUM: 0.9
OPTIMIZER_NAME: SGD
SEED: 1234
STEPS: (40, 70)
TRP_L2: False
WARMUP_EPOCHS: 20
WARMUP_FACTOR: 0.01
WARMUP_METHOD: cosine
WEIGHT_DECAY: 0.0001
WEIGHT_DECAY_BIAS: 0.0001
TEST:
DIST_MAT: dist_mat.npy
EVAL: True
FEAT_NORM: yes
IMS_PER_BATCH: 256
NECK_FEAT: before
RE_RANKING: False
WEIGHT: /home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/transformer_120.pth
MSMT17_V2 /home/hpds/Repositories/ml-models/dataset
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} cam_container
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} cam_container
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} cam_container
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} cam_container
=> MSMT17 loaded
2023-07-05 13:54:03 transreid.check INFO: Dataset statistics:
2023-07-05 13:54:03 transreid.check INFO: ----------------------------------------
2023-07-05 13:54:03 transreid.check INFO: subset | # ids | # images | # cameras
2023-07-05 13:54:03 transreid.check INFO: ----------------------------------------
2023-07-05 13:54:03 transreid.check INFO: train | 1041 | 32621 | 15
2023-07-05 13:54:03 transreid.check INFO: query | 3060 | 11659 | 15
2023-07-05 13:54:03 transreid.check INFO: gallery | 3060 | 82161 | 15
2023-07-05 13:54:03 transreid.check INFO: ----------------------------------------
using img_triplet sampler
using Transformer_type: vit_base_patch16_224_TransReID as a backbone
using stride: [16, 16], and patch number is num_y24 * num_x8
Resized position embedding from size:torch.Size([1, 129, 768]) to size: torch.Size([1, 193, 768]) with height:24 width: 8
Load 172 / 174 layers.
Loading pretrained ImageNet model......from /home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/vit_base_ics_cfs_lup.pth
===========building transformer===========
Loading pretrained model from /home/hpds/Repositories/ml-models/proto/TransReID-SSL/checkpoint/transformer_120.pth
2023-07-05 13:54:05 transreid.test INFO: Enter inferencing
True
torch.cuda.device_count() 1

The test feature is normalized
=> Computing DistMat with euclidean_distance
/home/hpds/Repositories/ml-models/proto/TransReID-SSL/transreid_pytorch/utils/metrics.py:12: UserWarning: This overload of addmm_ is deprecated:
addmm_(Number beta, Number alpha, Tensor mat1, Tensor mat2)
Consider using one of the following signatures instead:
addmm_(Tensor mat1, Tensor mat2, *, Number beta, Number alpha) (Triggered internally at ../torch/csrc/utils/python_arg_parser.cpp:1485.)
dist_mat.addmm_(1, -2, qf, gf.t())
distmat (11659, 82161) <class 'numpy.ndarray'>
2023-07-05 14:04:03 transreid.test INFO: Validation Results
2023-07-05 14:04:03 transreid.test INFO: mAP: 71.2%
2023-07-05 14:04:03 transreid.test INFO: CMC curve, Rank-1 :87.9%
2023-07-05 14:04:03 transreid.test INFO: CMC curve, Rank-5 :93.6%
2023-07-05 14:04:03 transreid.test INFO: CMC curve, Rank-10 :95.1%

[z666-123]

Did you solve this problem? I also can't reproduce the performance of the Market1501. My config is vit_base_ics_384.yml.
I just modified train.py in 75 line：

# model.load_param(“/pretrained_path”)
model.load_param(cfg.MODEL.PRETRAIN_PATH)

head -130  3_30_market_base_denoise.txt
nohup: ignoring input
2025-03-30 13:54:10 transreid INFO: Saving model in the path :./log/transreid/market/vit_base_denoise
2025-03-30 13:54:10 transreid INFO: Loaded configuration file configs/market/vit_base_ics_384.yml
2025-03-30 13:54:10 transreid INFO: Running with config:
DATALOADER:
  NUM_INSTANCE: 4
  NUM_WORKERS: 8
  REMOVE_TAIL: 0
  SAMPLER: softmax_triplet
DATASETS:
  NAMES: market1501
  ROOT_DIR: /media/data3/zhurh/data_78/
  ROOT_TRAIN_DIR: ../data
  ROOT_VAL_DIR: ../data
INPUT:
  PADDING: 10
  PIXEL_MEAN: [0.5, 0.5, 0.5]
  PIXEL_STD: [0.5, 0.5, 0.5]
  PROB: 0.5
  RE_PROB: 0.5
  SIZE_TEST: [384, 128]
  SIZE_TRAIN: [384, 128]
MODEL:
  ATT_DROP_RATE: 0.0
  COS_LAYER: False
  DEVICE: cuda
  DEVICE_ID: 3
  DEVIDE_LENGTH: 4
  DIFFUSION_LOSS_WEIGHT: 0.1
  DIST_TRAIN: False
  DROPOUT_RATE: 0.0
  DROP_OUT: 0.0
  DROP_PATH: 0.1
  FEAT_DIM: 512
  GEM_POOLING: False
  ID_LOSS_TYPE: softmax
  ID_LOSS_WEIGHT: 1.0
  IF_LABELSMOOTH: off
  IF_WITH_CENTER: no
  JPM: False
  LAST_STRIDE: 1
  METRIC_LOSS_TYPE: triplet
  NAME: transformer
  NECK: bnneck
  NO_MARGIN: True
  PRETRAIN_CHOICE: imagenet
  PRETRAIN_HW_RATIO: 2
  PRETRAIN_PATH: /media/data1/zhurh/pretrained_ckpt/vit_base_ics_cfs_lup.pth
  REDUCE_FEAT_DIM: False
  RE_ARRANGE: True
  SHIFT_NUM: 5
  SHUFFLE_GROUP: 2
  SIE_CAMERA: False
  SIE_COE: 3.0
  SIE_VIEW: False
  STEM_CONV: True
  STRIDE_SIZE: [16, 16]
  TRANSFORMER_TYPE: vit_base_patch16_224_TransReID
  TRIPLET_LOSS_WEIGHT: 1.0
OUTPUT_DIR: ./log/transreid/market/vit_base_denoise
SOLVER:
  BASE_LR: 0.0001
  BIAS_LR_FACTOR: 2
  CENTER_LOSS_WEIGHT: 0.0005
  CENTER_LR: 0.5
  CHECKPOINT_PERIOD: 120
  COSINE_MARGIN: 0.5
  COSINE_SCALE: 30
  EVAL_PERIOD: 120
  GAMMA: 0.1
  IMS_PER_BATCH: 128
  LARGE_FC_LR: False
  LOG_PERIOD: 20
  MARGIN: 0.3
  MAX_EPOCHS: 120
  MOMENTUM: 0.9
  OPTIMIZER_NAME: SGD
  SEED: 1234
  STEPS: (40, 70)
  TRP_L2: False
  WARMUP_EPOCHS: 20
  WARMUP_FACTOR: 0.01
  WARMUP_METHOD: cosine
  WEIGHT_DECAY: 0.0001
  WEIGHT_DECAY_BIAS: 0.0001
TEST:
  DIST_MAT: dist_mat.npy
  EVAL: True
  FEAT_NORM: yes
  IMS_PER_BATCH: 256
  NECK_FEAT: before
  RE_RANKING: False
  WEIGHT:
TRAIN:
  FT_EPOCH: 10
=> Market1501 loaded
2025-03-30 13:54:10 transreid.check INFO: Dataset statistics:
2025-03-30 13:54:10 transreid.check INFO:   ----------------------------------------
2025-03-30 13:54:10 transreid.check INFO:   subset   | # ids | # images | # cameras
2025-03-30 13:54:10 transreid.check INFO:   ----------------------------------------
2025-03-30 13:54:10 transreid.check INFO:   train    |   751 |    12936 |         6
2025-03-30 13:54:10 transreid.check INFO:   query    |   750 |     3368 |         6
2025-03-30 13:54:10 transreid.check INFO:   gallery  |   751 |    15913 |         6
2025-03-30 13:54:10 transreid.check INFO:   ----------------------------------------
/media/data3/zhurh/DenoiseRep/Person-ReID/TransReID-SSL/transreid_pytorch/model/backbones/vit_pytorch.py:356: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  param_dict = torch.load(model_path, map_location='cpu')
/media/data3/zhurh/DenoiseRep/Person-ReID/TransReID-SSL/transreid_pytorch/model/make_model.py:271: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  param_dict = torch.load(trained_path, map_location = 'cpu')
using img_triplet sampler
using Transformer_type: vit_base_patch16_224_TransReID as a backbone
using stride: [16, 16], and patch number is num_y24 * num_x8
Resized position embedding from size:torch.Size([1, 129, 768]) to size: torch.Size([1, 193, 768]) with height:24 width: 8
Load 172 / 174 layers.
Loading pretrained ImageNet model......from /media/data1/zhurh/pretrained_ckpt/vit_base_ics_cfs_lup.pth
===========building transformer===========
Loading pretrained model from /media/data1/zhurh/pretrained_ckpt/vit_base_ics_cfs_lup.pth
using soft triplet loss for training
2025-03-30 13:54:17 transreid INFO: ===========using cosine learning rate=======
2025-03-30 13:54:17 transreid.train INFO: start training
/media/data3/zhurh/DenoiseRep/Person-ReID/TransReID-SSL/transreid_pytorch/processor/processor.py:42: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
  scaler = amp.GradScaler()
/media/data3/zhurh/DenoiseRep/Person-ReID/TransReID-SSL/transreid_pytorch/processor/processor.py:63: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with amp.autocast(enabled=True):
2025-03-30 13:54:28 transreid.train INFO: Epoch[1] Iter[20/93] Loss: 29.797, Acc: 0.001, Base Lr: 5.95e-06
2025-03-30 13:54:35 transreid.train INFO: Epoch[1] Iter[40/93] Loss: 29.902, Acc: 0.001, Base Lr: 5.95e-06
2025-03-30 13:54:42 transreid.train INFO: Epoch[1] Iter[60/93] Loss: 29.828, Acc: 0.001, Base Lr: 5.95e-06

2025-03-30 15:06:47 transreid.train INFO: Epoch[119] Iter[40/93] Loss: 11.286, Acc: 0.258, Base Lr: 2.17e-07
2025-03-30 15:06:56 transreid.train INFO: Epoch[119] Iter[60/93] Loss: 11.176, Acc: 0.294, Base Lr: 2.17e-07
2025-03-30 15:07:04 transreid.train INFO: Epoch[119] Iter[80/93] Loss: 11.041, Acc: 0.353, Base Lr: 2.17e-07
2025-03-30 15:07:09 transreid.train INFO: Epoch 119 done. Time per epoch: 40.452[s] Speed: 284.8[samples/s]
2025-03-30 15:07:17 transreid.train INFO: Epoch[120] Iter[20/93] Loss: 11.430, Acc: 0.234, Base Lr: 2.00e-07
2025-03-30 15:07:24 transreid.train INFO: Epoch[120] Iter[40/93] Loss: 11.313, Acc: 0.264, Base Lr: 2.00e-07
2025-03-30 15:07:31 transreid.train INFO: Epoch[120] Iter[60/93] Loss: 11.198, Acc: 0.295, Base Lr: 2.00e-07
2025-03-30 15:07:38 transreid.train INFO: Epoch[120] Iter[80/93] Loss: 11.046, Acc: 0.344, Base Lr: 2.00e-07
2025-03-30 15:07:42 transreid.train INFO: Epoch 120 done. Time per epoch: 33.194[s] Speed: 350.9[samples/s]
/media/data3/zhurh/DenoiseRep/Person-ReID/TransReID-SSL/transreid_pytorch/utils/metrics.py:12: UserWarning: This overload of addmm_ is deprecated:
        addmm_(Number beta, Number alpha, Tensor mat1, Tensor mat2)
Consider using one of the following signatures instead:
        addmm_(Tensor mat1, Tensor mat2, *, Number beta = 1, Number alpha = 1) (Triggered internally at /opt/conda/conda-bld/pytorch_1729647352509/work/torch/csrc/utils/python_arg_parser.cpp:1642.)
  dist_mat.addmm_(1, -2, qf, gf.t())
The test feature is normalized
=> Computing DistMat with euclidean_distance
2025-03-30 15:08:58 transreid.train INFO: Validation Results - Epoch: 120
2025-03-30 15:08:58 transreid.train INFO: mAP: 88.8%
2025-03-30 15:08:58 transreid.train INFO: CMC curve, Rank-1  :94.7%
2025-03-30 15:08:58 transreid.train INFO: CMC curve, Rank-5  :98.1%
2025-03-30 15:08:58 transreid.train INFO: CMC curve, Rank-10 :98.8%