BagIt detection result is ignored in `process_zip_file`, preventing metadata extraction

[takuma-kasai-oss]

Description

Summary

When uploading a BagIt-formatted ZIP file containing metadata/converter.json, the metadata is not automatically extracted. The BagIt detection logic exists in process_source but its return value is ignored by the caller process_zip_file, causing the detection result to be lost.

Expected Behavior

1. Upload a BagIt ZIP file with metadata/converter.json
2. System detects BagIt format via bagit_metadata_file?
3. handle_bagit_result is called, setting con_state to CONVERTED
4. Converter.metadata extracts data from converter.json
5. Metadata is populated in the Dataset

Actual Behavior

1. Upload a BagIt ZIP file
2. process_source detects BagIt and returns { is_bagit: true, metadata: nil }
3. Return value is ignored by process_zip_file
4. Since BagIt files don't contain acqus/procs files, final_parameters remains empty
5. process_zip_file returns nil
6. NmrMapper.process_ds returns NONE
7. con_state remains 0, no metadata processing occurs

Environment

• LabIMotion version: 2.0.0
• Chemotion ELN version: 2.2.0

---

Root Cause

File: lib/labimotion/utils/mapper_utils.rb

The process_source method correctly detects BagIt files and returns { is_bagit: true, metadata: nil }, but process_zip_file discards this return value:

def process_zip_file(zip_file_url, source_map)
  final_parameters = {}

  Zip::File.open(zip_file_url) do |zip_file|
    source_map['sourceSelector'].each do |source_name|
      process_source(zip_file, source_map[source_name], final_parameters)
      # ↑ Return value is discarded
    end
  end

  return { is_bagit: false, metadata: final_parameters } if final_parameters.present?

  nil  # BagIt files end up here because final_parameters is empty
end

---

Steps to Reproduce

1. Create a BagIt-formatted ZIP file containing:

   data/sample.jdx
   metadata/converter.json
   bagit.txt
   manifest-sha256.txt
   

2. The converter.json should contain valid metadata, e.g.:

   {
     "ols": "CHMO:0000593",
     "matches": [
       {
         "identifier": { "outputLayer": "set", "outputKey": "temperature" },
         "result": { "value": 298.0 }
       }
     ]
   }

3. Upload the ZIP file to a Sample's Analysis in Chemotion ELN

4. Check the Attachment's con_state in Rails console:

   att = Attachment.where("filename LIKE '%.zip'").order(created_at: :desc).first
   att.con_state  # => 0 (NONE) - should be 6 (COMPLETED)

---

Workaround

Manually trigger metadata processing after upload:

att = Attachment.find(ID)
att.update_column(:con_state, 3)  # CONVERTED
Labimotion::Converter.process_ds(att.id, User.first)

This confirms that downstream processing works correctly when properly invoked.

---

Suggested Fix

One possible fix is to check the return value of process_source and return early when BagIt is detected:

  def process_zip_file(zip_file_url, source_map)
    final_parameters = {}

    Zip::File.open(zip_file_url) do |zip_file|
      source_map['sourceSelector'].each do |source_name|
-       process_source(zip_file, source_map[source_name], final_parameters)
+       result = process_source(zip_file, source_map[source_name], final_parameters)
+       return result if result.is_a?(Hash) && result[:is_bagit]
      end
    end

    return { is_bagit: false, metadata: final_parameters } if final_parameters.present?

    nil
  end

---

Additional Context

• The bagit_metadata_file? method exists and correctly identifies BagIt files
• The handle_bagit_result method in nmr_mapper.rb is implemented and functional
• The Converter.collect_metadata method correctly parses converter.json
• BagIt support was added in commit 078e1f5 (2024-01-15)

[cllde8]

Hi @takuma-kasai-oss

Thanks for reporting this. To help us understand the scenario better, could you clarify:

1. Did you manually create and upload your own BagIt ZIP file to the ELN?
2. Or were you uploading raw files, which the ELN would normally process into a ZIP internally?

The normal workflow expects users to upload raw data, after which the ELN generates ZIPs for metadata extraction. Knowing exactly what was uploaded will help us determine whether this behavior is expected or if a code adjustment is needed.

Thanks!

[takuma-kasai-oss]

Thank you for the question!

To clarify: yes, I created a BagIt ZIP using an external tool and uploaded it directly to Chemotion ELN.

Use case
We are developing a research data pipeline that pre-packages raw NMR data (Bruker and JEOL formats) as
BagIt ZIPs with metadata/converter.json for upload to Chemotion ELN.

Why I believe this is an intended code path
process_source already has explicit BagIt detection that returns { is_bagit: true, metadata: nil }.
I also note that this code path is only reachable via NmrMapper (con_state=NMR), which handles
directly-uploaded .zip files. The ChemConverter route stores its output with con_state=CONVERTED
and calls Converter.process_ds directly, bypassing process_zip_file entirely. So the BagIt
detection in process_source was clearly written for this exact scenario.

The bug is that process_zip_file discards the return value, so handle_bagit_result is never called.

Happy to provide any further details if needed!