Readuntil data reception issue

[wzyuan358]

Hello,
I tested the signal receiving function of the ReadUntil API with an accumulating cache and found that some reads are missing parts of the signal (e.g., signals from the first and third seconds are combined, but the second is missing). I observed this issue not only during playback, but also in real nanopore sequencing. How can I fix it.

Client I used:

from read_until import AccumulatingCache, ReadUntilClient
client=ReadUntilClient(
    cache_type=AccumulatingCache,
    filter_strands=True,
    one_chunk=False,
    #calibrated_signal=True,
)

[0x55555555]

Hello @wzyuan358 ,

I think you are trying to report an issue for the repo https://github.com/nanoporetech/read_until_api, not https://github.com/nanoporetech/minknow_api?

We will need more information on the issue you are seeing to debug:

• How do you know reads are missing parts of the signal?
• Can you provide a minimal example script?
• What is the environment you are running the test in (os, device, etc)?

Thanks,

• George

[wzyuan358]

Hi @0x55555555 ,

Sorry, but I couldn’t find a place to report an issue in the read_until_api repository.
I have tested playback on the OS with WSL and a real dRNA sequencing run on the device (both using SQK-RNA004 kit), and observed the same issue.
I compared the signals collected from ReadUntil with those of the same read saved as POD5 by MinKNOW, and found that the signals collected via ReadUntil are missing some parts. Moreover, the signal segments from the same read were generated several seconds apart during playback.
I provide an example script below that was used to collect ReadUntil signals; however, the bulk FAST5 file used for playback is too large to upload.

from read_until import AccumulatingCache, ReadUntilClient
import numpy as np 
import sys
import time

client=ReadUntilClient(
    cache_type=AccumulatingCache,
    filter_strands=True,
    one_chunk=False,
    #calibrated_signal=True,
)
client.run()

num_pore=client.cache_size
ids=["" for i in range(num_pore)]
dats=[None for i in range(num_pore)]

print("init")
formnames=set()
outputs=[]

while client.is_running:
    time.sleep(1)
    print("iter")
    for c,r in client.get_read_chunks(num_pore,last=True):
        c = c-1
        id = r.id
        data = np.frombuffer(r.raw_data,dtype=client.signal_dtype)
        if c<10:
            print(c,r.id,data.shape)
        if ids[c]!=id:
            if ids[c]!="":
                outputs.append((ids[c],len(dats[c]),",".join(str(e) for e in dats[c])))
            ids[c]=id
            dats[c]=data 
        else:
            try:
                assert (data[:len(dats[c])]==dats[c]).all()   
                #print(c,id,ids[c],data.shape,dats[c].shape)
            except:
                print("error")
                print(data.shape,dats[c].shape)
                sys.exit()
            dats[c]=data
    
    curnames=set(ids)
    if curnames!=formnames:
        print("update reads",len(curnames),len(formnames),len(curnames-formnames))
        formnames=curnames
    
    if len(outputs)>=4000:
        break 

fout="outputs.txt"
with open(fout,"w") as save:
    for o in outputs:
        save.write(f"{o[0]}\t{o[1]}\t{o[2]}\n")
print("finish")

client.reset()

Thanks.

[wzyuan358]

I checked the data reception for a single channel using the minknow api within readuntil, and found that if the read in the channel does not change (the read id remains the same), new signal segments are only received approximately every 4 seconds.