Prevent Forwarding of Same file Multiple Times

  • When routing, in case the network to destination node is temporarily down, Users may resend a heavy study many times. Conquest queues each image for sending as many times as it was received. This can (and does) cause a huge backlog, causing the user to re-send even more times.


    Is there a way Conquest could be set to send an image only once to a give destination even if more images of the same UID are received?


    Do with with immediate file-by-file forward and with delayed forward.

  • mukoya

    Changed the title of the thread from “Multiple Threads when Exporting” to “Prevent Forwarding of Same file Multiple Times”.
  • Forward only once:


    this would require memory of what has been forwarded before - sounds like a huge change to me, I don't think I will be able to help you with that.


    Although I guess it may be possible to check if the image has been already stored and block the delayed forward if it is. You can try:


    ImportConverter0 = ifnotempty "%QE0020,000d"; forward series to SERVER


    Hm, this will not work - it will stop the forward for the first image not the second. You would need a %Q command that does an image query - that is not there.


    You can code this idea in lua though - a reject() would probably stop any further processing.


    Marcel

  • Hi, Thank you for your response.


    Here is the ExportConverter:


    ExportConverter0 = ifequal "%o[,1]", "1"; forward compressed as j715 to CONQNL org MEDISOFT



    Here is the debug log from conquest, On-disk compression is set to j2:


    [CONQUESTSRV1] 0002,0010 22 UI TransferSyntaxUID "1.2.840.10008.1.2.4.91"

    [CONQUESTSRV1] OpenJP2 decompress started.

    [CONQUESTSRV1] Debug[OpenJP2 decompress]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] Info[OpenJP2(J2k)]: Start to read j2k main header (0).

    [CONQUESTSRV1] Info[OpenJP2(J2k)]: Main header has been correctly decoded.

    [CONQUESTSRV1] Info[OpenJP2(J2k)]: Header of tile 0 / 0 has been read.

    [CONQUESTSRV1] Info[OpenJP2(J2k)]: Tile 1/1 has been decoded.

    [CONQUESTSRV1] Info[OpenJP2(J2k)]: Image data has been updated with tile 1.

    [CONQUESTSRV1] OpenJP2 2.1.0 decompress time 16 milliseconds.

    [CONQUESTSRV1] JPEG compress started.

    [CONQUESTSRV1] Debug[CompressJPEG]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] JPEG Lossless

    [CONQUESTSRV1] , H = 512, W = 512, Bits = 16 in 16, Frames = 1

    [CONQUESTSRV1] JPEG Library compress 524288 bytes to 119751 bytes

    [CONQUESTSRV1] JPEG compress time 5 milliseconds.

    [CONQUESTSRV1] Query Tables: DICOMImages

    [CONQUESTSRV1] Columns : ObjectFile, DeviceName

    [CONQUESTSRV1] Where : SOPInstanc = '1.2.826.0.1.3680043.2.1125.5140671204796937584405629797026207043' AND ImagePat = 'P000803'

    [CONQUESTSRV1] Order : (null)

    [CONQUESTSRV1] Query Tables: DICOMImages

    [CONQUESTSRV1] Columns : SOPInstanc,SOPClassUI,ImageNumbe,ImageDate,ImageTime,EchoNumber,NumberOfFr,AcqDate,AcqTime,ReceivingC,AcqNumber,SliceLocat,SamplesPer,PhotoMetri,QRows,QColumns,BitsStored,ImageType,ImageID,ImagePat,SeriesInst

    [CONQUESTSRV1] Where : SOPInstanc = '1.2.826.0.1.3680043.2.1125.5140671204796937584405629797026207043' AND ImagePat = 'P000803'

    [CONQUESTSRV1] Order : (null)

    [CONQUESTSRV1] Update Table: DICOMImages

    [CONQUESTSRV1] Updates : SOPInstanc = '1.2.826.0.1.3680043.2.1125.5140671204796937584405629797026207043', SOPClassUI = '1.2.840.10008.5.1.4.1.1.2', ImageNumbe = '4', ImageDate = '20240422', ImageTime = '213156.810', NumberOfFr = '1', AcqDate = '20240422', AcqTime = '213132.149', AcqNumber = '2', SliceLocat = '548.0188', SamplesPer = '1', PhotoMetri = 'MONOCHROME2', QRows = '512', QColumns = '512', BitsStored = '16', ImageType = E'ORIGINAL\\PRIMARY\\AXIAL\\HELICAL', ImagePat = 'P000803', SeriesInst = '2.16.840.1.113662.2.1.99999.95746980938620838307892617361976', AccessTime = 1734182496, ObjectFile = E'P000803\\2.16.840.1.113662.2.1.99999.95746980938620838307892617361976_0002_000004_17341823920048.dcm', DeviceName = 'MAG0'

    [CONQUESTSRV1] Where : SOPInstanc = '1.2.826.0.1.3680043.2.1125.5140671204796937584405629797026207043' AND ImagePat = 'P000803'

    [CONQUESTSRV1] Rewritten file: c:\dicomserver\data\P000803\2.16.840.1.113662.2.1.99999.95746980938620838307892617361976_0002_000004_17341823920048.dcm

    [CONQUESTSRV1] Server Command := 0001

    [CONQUESTSRV1] Message ID := 0019

    [CONQUESTSRV1] 0000,0002 26 UI AffectedSOPClassUID "1.2.840.10008.5.1.4.1.1.2"

    [CONQUESTSRV1] 0000,0100 2 US CommandField 1

    [CONQUESTSRV1] 0000,0110 2 US MessageID 25

    [CONQUESTSRV1] 0000,0700 2 US Priority 0

    [CONQUESTSRV1] 0000,0800 2 US DataSetType 514

    [CONQUESTSRV1] 0000,1000 64 UI AffectedSOPInstanceU "1.2.826.0.1.3680043.2.1125.7217479768492378985935385367042781869"





    [CONQUESTSRV1] Exportconverter0.0 executes: ifequal "1", "1"

    [CONQUESTSRV1] ExportConverter0.1: forward c:\dicomserver\data\P000803\2.16.840.1.113662.2.1.99999.95746980938620838307892617361976_0002_000040_17341823930052.dcm to CONQNL

    [CONQUESTSRV1] Accepted compression: js

    [CONQUESTSRV1] JPEG decompress started.

    [CONQUESTSRV1] Debug[DecompressJPEGL]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] JPEG decompress time 3 milliseconds.

    [CONQUESTSRV1] JPEG-LS compress started.

    [CONQUESTSRV1] Debug[CompressJPEGLS]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] JPEG-LS compress 524288 bytes to 104792 bytes

    [CONQUESTSRV1] JPEGLS compress time 11 milliseconds.

  • I would like to try multiple threads to forward same study by creating several export converter lines based on last digit in SOPInstanceUID.


    Here is a sample line that seems to work:


    ExportConverter0 = ifequal "%o[,1]", "1"; forward compressed as j715 to CONQNL org MEDISOFT


    However, instead of creating 10 threads (0-9), I would like to create 5 by combining two digits for each line, (e.g 0 and 9, 2 and 8 etc).


    This is the converter I have created:


    ExportConverter0 = ifequal "%o[,1]", "1"; ifequal "%o[,1]", "9"; forward compressed as j715 to CONQNL org MEDISOFT


    However it looks like this syntax does not work in recent versions of conquest. Both conditions are ignored and all images even those that do not match any of the conditions are forwarded. Is there a trick we can use to combine two conditions in one converter with an "or" function?

  • Hm,


    about jpeg-ls, a compression factor of ~5 (524288 bytes to 104792) looks like lossy to me. You can check the log when you change j715


    For the other function (with delayed converters) have a look in association.lua


    ifequal skips one command, so the correct syntax is:


    ifequal "%o[,1]", "1"; forward compressed as j715 to CONQNL org MEDISOFT; ifequal "%o[,1]", "9"; forward compressed as j715 to CONQNL org MEDISOFT


    Marcel

  • About syntax for combining conditions, this is well understood. Thank you.



    About JPEG-LS, I have been observing file sizes as I change the compression values. File sizes remain the same.

    Below is an example. the compression was exactly the same at j702 and j719.

    The earlier example with 1:5 compression ration was for an image that had a lot of empty(black) spaces.

    Additionally, I was expecting conquest interface to report j7, it does report accepted compression as js.



    [CONQUESTSRV1] ExportConverter1.0: forward c:\dicomserver\data\P000803\2.16.840.1.113662.2.1.99999.19612974843351552399585725246290_0003_000029_17341854310000.dcm to CONQNL

    [CONQUESTSRV1] Accepted compression: js

    [CONQUESTSRV1] JPEG decompress started.

    [CONQUESTSRV1] Debug[DecompressJPEGL]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] JPEG decompress time 8 milliseconds.

    [CONQUESTSRV1] JPEG-LS compress started.

    [CONQUESTSRV1] Debug[CompressJPEGLS]: H = 512, W = 512, Bits = 16 in 16, Components = 1, Frames = 1

    [CONQUESTSRV1] JPEG-LS compress 524288 bytes to 227428 bytes

    [CONQUESTSRV1] JPEGLS compress time 10 milliseconds.

  • Logs on destination conquest v1.5.0e when sending as JPEG LS lossy:


    [MEDISOFT]

    [MEDISOFT] UPACS THREAD 10: STARTED AT: Mon Dec 16 12:42:23 2024

    [MEDISOFT] A-ASSOCIATE-RQ Packet Dump

    [MEDISOFT] Calling Application Title : "MEDISOFT "

    [MEDISOFT] Called Application Title : "CONQNL "

    [MEDISOFT] Application Context : "1.2.840.10008.3.1.1.1", PDU length: 32768

    [MEDISOFT] Number of Proposed Presentation Contexts: 2

    [MEDISOFT] Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1

    [MEDISOFT] Presentation Context 1 "1.2.840.10008.1.1" 1

    [MEDISOFT] Server Command := 0001

    [MEDISOFT] Message ID := 0cb1

    [MEDISOFT] 0000,0002 26 UI AffectedSOPClassUID "1.2.840.10008.5.1.4.1.1.2"

    [MEDISOFT] 0000,0100 2 US CommandField 1

    [MEDISOFT] 0000,0110 2 US MessageID 3249

    [MEDISOFT] 0000,0700 2 US Priority 0

    [MEDISOFT] 0000,0800 2 US DataSetType 258

    [MEDISOFT] 0000,1000 64 UI AffectedSOPInstanceU "1.2.826.0.1.3680043.2.1125.6003181448531765494251397096989978319"

    [MEDISOFT] 0002,0010 22 UI TransferSyntaxUID "1.2.840.10008.1.2.4.80"

    [MEDISOFT]

    [MEDISOFT] UPACS THREAD 11: STARTED AT: Mon Dec 16 12:42:23 2024

    [MEDISOFT] A-ASSOCIATE-RQ Packet Dump

    [MEDISOFT] Calling Application Title : "MEDISOFT "

    [MEDISOFT] Called Application Title : "CONQNL "

    [MEDISOFT] Application Context : "1.2.840.10008.3.1.1.1", PDU length: 32768

    [MEDISOFT] Number of Proposed Presentation Contexts: 2

    [MEDISOFT] Presentation Context 0 "1.2.840.10008.5.1.4.1.1.2" 1

    [MEDISOFT] Presentation Context 1 "1.2.840.10008.1.1" 1

    [MEDISOFT] Server Command := 0001

    [MEDISOFT] Message ID := 0d53

    [MEDISOFT] 0000,0002 26 UI AffectedSOPClassUID "1.2.840.10008.5.1.4.1.1.2"

    [MEDISOFT] 0000,0100 2 US CommandField 1

    [MEDISOFT] 0000,0110 2 US MessageID 3411

    [MEDISOFT] 0000,0700 2 US Priority 0

    [MEDISOFT] 0000,0800 2 US DataSetType 258

    [MEDISOFT] 0000,1000 64 UI AffectedSOPInstanceU "1.2.826.0.1.3680043.2.1125.3152129755873462350011746160147607818"

    [MEDISOFT] 0002,0010 22 UI TransferSyntaxUID "1.2.840.10008.1.2.4.80"

    [MEDISOFT] Query Tables: DICOMImages

    [MEDISOFT] Columns : ObjectFile, DeviceName

    [MEDISOFT] Where : SOPInstanc = '1.2.826.0.1.3680043.2.1125.3581730429409926774164550768619305617' AND ImagePat = 'P000803'

    [MEDISOFT] Order : (null)

    [MEDISOFT] FreeStore Left 1000891 on E:\

    [MEDISOFT] Add to Table: DICOMImages

    [MEDISOFT] Columns: SOPInstanc, SOPClassUI, ImageNumbe, ImageDate, ImageTime, NumberOfFr, AcqDate, AcqTime, AcqNumber, SliceLocat, SamplesPer, PhotoMetri, QRows, QColumns, BitsStored, ImageType, ImagePat, SeriesInst, AccessTime, ObjectFile, DeviceName

    [MEDISOFT] Values: '1.2.826.0.1.3680043.2.1125.3581730429409926774164550768619305617', '1.2.840.10008.5.1.4.1.1.2', '1', '20240422', '213156.723', '1', '20240422', '213131.650', '2', '563.0188', '1', 'MONOCHROME2', '512', '512', '16', 'ORIGINAL\PRIMARY\AXIAL\HELICAL', 'P000803', '2.16.840.1.113662.2.1.99999.95746980938620838307892617361976', 1734342144, 'Images\20240422\P000803\S101923\2\1.2.826.0.1.3680043.2.1125.3581730429409926774164550768619305617.dcm', 'MAG0'

    [MEDISOFT] Query Tables: DICOMSeries

    [MEDISOFT] Columns : SeriesInst,SeriesNumb,SeriesDate,SeriesTime,SeriesDesc,Modality,PatientPos,ContrastBo,Manufactur,ModelName,BodyPartEx,ProtocolNa,StationNam,Institutio,FrameOfRef,SeriesPat,StudyInsta

    [MEDISOFT] Where : SeriesInst = '2.16.840.1.113662.2.1.99999.95746980938620838307892617361976' AND SeriesPat = 'P000803'

  • Hi Marcel,


    Sorry to revisit this topic.


    The reason I was experimenting with JPEG-LS is that we have always had issues with JPEG-2000 lossy.


    A good number of our images( from certain equipment) are distorted when compressed with j2k lossy.


    Attached are 2 zip files with images from 2 of such equipment.

    I have included screenshots of how images look after jl20 compression.

    I have also included anonymized dicom files of before and after the compression.


    Hope we can find a solution for this.


    Thank you!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!