feat: Add support for labelmap seg images in any supported tsuid (also for compressed bitmap) (#5806)
* feat: Update to use pnpm (#6031) [simulated squash-merge] * feat: load DICOM SEG images via imageLoader * feat: load DICOM SEG images via imageLoader * PR review fixes * Test fragment of compressed multiframes * fix: Removed dependencies incorrectly * Update frame as a targetted change to avoid stripping remaining params * lock * Update test to agree with the missing representation branch * test(contour): contour color change coverage (#6042) * test(contour): contour interactions delete segment (#6069) --------- Co-authored-by: Bill Wallace <wayfarer3130@gmail.com> * chore(version): Update package versions to 3.13.0-beta.98 [skip ci] * feat(testing): OHIF Test Agent Skills (#5993) * chore(version): Update package versions to 3.13.0-beta.99 [skip ci] * test(contour): Add the ContourSegmentToggleLock.spec.ts test file to test contour locking (#6072) * chore(version): Update package versions to 3.13.0-beta.100 [skip ci] * chore(testing): Flock lock for playwright tests; Pin node version for OHIF; add more workers (#6099) * update Cypress apt deps for Ubuntu Noble (drop libgconf-2-4, libasound2→libasound2t64) * chore(version): Update package versions to 3.13.0-beta.101 [skip ci] * Debug fixes * perf(seg): pass explicit frame decode concurrency (16) to SEG loader Pass an explicit concurrency value (SEG_FRAME_DECODE_CONCURRENCY = 16) into createFromDicomSegImageId rather than relying on the adapter default, so the SEG frame fetch/decode parallelism is set at the call site and is ready to become configurable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(behaviours): add behaviours section + multiframe Part 10 prefetch proposal Start a "Behaviours" docs section for documenting how the system and UI behave end-to-end (observed behaviours, design proposals, and failure modes), with an index README and a Docusaurus category. First entry is the proposal for loading a multiframe SEG as a single Part 10 instance: prefetch the whole instance (gated by loadMultiframeAsPart10RaceTimeMs), parse it with dcmjs (handling multipart/related vs raw DICOM), and register the per-frame compressed pixels into the Cornerstone3D core image cache (the single uniform frame registry) so the per-frame load path is served locally while the standard decode path is unchanged. Best-effort: falls back to per-frame fetches on any failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Pre-cache the image data using a full part10 file for performance * Add customizations to specify type of segmentation save * Add clearcache of the cacheData * Bump @cornerstonejs/* pins 5.1.3 -> 5.4.10 to match libs/@cornerstonejs base libs/@cornerstonejs (fix/use-imageLoader-for-seg) is based on the released cs3d 5.4.10 (merge-base with origin/main is the 5.4.10 version bump), so pin OHIF to that release. The local branch changes still reach the app via the cs3d:link symlinks; these pins keep the lockfile/npm fallback aligned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix seg OOM * fix: file meta garbage element, bogus NumberOfFrames, unguarded source-map-loader - dicomWriter: drop the naturalized meta.TransferSyntaxUID assignment that dcmjs wrote as a garbage (0000,0000) element into every saved file's meta group (download / clipboard / local wadouri blob / local store); keep the hex 00020010 assignment, which is required for string-form _meta fallbacks. - registerNaturalizedDatasetForLocalWadouri: keep the computed frame count local so single-frame IODs (SR, RTSTRUCT) no longer gain a bogus NumberOfFrames element in their serialized form. - rsbuild.config: resolve source-map-loader opportunistically (it is not a project dependency; the rule only serves the gitignored cs3d-link workflow) so fresh clones no longer crash at config load. - Regression tests for both writer fixes (mutation-checked). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * PR review comment fixes * Update versions --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: diattamo <mmddiatta@gmail.com> Co-authored-by: ohif-bot <danny.ri.brown+ohif-bot@gmail.com> Co-authored-by: Joe Boccanfuso <109477394+jbocce@users.noreply.github.com>
This commit is contained in:
59 files changed
+2501
-322
No files matched your search
+1
-1
@@ -7,5 +7,5 @@
|
||||
#
|
||||
|
||||
PUBLIC_URL=/
|
||||
APP_CONFIG=config/default.js
|
||||
# Can choose a default app config, but this over-rides dev scripts: APP_CONFIG=config/default.js
|
||||
USE_HASH_ROUTER=false
|
||||
@@ -51,7 +51,7 @@
|
||||
"@cornerstonejs/codec-libjpeg-turbo-8bit": "1.2.2",
|
||||
"@cornerstonejs/codec-openjpeg": "1.3.0",
|
||||
"@cornerstonejs/codec-openjph": "2.4.7",
|
||||
"@cornerstonejs/dicom-image-loader": "5.1.3",
|
||||
"@cornerstonejs/dicom-image-loader": "5.4.12",
|
||||
"@emotion/serialize": "1.3.3",
|
||||
"@ohif/core": "workspace:*",
|
||||
"@ohif/i18n": "workspace:*",
|
||||
@@ -62,7 +62,7 @@
|
||||
"classnames": "2.5.1",
|
||||
"core-js": "3.45.1",
|
||||
"cornerstone-math": "0.1.10",
|
||||
"dcmjs": "0.49.4",
|
||||
"dcmjs": "0.52.0",
|
||||
"detect-gpu": "4.0.50",
|
||||
"dicom-parser": "1.8.21",
|
||||
"file-loader": "6.2.0",
|
||||
|
||||
@@ -37,7 +37,6 @@ window.config = {
|
||||
// '/remote/': 'https://cdn.example.com/ohif-custom/', // ?customization=/remote/siteA
|
||||
// },
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
showStudyList: true,
|
||||
// some windows systems have issues with more than 3 web workers
|
||||
maxNumberOfWebWorkers: 3,
|
||||
@@ -80,6 +79,11 @@ window.config = {
|
||||
supportsFuzzyMatching: true,
|
||||
supportsWildcard: true,
|
||||
staticWado: true,
|
||||
// Multiframe SEG loads fetch the whole instance as a single Part 10
|
||||
// object by default and wait for it: the per-frame endpoint is
|
||||
// efficient, but SEG frames are so small and numerous that one bulk
|
||||
// fetch beats hundreds of tiny requests. Per-frame loading is the
|
||||
// exception — set loadMultiframeAsPart10: false here to force it.
|
||||
singlepart: 'bulkdata,video',
|
||||
bulkDataURI: {
|
||||
enabled: true,
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
// URL-loaded customization: segmentation/binary
|
||||
//
|
||||
// Stores exported/saved DICOM SEG objects as a BINARY Segmentation — SOP Class
|
||||
// 1.2.840.10008.5.1.4.1.1.66.4 (one frame per segment) — instead of the OHIF
|
||||
// default Label Map Segmentation 1.2.840.10008.5.1.4.1.1.66.7. Binary SEG is
|
||||
// broadly compatible with existing PACS/viewers; Label Map was only added to
|
||||
// DICOM in 2024 and many back ends reject it.
|
||||
//
|
||||
// Applied in the `global` phase so it overrides the default registered by the
|
||||
// cornerstone-dicom-seg extension. A data source may still override this per
|
||||
// back end via `configuration.segmentation.store.defaultMode`.
|
||||
//
|
||||
// Load it with `?customization=segmentation/binary` (combine with
|
||||
// `&customization=segmentation/uncompressed` to also drop RLE compression).
|
||||
{
|
||||
"global": {
|
||||
"segmentation.store.defaultMode": { "$set": "bitmap" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// URL-loaded customization: segmentation/uncompressed
|
||||
//
|
||||
// Stores exported/saved DICOM SEG objects UNCOMPRESSED — Explicit VR Little
|
||||
// Endian (1.2.840.10008.1.2.1) — instead of the OHIF default of RLE Lossless
|
||||
// (1.2.840.10008.1.2.5). Use this for back ends that reject compressed SEG
|
||||
// PixelData.
|
||||
//
|
||||
// Applied in the `global` phase. OHIF already defaults to RLE Lossless; this
|
||||
// customization opts into uncompressed Explicit VR Little Endian instead. A data
|
||||
// source may still override per back end via
|
||||
// `configuration.segmentation.store.transferSyntaxUID`.
|
||||
//
|
||||
// Load it with `?customization=segmentation/uncompressed` (combine with
|
||||
// `&customization=segmentation/binary` to also switch the SOP class).
|
||||
{
|
||||
"global": {
|
||||
"segmentation.store.transferSyntaxUID": { "$set": "1.2.840.10008.1.2.1" }
|
||||
}
|
||||
}
|
||||
@@ -35,16 +35,16 @@
|
||||
"@cornerstonejs/codec-libjpeg-turbo-8bit": "1.2.2",
|
||||
"@cornerstonejs/codec-openjpeg": "1.3.0",
|
||||
"@cornerstonejs/codec-openjph": "2.4.7",
|
||||
"@cornerstonejs/core": "5.1.3",
|
||||
"@cornerstonejs/dicom-image-loader": "5.1.3",
|
||||
"@cornerstonejs/metadata": "5.1.3",
|
||||
"@cornerstonejs/core": "5.4.12",
|
||||
"@cornerstonejs/dicom-image-loader": "5.4.12",
|
||||
"@cornerstonejs/metadata": "5.4.12",
|
||||
"@ohif/ui": "workspace:*",
|
||||
"cornerstone-math": "0.1.10",
|
||||
"dicom-parser": "1.8.21"
|
||||
},
|
||||
"dependencies": {
|
||||
"@babel/runtime": "7.29.7",
|
||||
"dcmjs": "0.49.4",
|
||||
"dcmjs": "0.52.0",
|
||||
"dicomweb-client": "0.10.4",
|
||||
"gl-matrix": "3.4.3",
|
||||
"immutability-helper": "3.1.1",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import queryString from 'query-string';
|
||||
import dicomParser from 'dicom-parser';
|
||||
import { utilities } from '@cornerstonejs/core';
|
||||
import { imageIdToURI } from '../utils';
|
||||
import { baseImageURIForMetadata } from '../utils/imageIdToURI';
|
||||
import DicomMetadataStore from '../services/DicomMetadataStore';
|
||||
import fetchPaletteColorLookupTableData from '../utils/metadataProvider/fetchPaletteColorLookupTableData';
|
||||
import toNumber from '../utils/toNumber';
|
||||
@@ -24,12 +24,12 @@ class MetadataProvider {
|
||||
// This method is a fallback for when you don't have WADO-URI or WADO-RS.
|
||||
// You can add instances fetched by any method by calling addInstance, and hook an imageId to point at it here.
|
||||
// An example would be dicom hosted at some random site.
|
||||
const imageURI = imageIdToURI(imageId);
|
||||
const imageURI = baseImageURIForMetadata(imageId);
|
||||
this.imageURIToUIDs.set(imageURI, uids);
|
||||
}
|
||||
|
||||
addCustomMetadata(imageId, type, metadata) {
|
||||
const imageURI = imageIdToURI(imageId);
|
||||
const imageURI = baseImageURIForMetadata(imageId);
|
||||
if (!this.customMetadata.has(type)) {
|
||||
this.customMetadata.set(type, {});
|
||||
}
|
||||
@@ -76,7 +76,7 @@ class MetadataProvider {
|
||||
// check inside custom metadata
|
||||
if (this.customMetadata.has(query)) {
|
||||
const customMetadata = this.customMetadata.get(query);
|
||||
const imageURI = imageIdToURI(imageId);
|
||||
const imageURI = baseImageURIForMetadata(imageId);
|
||||
if (customMetadata[imageURI]) {
|
||||
return customMetadata[imageURI];
|
||||
}
|
||||
@@ -445,6 +445,9 @@ class MetadataProvider {
|
||||
if (imageId.includes('/frames')) {
|
||||
return getInformationFromURL('/frames', '/');
|
||||
}
|
||||
if (imageId.includes('?frame=')) {
|
||||
return getInformationFromURL('?frame=', '&');
|
||||
}
|
||||
if (imageId.includes('&frame=')) {
|
||||
return getInformationFromURL('&frame=', '&');
|
||||
}
|
||||
@@ -473,20 +476,7 @@ class MetadataProvider {
|
||||
};
|
||||
}
|
||||
|
||||
// Maybe its a non-standard imageId
|
||||
// check if the imageId starts with http:// or https:// using regex
|
||||
// Todo: handle non http imageIds
|
||||
let imageURI;
|
||||
const urlRegex = /^(http|https|dicomfile):\/\//;
|
||||
if (urlRegex.test(imageId)) {
|
||||
imageURI = imageId;
|
||||
} else {
|
||||
imageURI = imageIdToURI(imageId);
|
||||
}
|
||||
|
||||
// remove &frame=number from imageId
|
||||
imageURI = imageURI.split('&frame=')[0];
|
||||
|
||||
const imageURI = baseImageURIForMetadata(imageId);
|
||||
const uids = this.imageURIToUIDs.get(imageURI);
|
||||
const frameNumber = this.getFrameInformationFromURL(imageId) || '1';
|
||||
|
||||
|
||||
@@ -10,6 +10,12 @@ import { dicomSplit } from './dicomSplit';
|
||||
* will be ignored.
|
||||
* This can be safely called with an undefined frame in order to handle
|
||||
* single frame data. (eg frame is undefined is the same as frame===1).
|
||||
*
|
||||
* Note: instances carry non-enumerable runtime props (frameNumber, imageId,
|
||||
* url, wadoRoot, ...). This is intentional: dcmjs serialization skips them and
|
||||
* spreads/copies (including of anything from `metaData.get('instance', ...)`)
|
||||
* deliberately do not carry them — frameNumber in particular must not be
|
||||
* copied onto other objects. Read runtime props off the original instance.
|
||||
*/
|
||||
const combineFrameInstance = (frame, instance) => {
|
||||
const {
|
||||
@@ -83,6 +89,9 @@ const combineFrameInstance = (frame, instance) => {
|
||||
if (!instance._parentInstance) {
|
||||
Object.defineProperty(instance, '_parentInstance', {
|
||||
value: { ...instance },
|
||||
enumerable: false,
|
||||
writable: false,
|
||||
configurable: false,
|
||||
});
|
||||
}
|
||||
const sharedInstance = createCombinedValue(
|
||||
@@ -102,7 +111,7 @@ const combineFrameInstance = (frame, instance) => {
|
||||
Object.defineProperty(newInstance, 'frameNumber', {
|
||||
value: frameNumber,
|
||||
writable: true,
|
||||
enumerable: true,
|
||||
enumerable: false,
|
||||
configurable: true,
|
||||
});
|
||||
return newInstance;
|
||||
@@ -113,6 +122,9 @@ const combineFrameInstance = (frame, instance) => {
|
||||
if (!instance._parentInstance) {
|
||||
Object.defineProperty(instance, '_parentInstance', {
|
||||
value: { ...instance },
|
||||
enumerable: false,
|
||||
writable: false,
|
||||
configurable: false,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -144,7 +156,7 @@ const combineFrameInstance = (frame, instance) => {
|
||||
Object.defineProperty(newInstance, 'frameNumber', {
|
||||
value: frameNumber,
|
||||
writable: true,
|
||||
enumerable: true,
|
||||
enumerable: false,
|
||||
configurable: true,
|
||||
});
|
||||
|
||||
|
||||
@@ -10,3 +10,20 @@ export default function imageIdToURI(imageId) {
|
||||
|
||||
return imageId.substring(colonIndex + 1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalizes an imageId to the metadata lookup key (scheme stripped, frame query removed).
|
||||
* Must match MetadataProvider.getUIDsFromImageID lookup behavior.
|
||||
*/
|
||||
export function baseImageURIForMetadata(imageId) {
|
||||
const urlRegex = /^(http|https|dicomfile):\/\//;
|
||||
let imageURI;
|
||||
|
||||
if (urlRegex.test(imageId)) {
|
||||
imageURI = imageId;
|
||||
} else {
|
||||
imageURI = imageIdToURI(imageId);
|
||||
}
|
||||
|
||||
return imageURI.split('&frame=')[0].split('?frame=')[0];
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
id: README
|
||||
title: Behaviours
|
||||
sidebar_position: 0
|
||||
---
|
||||
|
||||
# Behaviours
|
||||
|
||||
This section documents **how the system and UI actually behave** — the
|
||||
end-to-end behaviours that emerge from the interaction of services, extensions,
|
||||
the data source, and Cornerstone3D, rather than the API of any single module.
|
||||
|
||||
It is the place for:
|
||||
|
||||
- **Observed behaviours** — how a feature works today across the stack (e.g. how
|
||||
a DICOM SEG is fetched, decoded, and rendered; what the viewport does on study
|
||||
change; how measurements round-trip to SR).
|
||||
- **Design proposals** — proposed or in-progress changes to a behaviour, captured
|
||||
before/while they are implemented so the intent and trade-offs are recorded.
|
||||
These are clearly marked as proposals until they land.
|
||||
- **Edge cases and failure modes** — what happens when something is missing,
|
||||
slow, or malformed, and how the system is expected to degrade.
|
||||
|
||||
The goal is a durable, discoverable record of *behaviour* — the kind of
|
||||
cross-cutting knowledge that is otherwise only in people's heads or scattered
|
||||
across code comments. Prefer linking to the relevant source files (with line
|
||||
references) so each behaviour doc stays anchored to the code it describes.
|
||||
|
||||
## Index
|
||||
|
||||
- [Segmentation: loading a multiframe SEG as a single Part 10 instance](./segmentation-multiframe-part10-prefetch.md)
|
||||
— _implemented, enabled by default_. Prefetch the whole instance in one request
|
||||
and register it into the Cornerstone3D NATURALIZED frame registry so the
|
||||
per-frame load path (WADO-RS and WADO-URI) is served locally, while keeping the
|
||||
standard decode path unchanged. Per-frame loading is the exception — disable
|
||||
via `loadMultiframeAsPart10: false` (data source config or the
|
||||
`cornerstone.segmentation.loadMultiframeAsPart10` customization).
|
||||
|
||||
## Writing a new behaviour doc
|
||||
|
||||
1. Add a `kebab-case.md` file in this directory.
|
||||
2. State whether it documents **current behaviour** or is a **proposal**.
|
||||
3. Link to the code (`path:line`) that implements or will implement it.
|
||||
4. Add it to the **Index** above.
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Behaviours",
|
||||
"position": 14,
|
||||
"link": {
|
||||
"type": "doc",
|
||||
"id": "behaviours/README"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,368 @@
|
||||
# Full-instance prefetch for segmentation (and multiframe) loading
|
||||
|
||||
Status: **Implemented — enabled and awaited to completion by default;
|
||||
per-frame loading is the explicit opt-out**
|
||||
|
||||
The boolean `loadMultiframeAsPart10` resolves, in order: the data source
|
||||
`configuration`, the global customization
|
||||
`cornerstone.segmentation.loadMultiframeAsPart10`, then the built-in default of
|
||||
`true` — i.e. by default the SEG load **waits for the whole-instance
|
||||
fetch+parse to complete or fail** (deliberately no timeout) and serves every
|
||||
frame from the registry. Set `loadMultiframeAsPart10: false` explicitly to
|
||||
force per-frame loading — the exception, for back ends that need to fetch the
|
||||
individual images instead (e.g. servers that cannot serve a whole-instance
|
||||
retrieve, or deployments where holding the full Part 10 object in memory is
|
||||
undesirable). A failed or unsupported instance fetch resolves quickly and
|
||||
falls back to per-frame regardless of the setting, so it never wedges the load.
|
||||
|
||||
Note the per-frame endpoint itself is not inefficient — each frame request is
|
||||
cheap — but SEG frames are so small and numerous that one bulk Part 10 fetch
|
||||
beats hundreds of tiny requests (see Problem below). This holds even for very
|
||||
large SEG objects (hundreds of MB): any finite race cap would simply expire on
|
||||
those and storm per-frame anyway while abandoning the bulk fetch's benefit,
|
||||
which is why there is no timeout.
|
||||
|
||||
Implemented across:
|
||||
|
||||
- `@cornerstonejs/dicom-image-loader`
|
||||
- `imageLoader/prefetchPart10Instance.ts` — registers a Part 10 instance into
|
||||
the frame registry (thin wrapper over `addDicomPart10Instance`).
|
||||
- `imageLoader/wadors/loadImageFromRegistry.ts` +
|
||||
`wadors/loadImage.ts` — WADO-RS loads now consult the registry first.
|
||||
- `@ohif/extension-default` `DicomWebDataSource` — `retrieve.prefetchInstanceFrames`.
|
||||
- `@ohif/extension-cornerstone-dicom-seg` `getSopClassHandlerModule.ts` — call
|
||||
site (resolves the config and awaits the prefetch).
|
||||
|
||||
Related: `@cornerstonejs/adapters` `labelmapImagesFromBuffer.ts`
|
||||
(`decodeSegPixelDataFromFrameIds`, bounded by `concurrency`, default 16).
|
||||
|
||||
## Problem
|
||||
|
||||
The SEG load path now fetches/decodes frames through the cornerstone image
|
||||
loader, one loadable `imageId` per frame, parallelized up to `N=16`
|
||||
(`concurrency`). For a SEG (or any multiframe instance) with **800+ small
|
||||
frames**, this means 800+ independent HTTP requests, each with its own
|
||||
request/response overhead. Even at 16-wide concurrency the *per-request* latency
|
||||
floor dominates: each frame is < 1 KB of payload but pays a full round trip.
|
||||
|
||||
Streaming **one large object** (the entire Part 10 / multiframe instance) is far
|
||||
cheaper than streaming hundreds of tiny ones — a single connection, a single set
|
||||
of headers, no per-frame TTFB. The whole instance for an 800-frame binary SEG is
|
||||
typically a few hundred KB to a few MB.
|
||||
|
||||
## Goal
|
||||
|
||||
Add a **generic data-source capability** that, _just before_ the segmentation
|
||||
loader runs, optionally fetches the **entire original instance** in one request
|
||||
and **registers it into the Cornerstone3D frame registry** (the
|
||||
`@cornerstonejs/metadata` NATURALIZED + `COMPRESSED_FRAME_DATA` framework, parsed
|
||||
by the dcmjs async reader) — so the existing per-frame `imageId` fetch path
|
||||
transparently hits local data instead of the network, while the **cornerstone
|
||||
decode path stays byte-for-byte identical** (same decompressor, same workers).
|
||||
|
||||
Crucially this is **best-effort**:
|
||||
|
||||
- By default (`loadMultiframeAsPart10` unset → `true`) the load **waits for the
|
||||
full-instance fetch+parse to complete or fail** — no timeout — then every
|
||||
frame is served from the registry.
|
||||
- If `loadMultiframeAsPart10` resolves to `false` (explicitly configured), the
|
||||
capability is **disabled** — no full-instance fetch is attempted. Per-frame
|
||||
loading is the exception, opted into per deployment.
|
||||
- If the full-instance fetch or parse **fails for any reason**, it must **never**
|
||||
fail the segmentation decode. We log and fall back to per-frame fetches.
|
||||
|
||||
## Why this is safe / transparent
|
||||
|
||||
There is **one uniform frame registry: the Cornerstone3D `@cornerstonejs/metadata`
|
||||
NATURALIZED framework**, populated by `addDicomPart10Instance` (which parses the
|
||||
Part 10 with the dcmjs `AsyncDicomReader`) and read per-frame via the
|
||||
`COMPRESSED_FRAME_DATA` typed provider. Frame imageIds are normalized to the base
|
||||
instance by `baseImageIdQueryFilter` (strips `/frames/N`, `?frame=N`, `&frame=N`),
|
||||
so one registration under the instance serves every frame.
|
||||
|
||||
This is the **same registry the WADO-URI / `dicomweb` loader already uses**:
|
||||
`wadouri/loadImage.ts`'s `loadImageFromNaturalizedMetadata` resolves each frame
|
||||
from `COMPRESSED_FRAME_DATA` and decodes it with `createImage`. The gap was that
|
||||
**WADO-RS** (`wadors/loadImage.ts`) always issued a per-frame `/frames/N` request
|
||||
and never consulted the registry. The adapter closes that gap:
|
||||
|
||||
- `wadors/loadImageFromRegistry.ts` — `loadImageFromCompressedFrameRegistry(imageId)`
|
||||
looks up `COMPRESSED_FRAME_DATA` for the frame; if present it decodes via the
|
||||
**same `createImage`** path and returns the image; if absent it returns
|
||||
`undefined`.
|
||||
- `wadors/loadImage.ts` calls it first and short-circuits when the registry has
|
||||
the frame, otherwise falls through to the existing network path.
|
||||
|
||||
So once `prefetchPart10Instance` has registered the instance, **both WADO-URI and
|
||||
WADO-RS** serve every frame from the single registry, with the **same decode
|
||||
pipeline** (`createImage` → worker decoder). From cornerstone's perspective
|
||||
nothing changed except the compressed bytes came from the registry instead of the
|
||||
wire.
|
||||
|
||||
> Rejected alternatives: (1) a bespoke `Map<imageId, frame>` registry plus
|
||||
> `getPixelData` shims — duplicates a registry that already exists; (2) seeding
|
||||
> the core image cache (`cache.putImageLoadObject`) per frame — works, but the
|
||||
> per-frame compressed data already lives in the NATURALIZED registry, so basing
|
||||
> the design on that registry (and teaching WADO-RS to read it) keeps a single
|
||||
> source of truth instead of copying frames into a second cache.
|
||||
|
||||
## API (as implemented)
|
||||
|
||||
A generic capability on the data source's `retrieve` namespace (so it works for
|
||||
any multiframe instance, not just SEG, and alternate data sources can override
|
||||
it), plus the SEG handler call site that awaits it.
|
||||
|
||||
`IWebApiDataSource.create` passes `retrieve` through verbatim, so the capability
|
||||
is added there (no `@ohif/core` change):
|
||||
|
||||
```ts
|
||||
// On the data source (DicomWebDataSource):
|
||||
dataSource.retrieve.prefetchInstanceFrames({
|
||||
instance, // study/series/sop UIDs for retrieveInstance
|
||||
imageId, // SEG instance imageId (frame qualifiers normalized away)
|
||||
}): {
|
||||
done: Promise<boolean>; // resolves true if fetched+registered, false if skipped/failed
|
||||
cancel: () => void;
|
||||
};
|
||||
```
|
||||
|
||||
The implementation fetches the instance with `wadoDicomWebClient.retrieveInstance`
|
||||
(which returns the Part 10 as an `ArrayBuffer`, unwrapping `multipart/related`
|
||||
transparently) and passes a **lazy resolver** to
|
||||
`dicomImageLoader.prefetchPart10Instance(imageId, resolver)`. All work is wrapped
|
||||
so any failure resolves `done` to `false` — it never throws into the loader.
|
||||
|
||||
### Call site (just before the loader)
|
||||
|
||||
In `getSopClassHandlerModule.ts`, immediately before
|
||||
`createFromDicomSegImageId(...)` (abridged from the actual code):
|
||||
|
||||
```ts
|
||||
const loadMultiframeAsPart10 =
|
||||
(dataSource?.getConfig?.()?.loadMultiframeAsPart10 as boolean | undefined) ??
|
||||
(customizationService?.getCustomization?.(
|
||||
'cornerstone.segmentation.loadMultiframeAsPart10'
|
||||
) as boolean | undefined) ??
|
||||
true;
|
||||
|
||||
let prefetch;
|
||||
if (loadMultiframeAsPart10) {
|
||||
prefetch = dataSource.retrieve?.prefetchInstanceFrames?.({
|
||||
instance,
|
||||
imageId: segImageIdForMetadata,
|
||||
});
|
||||
if (prefetch?.done) {
|
||||
// Wait for the bulk fetch to complete or fail — no timeout.
|
||||
await prefetch.done;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
results = await adaptersSEG.Cornerstone3D.Segmentation.createFromDicomSegImageId(
|
||||
imageIds,
|
||||
segImageIdForMetadata,
|
||||
{ metadataProvider: metaData, tolerance, parserType, frameImageIds,
|
||||
concurrency: SEG_FRAME_DECODE_CONCURRENCY }
|
||||
);
|
||||
} finally {
|
||||
eventTarget.removeEventListener(Enums.Events.SEGMENTATION_LOAD_PROGRESS, onProgress);
|
||||
prefetch?.cancel?.();
|
||||
}
|
||||
```
|
||||
|
||||
The SEG loader and adapter are **unchanged** — the loader still asks the image
|
||||
loader for each frame. Registered frames resolve from the registry; the rest
|
||||
fetch normally.
|
||||
|
||||
## Mechanism (the 4 requested points)
|
||||
|
||||
### 1. Async DICOM reader (dcmjs) to parse the instance
|
||||
|
||||
Parsing is done by `@cornerstonejs/metadata`'s `addDicomPart10Instance`, which
|
||||
uses the dcmjs **`AsyncDicomReader`** (`naturalizePart10Buffer` in
|
||||
`naturalizedHandlers.ts`) and understands encapsulated PixelData fragmentation,
|
||||
so the NATURALIZED instance exposes pixel data as an array of per-frame
|
||||
ArrayBuffer fragments. `COMPRESSED_FRAME_DATA` then slices out the requested
|
||||
frame. We did not re-implement any parsing — the prefetch just feeds the fetched
|
||||
Part 10 buffer to this existing reader.
|
||||
|
||||
**Implemented as full-buffer v1**: the whole instance is fetched, then parsed.
|
||||
`addDicomPart10Instance` accepts a lazy resolver, so the fetch is what the race
|
||||
waits on. Progressive/streaming registration (registering frames as bytes arrive)
|
||||
is a future optimization — see rollout step 4.
|
||||
|
||||
#### Response envelope: `multipart/related` vs raw DICOM
|
||||
|
||||
The full-instance fetch can come back in **two shapes**, and the parser must
|
||||
handle both before any DICOM parsing happens:
|
||||
|
||||
- **`multipart/related`** — a WADO-RS instance retrieve
|
||||
(`GET …/instances/{sop}` with `Accept: multipart/related; type="application/dicom"`)
|
||||
wraps the Part 10 object(s) in a MIME multipart envelope: leading part headers,
|
||||
a `--boundary` marker, the DICOM bytes, then a terminal `--boundary--`. The
|
||||
inner DICOM byte offsets are **shifted** by the MIME header length, so dcmjs
|
||||
must be fed the *unwrapped* inner part, never the raw response.
|
||||
- **raw DICOM** — WADO-URI, a direct file/object URL, or a `bulkDataURI` that
|
||||
returns a single `application/dicom` body: the response **is** the Part 10
|
||||
bytes, no envelope.
|
||||
|
||||
**As implemented**, the full-buffer path delegates this to **`dicomweb-client`'s
|
||||
`retrieveInstance`**, which performs the WADO-RS instance retrieve and returns the
|
||||
Part 10 as an `ArrayBuffer` with the `multipart/related` envelope already
|
||||
unwrapped (and returns the single-part body as-is). `prefetchInstanceFrames`
|
||||
additionally tolerates an `[ArrayBuffer]` or `ArrayBufferView` shape defensively.
|
||||
So the implemented path is:
|
||||
|
||||
```
|
||||
retrieveInstance() → ArrayBuffer (multipart unwrapped by dicomweb-client)
|
||||
→ prefetchPart10Instance → addDicomPart10Instance → dcmjs AsyncDicomReader
|
||||
```
|
||||
|
||||
Detection by `Content-Type` and manual unwrapping (the approach `wadors/extractMultipart.ts`
|
||||
uses: `contentType.indexOf('multipart') === -1` → bytes as-is; else strip the
|
||||
boundary/part headers) becomes relevant only for the **future streaming path**
|
||||
(rollout step 4), where these wrinkles must be handled directly:
|
||||
|
||||
- The multipart preamble/boundary stripped at the **start** of the stream and the
|
||||
terminal boundary detected at the **end**; a boundary token can **straddle chunk
|
||||
boundaries**, so the unwrapper must buffer across chunks (the existing
|
||||
`findIndexOfString` + `tokenIndex` carry-over in `extractMultipart` is built for
|
||||
incremental calls).
|
||||
- `multipart/related` may contain **multiple parts**; a single-instance retrieve
|
||||
yields one — guard for >1.
|
||||
- A truncated/partial body (no terminal boundary yet) must not be mis-parsed.
|
||||
|
||||
### 2. Register the compressed data into the frame registry
|
||||
|
||||
The fetched Part 10 buffer is registered **once per instance** via the
|
||||
`prefetchPart10Instance` adapter (a thin wrapper over `addDicomPart10Instance`):
|
||||
|
||||
```ts
|
||||
// @cornerstonejs/dicom-image-loader: imageLoader/prefetchPart10Instance.ts
|
||||
import { utilities } from '@cornerstonejs/metadata';
|
||||
const { addDicomPart10Instance } = utilities;
|
||||
|
||||
export function prefetchPart10Instance(baseImageId, part10 /* buffer | resolver */) {
|
||||
return addDicomPart10Instance(baseImageId, part10);
|
||||
}
|
||||
```
|
||||
|
||||
That populates NATURALIZED for the instance; `COMPRESSED_FRAME_DATA` then yields
|
||||
each frame's compressed bytes + transfer syntax on demand. Frame imageIds
|
||||
normalize to the instance key automatically (`baseImageIdQueryFilter`), so there
|
||||
is no per-frame registration loop and no second cache to keep coherent — the
|
||||
registry **is** the existing NATURALIZED framework.
|
||||
|
||||
The "few new adapters" the task calls for:
|
||||
|
||||
1. `prefetchPart10Instance(baseImageId, part10)` — entry point to register a
|
||||
fetched instance into the registry (exported from
|
||||
`@cornerstonejs/dicom-image-loader`).
|
||||
2. `loadImageFromCompressedFrameRegistry(imageId, options)` in
|
||||
`wadors/loadImageFromRegistry.ts` — teaches the **WADO-RS** loader to read the
|
||||
registry. It looks up `COMPRESSED_FRAME_DATA` for the frame and, if present,
|
||||
decodes via the same `createImage` the network path uses; `wadors/loadImage.ts`
|
||||
calls it first and short-circuits on a hit. (WADO-URI already reads the
|
||||
registry via `loadImageFromNaturalizedMetadata`.)
|
||||
|
||||
Metadata the decoder needs (`imagePixelModule`, transfer syntax) comes from the
|
||||
same NATURALIZED instance, plus the SEG handler's existing
|
||||
`_ensureSegInstanceMetadataAvailable` per frame imageId.
|
||||
|
||||
### 3. Cornerstone uses its existing decompressor — identical downstream
|
||||
|
||||
Both the WADO-URI path and the new WADO-RS registry path decode via
|
||||
`createImage` (→ the same web-worker decoder used for every network frame), so
|
||||
encapsulated transfer syntaxes are decompressed exactly as if fetched per-frame
|
||||
and native/uncompressed frames pass through unchanged. No new decode path, no
|
||||
codec changes, no divergence in pixel output — the only difference is that the
|
||||
compressed bytes came from the registry instead of the wire.
|
||||
|
||||
### 4. Failure is non-fatal — fall back to per-frame fetches
|
||||
|
||||
`prefetchInstanceFrames` wraps the fetch + parse so that **any** failure (network
|
||||
error, cancel, parse error, unexpected `retrieveInstance` shape) resolves
|
||||
`done` to `false` and logs a warning — it never throws into the SEG load. When
|
||||
the registry has no data for a frame, `loadImageFromCompressedFrameRegistry`
|
||||
returns `undefined` and the WADO-RS loader falls through to its normal
|
||||
`/frames/N` request. So if the prefetch is slow, disabled, or fails, loading is
|
||||
exactly today's per-frame behaviour.
|
||||
|
||||
Edge cases that degrade gracefully:
|
||||
|
||||
- `cancel()` (viewport closed / segmentation removed mid-load) → the resolver
|
||||
throws on its cancelled flag; `done` resolves `false`; any already-registered
|
||||
data stays usable.
|
||||
- A misbehaving metadata provider → the registry lookup is wrapped in try/catch
|
||||
and returns `undefined`, so per-frame loading is never broken.
|
||||
|
||||
## Semantics (the `loadMultiframeAsPart10` knob)
|
||||
|
||||
- Unset (default) or `true` → start the full-instance fetch+parse and **await
|
||||
it to completion or failure** (`await prefetch.done`) — deliberately no
|
||||
timeout; every frame is then served from the registry. Fetch failure resolves
|
||||
`done` quickly, so unsupported servers fall straight through to per-frame.
|
||||
- `false` (explicitly configured) → **disabled**; per-frame loading is the
|
||||
exception, opted into per deployment.
|
||||
|
||||
> `addDicomPart10Instance` registers the instance atomically (one parse), so
|
||||
> registration is all-or-nothing rather than progressive. Progressive
|
||||
> registration (rollout step 4) would let frames be served as they arrive.
|
||||
|
||||
There is no timeout because a bounded wait loses on exactly the objects where
|
||||
the prefetch matters most: a large SEG (hundreds of MB) cannot finish inside
|
||||
any small cap, so a capped race expires and storms per-frame anyway while
|
||||
abandoning the bulk fetch's benefit. The only failure mode a timeout would
|
||||
bound — a fetch that never settles — is already covered by the browser's
|
||||
network-level failure surfacing through `done`.
|
||||
|
||||
## Where the generic capability lives
|
||||
|
||||
- The **registry is the existing `@cornerstonejs/metadata` NATURALIZED framework**
|
||||
— no new cache. New code in `@cornerstonejs/dicom-image-loader` is the thin
|
||||
adapter `prefetchPart10Instance()` plus the WADO-RS read path
|
||||
(`wadors/loadImageFromRegistry.ts`, wired into `wadors/loadImage.ts`).
|
||||
- The **fetch + register orchestration** lives in the **data source**
|
||||
(`extensions/default/.../DicomWebDataSource` → `retrieve.prefetchInstanceFrames`),
|
||||
because it knows how to retrieve a full instance (auth headers, `wadoRoot`,
|
||||
WADO-RS instance retrieve via `dicomweb-client`, CORS), and so alternate data
|
||||
sources can implement/override it.
|
||||
- The SEG handler only **reads the configuration and awaits the prefetch**.
|
||||
|
||||
This keeps the layering clean: data source = "how to get bytes", image loader =
|
||||
"how to register/decode bytes", SEG handler = "when to ask".
|
||||
|
||||
## Open questions / risks
|
||||
|
||||
1. **NATURALIZED registry key alignment.** Both registration
|
||||
(`addDicomPart10Instance`) and the WADO-RS read normalize frame imageIds to the
|
||||
base instance via `baseImageIdQueryFilter`, so they agree. Worth a sanity check
|
||||
per data source that the SEG `imageId` passed to `prefetchInstanceFrames` and
|
||||
the frame imageIds the SEG loader requests share the same instance base.
|
||||
2. **Memory.** Holding the full instance plus the still-compressed frames in the
|
||||
registry raises memory for that instance until it is evicted. Compressed frames
|
||||
are small and decode is on-demand. Registry/cache eviction lifecycle for
|
||||
prefetched instances should be verified (especially repeated SEG loads).
|
||||
3. **Streaming availability.** v1 uses a single full-buffer `retrieveInstance`. A
|
||||
future streaming path (step 4) would register frames as bytes arrive.
|
||||
4. **Auth / headers.** Handled — the resolver sets
|
||||
`wadoDicomWebClient.headers = getAuthorizationHeader()` before retrieve, reusing
|
||||
the data source's existing decoration.
|
||||
5. **Per-frame overhead.** Every WADO-RS load now does one cheap
|
||||
`COMPRESSED_FRAME_DATA` lookup (a Map get after base normalization) before
|
||||
falling through. Negligible, and only short-circuits when an instance was
|
||||
actually prefetched.
|
||||
|
||||
## Incremental rollout
|
||||
|
||||
1. ✅ `prefetchPart10Instance` adapter + WADO-RS registry read
|
||||
(`wadors/loadImageFromRegistry.ts`) in dicom-image-loader. No behaviour change
|
||||
until something registers an instance.
|
||||
2. ✅ `retrieve.prefetchInstanceFrames` in DicomWebDataSource (full-buffer v1 via
|
||||
`retrieveInstance`); whether to call it is the caller's policy.
|
||||
3. ✅ SEG handler call site. Now **enabled and awaited to completion by
|
||||
default** (no timeout); per-frame loading requires an explicit
|
||||
`loadMultiframeAsPart10: false`.
|
||||
4. ⏳ Progressive (streaming) parse to register frames as they arrive (handle the
|
||||
`multipart/related` unwrap + boundary-straddle directly).
|
||||
5. ⏳ Extend to other multiframe loaders (the registry path is generic).
|
||||
@@ -232,3 +232,122 @@ integration must trigger loading explicitly (for example by calling
|
||||
`customizationService.applyCustomizationUrlSearchParams` or `customizationService.requires` with
|
||||
the new list). New module keys not seen before can still be loaded that way; unloading or
|
||||
replacing an already-loaded pack is not supported out of the box.
|
||||
|
||||
## `segmentation.store.*` (DICOM SEG export encoding)
|
||||
|
||||
These customizations control how the `@ohif/extension-cornerstone-dicom-seg` extension
|
||||
encodes a DICOM SEG when it is **stored or downloaded**, and are read when a SEG is
|
||||
generated.
|
||||
|
||||
| Key | Values | Default | Controls |
|
||||
| --- | --- | --- | --- |
|
||||
| `segmentation.store.defaultMode` | `'labelmap'` \| `'bitmap'` | `'labelmap'` | SEG SOP Class |
|
||||
| `segmentation.store.transferSyntaxUID` | a transfer-syntax UID string | RLE Lossless (`1.2.840.10008.1.2.5`) | PixelData encoding |
|
||||
|
||||
- **`segmentation.store.defaultMode`**
|
||||
- `'labelmap'` → Label Map Segmentation Storage (`1.2.840.10008.5.1.4.1.1.66.7`). One
|
||||
multi-valued frame per slice. Added to DICOM in 2024, so many PACS/viewers do not
|
||||
accept it yet. **This is the OHIF default.**
|
||||
- `'bitmap'` → (binary) Segmentation Storage (`1.2.840.10008.5.1.4.1.1.66.4`). One frame
|
||||
per segment; broadly compatible with existing back ends and viewers. Opt in via
|
||||
customization when needed.
|
||||
- **`segmentation.store.transferSyntaxUID`**
|
||||
- Default → RLE Lossless (`1.2.840.10008.1.2.5`, compressed). Set by
|
||||
`getSegmentationSaveOptions` in `@ohif/extension-cornerstone-dicom-seg` and registered
|
||||
as an extension customization — no app config required.
|
||||
- `1.2.840.10008.1.2.1` → Explicit VR Little Endian (**uncompressed**). Opt in via
|
||||
customization for back ends that reject compressed SEG PixelData.
|
||||
|
||||
> OHIF's effective default is **Label Map + RLE Lossless (compressed)**. Customizations
|
||||
> (or per-data-source `configuration.segmentation.store`) are only needed to switch to
|
||||
> bitmap and/or uncompressed.
|
||||
|
||||
### URL-loaded files (recommended)
|
||||
|
||||
Two ready-made public customization files ship under
|
||||
`platform/app/public/customizations/segmentation/`, so a user can flip either default
|
||||
straight from the URL (requires `customizationUrlPrefixes.default` to be configured — it
|
||||
is in the `dev` / `netlify` configs):
|
||||
|
||||
| File | `?customization=` value | Effect |
|
||||
| --- | --- | --- |
|
||||
| `segmentation/uncompressed.jsonc` | `segmentation/uncompressed` | Explicit VR Little Endian instead of RLE |
|
||||
| `segmentation/binary.jsonc` | `segmentation/binary` | Binary SEG (66.4) instead of Label Map (66.7) |
|
||||
|
||||
They are independent and combine, so you can apply both at once:
|
||||
|
||||
```
|
||||
http://host/viewer?customization=segmentation/uncompressed&customization=segmentation/binary
|
||||
```
|
||||
|
||||
Each file just `$set`s one key in the `global` phase, e.g. `segmentation/binary.jsonc`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"global": {
|
||||
"segmentation.store.defaultMode": { "$set": "bitmap" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The remaining forms below set the same keys directly in `window.config` instead of via the
|
||||
URL.
|
||||
|
||||
### Store an uncompressed SEG instead of the compressed default
|
||||
|
||||
```js
|
||||
window.config = {
|
||||
customizationService: [
|
||||
{
|
||||
'segmentation.store.transferSyntaxUID': {
|
||||
$set: '1.2.840.10008.1.2.1', // Explicit VR Little Endian (uncompressed)
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Store a binary SEG (66.4) instead of the default Label Map (66.7)
|
||||
|
||||
```js
|
||||
window.config = {
|
||||
customizationService: [
|
||||
{
|
||||
'segmentation.store.defaultMode': { $set: 'bitmap' },
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Per data source override
|
||||
|
||||
A data source may override the app-wide default for one back end only, under
|
||||
`configuration.segmentation.store`. **The data source value wins over the customization
|
||||
default**, so different back ends can pick the SEG encoding they support:
|
||||
|
||||
```js
|
||||
window.config = {
|
||||
// App-wide default (optional): Label Map + RLE unless set here.
|
||||
customizationService: [
|
||||
{ 'segmentation.store.defaultMode': { $set: 'labelmap' } },
|
||||
],
|
||||
dataSources: [
|
||||
{
|
||||
namespace: '@ohif/extension-default.dataSourcesModule.dicomweb',
|
||||
sourceName: 'myPacs',
|
||||
configuration: {
|
||||
// ...wado/qido/stow roots...
|
||||
segmentation: {
|
||||
store: {
|
||||
defaultMode: 'bitmap', // this PACS only accepts binary SEG
|
||||
transferSyntaxUID: '1.2.840.10008.1.2.1', // and only uncompressed
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
When storing, the override is resolved against the data source being written to; for
|
||||
download (no target data source) the active data source's override applies.
|
||||
Reference in new issue
Block a user