OpenDCX

Deep Compositing Extended

OpenDCX 2.2.2 documentation

DeepPixel

DeepMetadata

class DeepMetadata

Stores extra information about a DeepSegment. For the moment it only stores the subpixel mask and the flags, but this could be made to hold arbitrary attributes.

These values are packed & unpacked via float & half channels for file IO.

TODO: support arbitrary attributes?

DeepMetadata::DeepMetadata()

Default constructor leaves junk in vars.

DeepMetadata::DeepMetadata(const SpMask8 &_mask, const DeepFlags &_flags)

Constructor

bool DeepMetadata::isThin() const

Is Z-depth of segment zero (thin) or non-zero (thick)?

bool DeepMetadata::isHardSurface() const

Does the segment represent a solid (hard) or volumetric surface? If hard-surface - use linear interpolation between Zf/Zb. If volumetric - use log interpolation between Zf/Zb.

bool DeepMetadata::isMatte() const

Should the segment cutout (act as a matte) the segments behind it?

bool DeepMetadata::isAdditive() const

Sample should be added to surrounding samples rather than under-ed This is used primarily for partial subpixel-coverage.

uint32_t DeepMetadata::surfaceFlags() const

Does the sample have partial subpixel coverage baked into color, alpha, etc? If so the sample is handled as additive when composited. This normally indicates filtering or resampling of subpixel masks has been applied.

void DeepMetadata::printFlags(std::ostream&) const

Print the list of enabled flags

DeepSegment

class DeepSegment

A single deep sample describing a linear segment of Z-space where Zf <= Zb.

The color channel data describes the values at Zb, so finding values between Zf and Zb requires linear or log interpolation depending on the interpolation-type flag in the metadata.

Note that this class does not store the actual channel data so that the adding, deleting, sorting, etc of DeepSegments is lightweight and fast.

class Edge
struct DeepSegment::Edge
Two of these are created for each DeepSegment to track the active segments during merging routines.
DeepSegment()

Default constructor leaves junk in vars.

DeepSegment(float _Zf, float _Zb, int _index = -1, const DeepMetadata &_metadata = DeepMetadata(SpMask8::fullCoverage, DeepFlags::ALL_BITS_OFF))

Constructor If Zf > Zb, Zf is clamped to Zb.

void setDepths(float _Zf, float _Zb)

Set both Zf & Zb - checks for bad Z’s and tries to find reasonable solution...

bool operator<(const DeepSegment&) const

Used by the sort routine

const DeepFlags &flags() const

DeepFlags metadata access

bool isHardSurface() const

Does the segment represent a solid (hard) or volumetric surface? If hard-surface use linear interpolation between Zf/Zb. If volumetric use log interpolation between Zf/Zb.

bool isThin() const

Is Z-depth of segment zero (thin) or non-zero (thick)?

bool isMatte() const

Should the segment cutout (act as a matte) the segments behind it?

bool isAdditive() const

Sample should be added to surrounding samples rather than under-ed

bool hasPartialSpCoverage() const

Does the sample have partial subpixel coverage baked-in to color, alpha, etc values? If so the sample is handled as additive when composited. This normally indicates filtering or resampling of subpixel masks has been applied.

SpMask8 &spMask()

Subpixel mask access

bool zeroCoverage() const

Are all subpixel mask bits off or on?

If all are OFF this usually indicates a ‘legacy’ deep sample containing no subpixel mask data which is interpreted as volumetric.

If all bits are ON this can simplify and speed up sample operations, especially compositing (flattening) as iterating through subpixels is uneccessary.

bool maskBitsEnabled(const SpMask8 &check_bits) const

Return true if specific bits are enabled in the subpixel mask. Note that all-bits-off is considered all-bits-on for these tests.

float getCoverage() const

Convert the subpixel mask to coverage weight. This is the same as float(spMask8BitsOn(metadata.mask))/64.0f Note that zero-bits-on is considered all-bits-on for this.

void applyMask(const ChannelSet &channels, Pixelf &color)

Apply the subpixel mask weight (coverage) to the channels

void interpolate(const Pixelf &in, float t, const ChannelSet &do_channels, Pixelf &out) const

Sample interpolation Do linear or log interpolation at position t (0-1) within Zf-Zb, depending on interpolation-type flag.

The input pixel represents the value at Zf. Interpolation is performed on the intersection of ‘channels’ and the input pixel’s ChannelSet.

The output pixel is a copy of the input pixel with changes to only the interpolated channels.

static void interpolateLog(const Pixelf &in, float t, const ChannelSet &do_channels, Pixelf &out)

Logarithmic sample interpolation Do log interpolation at position t (0-1) within Zf-Zb.

Note that this uses the density math from the OpenEXR deep docs translated to ChannelSet-loop use.

The input pixel represents the value at Zf. Interpolation is performed on the intersection of ‘channels’ and the input pixel’s ChannelSet.

The output pixel is a copy of the input pixel with changes to only the interpolated channels.

static void interpolateLin(const Pixelf &in, float t, const ChannelSet &do_channels, Pixelf &out)

Linear sample interpolation Do linear interpolation at position t (0-1) within Zf-Zb.

The input pixel represents the value at Zf. Interpolation is performed on the intersection of ‘channels’ and the input pixel’s ChannelSet.

The output pixel is a copy of the input pixel with changes to only the interpolated channels.

static void mergeLog(const Pixelf &A, const ChannelSet &channels, Pixelf &B)

Log-merge two samples. Volumetric log merging math from Florian’s deep doc adapted for channel loops. ‘A’ is the first merge source ‘B’ is the second merge source and the output (it’s modified)

void printInfo(std::ostream&, bool show_mask = true) const

Print info about DeepSegment to output stream

DeepPixel

class DeepPixel

Contains a series of DeepSegments and their associated Pixels (containing float channel data,) which combined together comprise a series of deep samples.

Supports metadata storage for each deep sample and offers methods to aid the manipulation of samples within the deep pixel, including compositing (flattening,) and sorting. Because a DeepSegment is lightweight the list can be rearranged and sorted very quickly. The list of large Pixel channel data structures is kept static.

TODO: investigate cost of using varying-sized Pixel channel array
TODO: extend flatten methods to accept near/far depth range to flatten within
enum class InterpolationMode

Segment interpolation modes for flattening operations

class CombineContext

struct DeepPixel::CombineContext

Threshold values for combining similar segments together, passed to the DeepPixel combiner methods. If color values are within the +/- threshold range the values are considered matched.

color_threshold is used for all color channels - default is 0.001 id_threshold is used for ID channels - default is 0.495

TODO: change color_threshold to a Pixel so that each channel can
be thresholded individually? Non-color channels may require larger values than color channels...
DeepPixel::DeepPixel(const ChannelSet &channels)

Constructors

const ChannelSet &DeepPixel::channels() const
Read-only ChannelSet access
This ChannelSet is shared between all DeepSegments.
void DeepPixel::setChannels(const ChannelSet &c)
Assign a ChannelSet
This ChannelSet is shared between all DeepSegments.
int DeepPixel::x() const
Get/set the xy location
Currently used for debugging
void DeepPixel::transformDepths(float translate, float scale, bool reverse = false)

Transform the Zf & Zb coords for all segments.

void DeepPixel::clear()

Empty the segment list and clear most shared values. The shared ChannelSet is unaffected.

bool DeepPixel::empty() const

DeepSegment list management

DeepSegment &DeepPixel::operator[](size_t segment)

DeepSegment Handling

SpMask8 DeepPixel::getAccumOrMask() const

Accumulated spmask and flags for all segments

void DeepPixel::findNearestMatches(const DeepSegment &segment, float depth_threshold, std::vector<int> &matched_segments_list)

Segment Searches

DeepPixel &DeepPixel::operator+=(float val)

Arithmetic ops Note that there are no *, /, -, + operators to avoid the high cost of constructing & destroying DeepPixels. Try to use these in-place modifiers.

const Pixelf &DeepPixel::getPixel(size_t pixel_index) const

Read/Write DeepSegment Pixel access

void DeepPixel::getSegmentPixelWithMetadata(size_t segment_index, Pixelf &out) const

These Pixel read methods also copy metadata values from the DeepSegment into the output Pixel’s predefined channels: Chan_ZFront, Chan_ZBack, Chan_SpBits1, Chan_SpBits2, Chan_DeepFlags and Chan_SpCoverage.

float DeepPixel::getChannel(size_t segment, ChannelIdx z) const

Get a Pixel sampled at depth Z, possibly using interpolation. Z is clamped between Zf-Zb - no extrapolation is performed. (TODO: implement!)

size_t DeepPixel::buildSegmentEdges(const SpMask8 &spmask, SegmentEdgeList &segment_set)

Build / update SegmentEdgeSet Only segments with enabled sp bits in ‘spmask’ will be added to the edge set. Returns the number of built edges.

void DeepPixel::flatten(const ChannelSet &out_channels, Pixelf &out, InterpolationMode interpolation = INTERP_AUTO)

Flattening (segment compositing)

Flatten the DeepSegments down into an output Pixel, determining whether segment overlaps exist. All subpixels are flattened separately and the final output pixel is the weighted accumulation of all subpixel results. If the subpixel mask for all segments are full-coverage then only one flatten operation is performed.

‘interpolation’ determines the per-segment interpolation behavior - default is INTERP_AUTO.

Calls either flattenNoOverlaps() or flattenOverlapping() depending on overlap status.

DeepSegments are depth-sorted and composited in front-to-back order. Each segment is composited with the previous using an UNDER or PLUS operation depending on whether the segment is flagged as additive.

void DeepPixel::flattenSubpixels(const ChannelSet &out_channels, Pixelf &out, const SpMask8 &spmask, InterpolationMode interpolation = INTERP_AUTO)

Flattening (segment compositing)

Flatten only the DeepSegments that have enabled subpixels matching the input ‘spmask’ subpixel mask into an output pixel. Segment overlaps are determined and either flattenNoOverlaps() or flattenOverlapping() is called.

‘interpolation’ determines the per-segment interpolation behavior - default is INTERP_AUTO.

DeepSegments are depth-sorted and composited in front-to-back order. Each segment is composited with the previous using an UNDER or PLUS operation depending on whether the segment is flagged as additive.

void DeepPixel::flattenNoOverlaps(const ChannelSet &out_channels, Pixelf &out, const SpMask8 &spmask)

Flattening (segment compositing) with no overlap handling

Flatten the DeepSegments for all enabled subpixels in ‘spmask’ into an output Pixel.

DeepSegments are depth-sorted and composited in front-to-back order. Each segment is composited with the previous using an UNDER or PLUS operation depending on whether the segment is flagged as additive.

void DeepPixel::flattenOverlapping(const ChannelSet &out_channels, Pixelf &out, const SpMask8 &spmask, InterpolationMode interpolation = INTERP_AUTO)

Flattening (segment compositing)

‘interpolation’ determines the per-segment interpolation behavior - default is INTERP_AUTO.

DeepSegments are depth-sorted and composited in front-to-back order. Each segment is composited with the previous using an UNDER or PLUS operation depending on whether the segment is flagged as additive.

void DeepPixel::flattenOverlappingLegacy(const ChannelSet &out_channels, Pixelf &out)

Flattening (segment compositing) Use legacy flattening math - has matte support, but no spmask or interpolation-mode support.

void DeepPixel::compositeSegmentUnder(Pixelf &B, const DeepSegment &segment, const ChannelSet &color_channels)

Segment compositing

Composite segment ‘A’ underneath accumulation pixel ‘B’. Normally used for front-to-back flattening of non-overlapping segments. The ChannelSet should contain only the color channels to composite - accumulation alpha/viz/depth and cutout alpha are handled explicitly.

Before calling this method on the first segment, initialize the color and accumulation channels in B like so:

B.erase(comp_color_channels);
B[Chan_A         ] = 0.0f;
B[Chan_ACutout   ] = 0.0f;
B[Chan_Visibility] = 1.0f;
B[Chan_SpCoverage] = 0.0f;
B[Chan_ZFront    ] =  INFINITYf;
B[Chan_ZBack     ] = -INFINITYf;

After the method runs they will contain:

B[Chan_A         ] = Accumulated alpha
B[Chan_ACutout   ] = Accumulated alpha with cutouts (holes) applied
B[Chan_Visibility] = Accumulated visibility (1 - B[Chan_A]),
                     updated when accumulated subpixel
                     coverage weight is >= 1.0.
B[Chan_SpCoverage] = Accumulated subpixel coverage weight
B[Chan_ZFront    ] = Minimum Chan_ZFront depth
B[Chan_ZBack     ] = Maximum Chan_ZBack depth
void DeepPixel::mergeSectionLegacy(const SegmentEdgeSet &segment_set, double Zf, double Zb, const ChannelSet &color_channels, Pixelf &out, DeepMetadata &merged_metadata)

Legacy segment merging

Segment ‘subsegment’ contains the Zf-Zb depth range to merge which must fall within the min/max depth range of the segment set - no checks are performed to verify this!

void DeepPixel::extractSubSectionLog(const DeepSegment &segment, double Zf, double Zb, const ChannelSet &interpolate_channels, Pixelf &out)

Extract the color from a segment subsection.

The output pixel is a copy of the segment pixel with changes to only the interpolated channels.

void DeepPixel::printInfo(std::ostream&, const char *prefix, int padding = 2, bool show_mask = true)

Print info about DeepPixel to output stream