PeacoQC¶
A PeacoQC node checks the quality of each well's acquisition. A clog or a change of flow speed during a run disturbs the measurement for a stretch of it; the node follows the peaks of its channels through the acquisition and rejects the stretches where they move. It can also remove margin events and doublets first. The result is two gates, PQC good and PQC rejected, which share out between them every event the node works on.
The method is PeacoQC (Emmaneel et al., Cytometry A, 2022). PlatAnJr carries the numerical filters of the R package by Annelies Emmaneel and the Saeys lab, version 1.23.0, in its own code, with the package's defaults: R does not have to be installed.
Create one with PeacoQC on the project panel toolbar. Nothing runs until you press Run QC.
What it works on¶
- The events reaching the node, in the wells switched on when it runs. With no input link that is every event of the data, as R's PeacoQC takes a whole file; with input links, the events in any of their gates. The wells switched on are those selected on the heat map - all of them while none is selected - less any disabled well. The selection only picks the wells a run checks: selecting wells afterwards leaves the result as it is (see Wells not checked).
- One well at a time. Each well is a sample of its own, as each file is in R, and its events are taken in the order the file holds them - the order they were acquired. The node does not read the Time parameter. Events that belong to no well (all of them when the data has no wells) make one more sample, called Sample.
- Two kinds of values. The stability check reads the QC channels as PlatAnJr plots them: compensated or unmixed, and through each channel's scale. The margin and doublet filters read the raw values the file holds - linear, before compensation and before any scale - so they work only on the file's own channels, not on parameters PlatAnJr has computed.
The data is never changed: the node adds no parameter and deletes no event. It only sorts the events into its two gates.
Before you run it¶
- Compensate or unmix the data (Compensation, Spectral unmixing) and give every QC channel the scale you mean to keep. R's PeacoQC, too, wants compensated and transformed data. Changing the compensation or a QC channel's scale afterwards makes the result stale.
- Put it first. R runs PeacoQC on whole files, before any gating, and goes on with the cleaned file. The node does the same with no input: hang the rest of the analysis - debris, doublets, live cells, populations - off PQC good. Anything that changes the node's input, an edited upstream gate included, calls for another run.
- Try the settings on a few wells. Select them on the heat map, run, and look at the timeline; then clear the selection and Run QC again. With the settings as they were, it checks only the wells not checked yet; after a change of settings, the whole plate.
Not in preview mode
The node judges the events that were loaded. In preview mode that is a thinned-out acquisition - with Preview Max Cells/Well, only its start, so a clog later in the run is never seen. Release Preview mode and reopen the plate before you rely on the result.
The controls¶
Along the top of the node:
| Control | Effect |
|---|---|
| Channels | The QC channels the stability check follows, picked in the parameter picker. Use compensated, transformed channels, and leave out Time and event counters. A new node starts with the unmixed parameters - or, on data without any, every parameter but Time and EventCount |
| Settings | Bin size, thresholds and the doublet channels - see Settings |
| Run QC | Runs the QC on every well with events reaching the node and updates the two gates - see Running it. It reads Stop QC while a run goes on |
| Mode box | What rejects events - see below |
Under them:
| Control | Effect |
|---|---|
| Ignore zeros in peaks | Leaves zeros out of the peak search, as R's remove_zeros, which R recommends for mass cytometry. Zeros are not rejected for being zeros |
| Remove margins | Removes margin events before anything else |
| Margin channels | The channels checked for margin events. Only with Remove margins ticked; its tooltip lists the channels in use |
| Remove doublets | Removes doublets after the margin events |
Below those are the status line, the well timeline and the report.
What rejects events¶
| Choice | Rejects | In R (determine_good_cells) |
|---|---|---|
| Isolation tree + MAD (default) | The bins the isolation tree sets apart, then those the MAD (median absolute deviation) test finds unstable | "all" |
| Isolation tree | The isolation tree's bins only | "IT" |
| MAD | The MAD's bins only | "MAD" |
| Peaks only | Nothing for stability: the node finds the peaks and trends and reports them. The margin and doublet filters still apply | FALSE |
In the first three, a stretch of good bins too short to trust is rejected as well - see How the stability check works.
Settings¶
Settings opens one dialog for all of them. The defaults are R's.
| Field | Default | Meaning | In R |
|---|---|---|---|
| Events per bin (0 = automatic) | 0 | The events in a bin. Automatic sizes the bins from the well's events and the three automatic settings below - see How many bins | events_per_bin |
| MAD multiplier | 6 | A bin is rejected when a smoothed peak trace lies more than this many MADs from its median. Higher is less strict | MAD |
| Isolation-tree gain (0..1) | 0.6 | A split of the isolation tree must cut a trace's spread by more than this fraction. Higher is less strict | IT_limit |
| Minimum consecutive good bins | 5 | A run of good bins shorter than this is rejected too | consecutive_bins |
| Minimum bins for isolation tree | 150 | With fewer bins the isolation tree is not run | force_IT |
| Minimum relative peak height (0..<1) | 1/3, shown as 0.333333333333333 | A density maximum counts as a peak when it is higher than this fraction of the highest one | peak_removal |
| Doublet area channel | FSC-A |
The area channel of the doublet filter | RemoveDoublets' channel1 |
| Doublet height channel | FSC-H |
Its height channel | channel2 |
| Doublet MAD multiplier | 4 | An event is a doublet when its area/height ratio is this many MADs or more above the median ratio | nmad |
| Doublet height offset | 0 | Added to the height in that ratio | b |
| Minimum cells per bin | 150 | The least an automatic bin holds: this many events and, with Ignore zeros in peaks, on average this many non-zero values of the sparsest channel | min_cells |
| Maximum bins (automatic) | 500 | Automatic bins are made large enough that there are no more than this many | max_bins |
| Bin-size step (automatic) | 500 | Automatic bin sizes are multiples of this | step |
| Peak frequency percent (0..<100) | 10 | A channel's usual number of peaks is the largest number found in more than this percentage of the bins | min_nr_bins_peakdetection |
A value out of range is refused with a message, and the settings stay as they were. A new node starts from the defaults.
Mass cytometry
R's PeacoQC suggests, for mass cytometry: tick Ignore zeros in peaks, transform the channels with arcsinh, cofactor 5, and keep Isolation-tree gain between 0.6 and 0.65. Margin events and compensation it does not need.
Margin events¶
Margin events sit at the ends of a detector's range, where the value says only that the
signal was off scale. Remove margins removes them first, in the Margin channels
only and on their raw values, as R's RemoveMargins does with the ranges flowCore's
read.FCS gives it. An event goes when, in any margin channel, it lies
- at or below
min(0, max(-111, m)),mbeing the channel's lowest raw value in the well - over all the well's events, not only those reaching the node. -111 is flowCore's own floor; - at or above
$PnR - 1, the top of the channel's range in the well's FCS header.
As in R, each limit is first clipped to the lowest and highest value among the events checked, so the events at the very bottom and the very top always go - a few even from a well with no pile-up at all.
Margin channels start as the raw scatter area channels: every parameter whose name
starts with FSC or SSC and ends in -A (FSC-A, SSC-A, SSC-B-A). Only the
file's own channels can be picked. Keep the fluorescence channels out: their raw values
are often negative, and on spectral data margins on them remove most of the events -
over 90%, with R's limits too.
Doublets¶
Remove doublets works on the events the margin filter kept, on the raw values of the
Doublet area channel and Doublet height channel - FSC-A and FSC-H unless you
change them in Settings. Two cells passing together make a pulse with more
area for its height than one cell does. Each event's ratio
area / (1e-10 + height + offset) is taken, and an event is removed when its ratio is at
or above the well's median ratio plus Doublet MAD multiplier times the ratio's MAD
(R's stats::mad, scaled by 1.4826). Only the high side is cut, as in R's
RemoveDoublets. Both channels must be the file's own.
How the stability check works¶
On the events the margin and doublet filters kept, well by well:
- Bins. The events are cut, in acquisition order, into bins of Events per bin events, each bin starting half a bin after the one before, so that most events are in two bins.
- Trends. Each QC channel's median in every bin is smoothed as R's
ksmoothdoes (box kernel, bandwidth 50 bins). A channel whose smoothed median reaches a new high in more than three quarters of the bins is reported as an Increasing trend, new lows as a Decreasing trend. Trends are only reported: they reject nothing. - Peaks. In every bin, each QC channel's density is estimated as R's
densitydoes (Gaussian kernel, 512 points, R's default bandwidth) and its peaks are found: the local maxima higher than Minimum relative peak height times the highest. The channel's usual number of peaks is the largest number found in more than Peak frequency percent of the bins, and each bin's peaks are matched to the nearest of that many typical positions. That makes peak traces: where one peak lies, bin after bin. A trace found in fewer than half the bins is dropped; where a bin lacks its peak, the trace's median stands in. - Isolation tree. The bins are split, again and again, on whichever trace and value cut a trace's spread most, as long as that gain - the drop in its standard deviation, as a fraction of it - beats Isolation-tree gain. Each split then sets the bar for the ones after it. The largest group of bins left at the end is kept, and every other bin is rejected. With fewer bins than Minimum bins for isolation tree it is skipped. This is PeacoQC's tree, not an isolation forest: nothing in it is random, and the same data always gives the same result.
- MAD. Each trace, over the bins still kept, is smoothed with R's
smooth.spline(spar0.5). A bin is rejected when, in any trace, its smoothed value lies more than MAD multiplier MADs from the trace's median. With fewer than four bins left it is skipped, with a warning. - Short stretches. A run of good bins shorter than Minimum consecutive good bins is rejected too, wherever it is - between rejected bins or at either end of the well.
An event is rejected when a bin it is in is rejected, even if the other bin it is in was kept.
How many bins¶
Events per bin at 0 makes the bins twice the events over Maximum bins, rounded up to the next multiple of Bin-size step - to the next one even when it already is a multiple - and no smaller than Minimum cells per bin. With Ignore zeros in peaks, a channel with many zeros makes the bins larger still.
With the default settings, what a well gets depends on how many of its events reach the stability check:
| Events | Bins | What happens |
|---|---|---|
| Up to 750 | 1 to 3, of 500 events | Too few: stability is not assessed and every event is kept; the report says so |
| 751 to 1,000 | 4, of 500 | Every event is rejected: the well's one run of good bins is shorter than Minimum consecutive good bins, as in R |
| 1,001 to 37,250 | 5 to 149, of 500 | MAD only: too few bins for the isolation tree |
| 37,251 to 124,750 | 150 to 499, of 500 | Isolation tree and MAD |
| More | 250 to 500, of 1,000 or more - 445 of 4,500 at a million events | Isolation tree and MAD |
Running it¶
Run QC checks the channels and settings, then works through the wells one after
another in the background, with a progress window naming the well, how far through the
wells it is (A3 (3/12)) and the stage. You can go on working meanwhile.
The wells it works through are the wells switched on that the result does not cover yet: the first time, every well switched on. When the result already covers them all, it checks them all again. Either way the wells it does not check keep their result: see Wells not checked.
Cancel in the progress window, Esc there or closing it, or Stop QC on the node stops the run, and so does a change of the node's settings. A run that stops or fails never becomes a result: the gates keep the previous one, if it is still valid. If the data or the node's input changes before the run is over, the run is discarded and the status line asks you to run it again. Selecting wells on the heat map meanwhile does not matter.
Reading the result¶
The status line¶
| Status | Meaning |
|---|---|
| Choose channels, then Run QC | A new node |
| PeacoQC is running... | A run is going on |
68738 / 71500 retained; 3.86% rejected in 2 sample(s) |
The result: the events kept, of those reaching the node in the wells checked; the share rejected; the wells (samples) checked |
...; not checked: B3, B4 - Run QC |
Wells switched on now, with events reaching the node, that the result does not cover; past four, ...; 12 wells not checked - Run QC. See Wells not checked |
| Settings changed - Run QC | See When it has to run again |
| Input or data changed - Run QC | The same |
| QC cancelled | The run was stopped |
| QC failed: and a reason | The run hit an error - see When it will not run |
| Saved by an earlier PeacoQC node - Run QC | A project saved by the node's first version: its settings are kept, its result is not |
The well timeline¶
One row per well, named at its left: a bar from the well's first event to its last, in the order they were acquired, as long for every well whatever its number of events. It is green where the events reaching the node were retained, red where any was rejected, and grey where none reached the node. A well the result does not cover is grey and says not checked. The rows share the box while they fit; with more wells each keeps its height and the box scrolls, so you can scroll to every well.
The report¶
The report under the timeline lists every well checked, in plate order, after the channels used. For example:
Channels: CD3, CD4, CD8, CD19
Margin channels (raw values): FSC-A, SSC-A
Green: retained; red: rejected; grey: outside the input or not checked. Acquisition order, per well.
A1: 47506 / 50000 retained (4.99% rejected)
Margins 14; doublets 1730; stability 1.55% of 48256 prefiltered events.
194 bins, 500 events/bin; IT 1.55%, MAD 0.00%, short regions 0.00% (overlap possible).
Increasing trend: CD8
A2: 21232 / 21500 retained (1.25% rejected)
Margins 4; doublets 264; stability 0.00% of 21232 prefiltered events.
85 bins, 500 events/bin; IT 0.00%, MAD 0.00%, short regions 0.00% (overlap possible).
Isolation tree was not run.
Too few bins for the isolation tree
For each well: the events kept of those reaching the node; how many the margin and doublet filters removed, and what share of the rest the stability check rejected; the bins; and what each stage - IT (the isolation tree), MAD and short regions - rejected, as a share of the events after the filters. An event can be in a bin one stage rejected and in the next bin, which another stage rejected, so the stages' shares can add up to more than the total. Then the stages the mode asked for but that could not run, the trends, and the warnings:
| Warning | Meaning |
|---|---|
| Fewer than four bins: signal stability was not assessed | Every event of the well passed the stability check unexamined |
| Too few bins for the isolation tree | Fewer than Minimum bins for isolation tree |
| Fewer than four retained bins: MAD filtering was skipped | The isolation tree left too little for the MAD |
| Channel 2 has no persistent peaks | No peak of that channel - counted along the Channels line - turned up in half the bins, so it played no part |
| No persistent peaks: signal stability was not assessed | The same for every channel; every event passed |
| More than 70% of events were removed | Look at the timeline and the settings before trusting the well |
The two gates¶
The first run adds two output pins to the node. PQC good (green) holds the events retained; PQC rejected (red) holds the rest of the events that reached the node, margin events and doublets included. Events outside the node's input, or in a well the result does not cover, are in neither. They are ordinary gates: they hold every well checked, whatever the heat map has selected, and the charts fed from them show the wells selected, as from any gate. Link PQC good to the nodes that go on with the analysis, and PQC rejected to a chart to see what was thrown away.
To hand the cleaned data to other software, use
Save Gated Community on the node menu and pick
PQC good: a new plate of FCS files holding only the retained events, as R's
save_fcs writes.
When it has to run again¶
A result belongs to the input, the data and the settings it was made from. Change one of them and both gates go empty, the timeline clears, and the status line asks for Run QC:
- Settings changed - Run QC - the channels, a check box, the mode, or the Settings dialog. Ok there or in a picker counts even when nothing was changed; Cancel does not.
- Input or data changed - Run QC - the node's input gates, in the wells the result covers, or the data: another plate, a new compensation, a QC channel's scale, a different plate map.
The nodes fed from either gate then see nothing until the next run, and putting things back as they were does not bring the old result back. Editing a gate elsewhere in the project that does not feed the node leaves the result alone, and so does selecting wells on the heat map.
Wells not checked¶
A run checks the wells switched on when it runs, and the result covers those wells only. Selecting wells afterwards changes nothing in it: both gates keep every well checked, and the charts fed from them show the wells selected, as for any gate. A well switched off after the run keeps its result too.
A well switched on now that the result does not cover - not selected, or switched off,
when the node ran - is in neither gate. Its row in the timeline says not checked,
and if events of it reach the node, the status line names it
(...; not checked: B3 - Run QC). Run QC then checks just the wells switched on
that are not checked and adds them to the result, keeping the wells already checked. So
a plate can be checked a few wells at a time: select them, run, select the next.
The settings, the channels, the result and the report are saved with the project. Reopened on the same data, the node shows its result again without a run, once it has checked a fingerprint of the events it checked, their values, the wells and the margin limits. On other data it asks for a run.
Compared with R¶
The node's arithmetic is a translation of the R package's, and it has been checked against R PeacoQC 1.23.0 running in R 4.6: on synthetic data and on real spectral samples it gives the same bins, trend flags, peak traces, isolation-tree and MAD results and event masks, and the same margin and doublet events.
| In PlatAnJr | In R |
|---|---|
| A PeacoQC node, each well on its own | PeacoQC(), on each file |
| The events reaching the node | The flowFrame given to it |
| Channels | channels |
| Mode box | determine_good_cells |
| Ignore zeros in peaks | remove_zeros |
| Settings | The arguments named in Settings |
| Remove margins, Margin channels | RemoveMargins(ff, channels) on the file as read, with the minRange and maxRange of read.FCS |
| Remove doublets | RemoveDoublets(ff, channel1, channel2, nmad, b) |
| PQC good, PQC rejected | GoodCells, and the cleaned flowFrame FinalFF |
| Save Gated Community of PQC good | save_fcs |
| The timeline and the report | Not ported: R's PlotPeacoQC figures and its report file |
Where the two differ:
- Margins always on raw values. R's vignette has
RemoveMarginsrun before compensation, which changes the ranges it reads. The node always reads the raw values, so the order does not matter. - A gated input. R takes the whole file; the node can take a gate. The lower margin
limit still comes from the whole well, as
read.FCStakes it from the whole file; the clipping to the extremes uses the events checked. - Edge cases. Where R's code would fail or mislead, the node settles them:
- fewer than four bins, or no persistent peaks: the events are kept and the report says stability was not assessed;
- fewer than four bins after the isolation tree: the MAD is skipped, with a warning;
- no number of peaks found in more than Peak frequency percent of the bins: the most frequent number is used, where R stops with an error;
- with Ignore zeros in peaks, a channel with fewer than three non-zero values does not size the bins;
- an exactly constant channel gives a constant trace.
- Exact ties. On heavily tied data two points of a density can be exactly equal, and which is the peak then comes down to rounding: R and PlatAnJr can pick differently.
When it will not run¶
Run QC names the problem in a message - or, once the run has started, in the status line:
| Message | What to do |
|---|---|
| Open cytometry data first | Open a plate |
| Choose at least one QC channel | Pick them with Channels |
| Missing QC channel: and a name | The data has no parameter of that name - a project opened on another plate, say. Pick the channels again |
| No events reach this node in the enabled wells | The input gates hold no events in the wells switched on |
| Choose the margin channels (raw scatter channels such as FSC-A and SSC-A) | Pick them with Margin channels |
| Missing margin channel: and a name | The data has no such parameter. Pick them again |
| Margin removal needs original detector channels: and a name | A computed parameter is among the margin channels. Pick the file's own |
| Choose existing doublet area and height channels | The data has no FSC-A or FSC-H, or whatever Settings names. Name the right ones |
| Doublet removal needs original detector channels | A computed parameter is a doublet channel. Name the file's own |
| The input, data or settings changed during QC; run it again | Something changed while it ran. Run it again |