Skip to content

Dimensionality reduction

The Embedding chart tab projects high-dimensional data onto two axes, so populations that no single pair of parameters separates can be seen at once. PlatAnJr offers four methods, each with its own sub-tab, plus an All together tab showing them side by side.

A population rather than every event

The Embedding tab embeds every event. To embed a gated population - and still give every event a place - use an embedding node in the gating graph.

Choosing the input

The parameter checklist on the Embedding tab decides what the methods see: check the channels that take part. At least three are required - with two you would not be reducing anything. PCA, tSNE and UMAP use every event; the active node does not restrict them. The autoencoder embeds every event too, but learns from the events of the selected node when there is one.

Clean data first

An embedding of raw data spends most of its resolution separating debris from cells. Since the embeddings take every event, open data that is already cleaned up - or embed, then gate the debris away on the new parameters and embed again.

Preview mode matters here more than anywhere else: these are the slowest operations in the application, and they scale badly with event count. Work out your parameter list on previewed data, then run the final embedding on the full dataset.

PCA

Click PCA. The projection is computed immediately - it is by far the fastest of the four - and PC1 and PC2 are added to the dataset as two linear parameters. They are linear because principal components are centred on zero and run negative, which no log scale can show.

Because they become real parameters, you can plot PC1 against PC2 on an ordinary chart node, gate there, and carry the gate into the rest of the analysis exactly like any other.

PCA is linear, so it will not untangle populations that differ non-linearly - but it costs nothing to run and often shows immediately whether the panel has any structure at all.

tSNE

Click tSNE. A box first asks for the parameter name - the result's two parameters are called name-1 and name-2 - offering tSNE; OK starts the run, Cancel does not. PlatAnJr runs FIt-SNE (FFT-accelerated interpolation-based t-SNE, Linderman et al., 2019) on the checked parameters of every event - the active node does not restrict it - after scaling each parameter to its 0-1 range. The computation runs in the background on all processor cores, so you can keep working while it goes on:

  • A progress window stays on top while it runs. Its bar fills up as it goes - the first quarter is the nearest-neighbour search, the rest the 750 iterations - with the current step above it on the left, and on the right the cost (the KL divergence, which should fall) each time it is evaluated, every 50 iterations. The window closes by itself when the run ends.
  • The log panel keeps the algorithm's messages, cost lines included, so the run can be read back afterwards.
  • To abandon a run, click Cancel in the progress window, press Esc there or close the window - or click the tSNE button, which reads Stop while a run is going on. The run stops within a fraction of a second, whatever stage it is in, the log says tSNE stopped, and the plot is not replaced.
  • When the run finishes, the plot appears, the button reads tSNE again and the log gives the run time.

Like PC1 and PC2, the result is added to the dataset as two linear parameters, name-1 and name-2 - tSNE-1 and tSNE-2 with the name offered. You can plot them on an ordinary chart node, gate there and carry the gate into the rest of the analysis. Running tSNE again under the same name replaces their values rather than adding another pair; under another name it adds a pair of its own. Like all derived parameters, they are discarded when the data is reopened.

Closing PlatAnJr during a run stops the run.

The settings are the FIt-SNE defaults: perplexity 30, 750 iterations (the first 250 with early exaggeration 12), learning rate the larger of 200 and events/12, and a PCA initialisation, so the layout starts from the data's two main directions rather than from noise.

  • tSNE needs more than 90 events (three times the perplexity); with fewer it says so and does not start.
  • If you open other data while tSNE is running, the result is discarded when it arrives instead of being drawn over the wrong events.
  • Run time grows roughly linearly with the event count, and also with how far apart the clusters end up. Try the parameter list on previewed data first.

Running it on the graphics card

Use GPU for tSNE - on the Options tab, Data sub-tab, under Show AE Axes - moves the iterations onto an NVIDIA graphics card. The setting is remembered. Where there is no CUDA card the box is disabled, and its tooltip names the card where there is one.

It is the same method and the same settings - only the arithmetic moves - so the picture is the one the processor would have drawn, give or take the last digits: the runs above ended within a few percent of the same cost. The log says FIt-SNE (GPU) when the card is in use, and so does the progress window's title. Everything else is unchanged: the progress, Stop, and the two parameters.

What it buys you is the iterations, which are tens of times faster on the card - measured here, 3 ms against 154 ms each on 180000 events. The nearest-neighbour search runs on the processor either way, though, and on large files that is most of the wait: 180000 events took 25 s against 39 s, while a small file finished about ten times sooner.

The card works in single precision and needs roughly a kilobyte of its memory per event (mostly the neighbour lists, at the default perplexity), plus up to half a gigabyte for the interpolation grid once the embedding spreads out. A card that cannot fit the data fails the run with a message, and the processor is always there to fall back on.

Files

tSNE needs libfitsne.dll and libfftw3-3.dll in the program folder; the installer puts them there. If they are missing, clicking tSNE says which file could not be loaded. The GPU option needs nothing else installed - no CUDA toolkit - only an NVIDIA card and its driver. FFTW is distributed under the GNU GPL - its licence is installed as fftw_COPYING.txt, and FIt-SNE's as FIt-SNE_LICENSE.txt.

tSNE preserves local neighbourhoods: points near each other in the projection were near each other in the full space. It does not preserve distances between well-separated clusters - the gap between two islands carries no meaning, and neither does their size.

Mapping other data without running tSNE

tSNE has no model: it arranges the events it is given and knows nothing about any others, so a second file cannot simply be dropped into an existing layout. The bar above the tSNE plot builds one:

Train Map trains a small neural network on the result that is on screen - the parameters the run used in, the tSNE coordinates out. That network is a model: applying it to another file gives its events tSNE-like coordinates in a moment, with no tSNE run. It asks for epochs, a batch size and a name - the run's name is offered - and trains like the autoencoder, on the card if Use GPU for Autoencoder is on. It learns from the parameters exactly as the run saw them, through the scales they had then, even if a scale has been changed since. The plot keeps showing the tSNE result while it learns.

Every tenth event is kept back from the training. When it ends, the log says how far those events land from where tSNE itself put them - the honest measure, since the network never learnt from them - as a distance and as a percentage of the layout's width. A few percent means the map follows the layout closely.

Save Map keeps the map, adds it to the list on the left (see Where models are kept) and applies it to the data now open: the plot then comes from the map instead of from the run, and its coordinates are added as two parameters named after it - a map called PBMC gives PBMC-1 and PBMC-2. Saved under the run's name, the map's coordinates replace the run's, so the data then carries exactly what the map gives any other data it is applied to, and a gate drawn on them holds there too. Picking a map from the list applies it the same way to whatever data is open.

Save Map asks for the map's name. A map saved already, under any name, is not saved a second time. A name that a saved map of the method already has is asked about: Yes replaces that map - unless a gate uses it - No asks again with a free name offered, tSNE2, and Cancel saves nothing.

A map knows its parameters by name and keeps the scales they were plotted through - see Saving and reapplying a model. Applied to data that lacks any of them, it says which and changes nothing.

A map is a likeness, not the thing

The network learns a smooth function of the parameters. It reproduces the layout where the training events were, and guesses everywhere else - a population that was not in the file it was trained on will be put somewhere, with confidence and without warning. Use a map to place more of the same, and run tSNE itself when the sample may hold something new.

UMAP

Left-click UMAP to set up a run. The UMAP parameter dialog comes first, with the settings below and a Parameter Name, offering UMAP - the result's parameters are name-1 and name-2, and a map of this run starts with the same name; OK starts the run. PlatAnJr then runs UMAP (McInnes, Healy & Melville, 2018) with umappp, a C++ implementation derived from R's uwot, on the checked parameters of every event, each scaled to its 0-1 range - the same input as tSNE. Right-click the button to load an embedding from a file instead.

A run behaves like a tSNE run:

  • The progress window shows it: the first 30% of the bar is the nearest-neighbour search, the rest the optimisation epochs, with the current step on the left. UMAP has no cheap cost to show, so the right-hand field stays empty.
  • Cancel, Esc in the window, closing the window, or the button - which reads Stop during a run - stops it. The one step that cannot be interrupted is the spectral start, usually well under a second; a cancel during it takes effect when it ends.
  • The result is added to the dataset as two linear parameters, name-1 and name-2 - UMAP-1 and UMAP-2 with the name offered - replaced by the next run under the same name.
  • If you open other data during the run, the result is discarded when it arrives.
  • tSNE and UMAP share the progress window, so only one of them runs at a time.

The dialog's settings are umap-learn's:

Setting What it does
n_neighbors Neighbourhood size, counting the event itself (15). Larger values favour the global layout, smaller ones local detail.
n_epochs Optimisation epochs. 0 chooses: 500 up to 10000 events, fewer for larger files.
learning_rate Step size of the optimisation (1).
min_dist How tightly points may pack in the embedding (0.1). Must not be larger than spread.
spread The scale of the embedded points (1).
set_op_mix_ratio How the neighbourhoods are merged: 1 is a fuzzy union, 0 an intersection.
local_connectivity How many nearest neighbours are always taken as connected (1).
repulsion_strength Weight of the repulsion between non-neighbours (1).
negative_sample_rate Non-neighbours sampled per neighbour in each step (5).
metric euclidean, manhattan, cosine or correlation. The list also offers umap-learn's other metrics; choosing one of those stops with a message.
transform_queue_size, target_weight Ignored: they belong to umap-learn's transform and supervised modes, which this UMAP does not have.

It is fast: the neighbours come from an HNSW index built and searched on all processor cores, and the optimisation runs on all cores at once. Measured here, 120000 events of 10 parameters took about 8 seconds and 6000 events about 2. With more than about 4000 events the parallel optimisation makes two runs with the same settings differ slightly - the picture, not the populations.

Mapping other data without running UMAP

UMAP has no model either. The bar above the UMAP plot works as the tSNE one does - see Mapping other data without running tSNE: Train Map trains a network on the UMAP result on screen, with every tenth event kept back to measure it, Save Map keeps it and applies it to the data now open, and picking a map from the list applies it to whatever data is open. A map starts with the name given in the UMAP dialog for the run.

Autoencoder

An autoencoder is a small neural network that squeezes the checked parameters through a bottleneck of two numbers and learns to rebuild them from those two; the two numbers are the embedding. PlatAnJr's is a contractive autoencoder: it is also penalised for how sharply the two numbers change with the input, which keeps events that are close in the data close on the map. Unlike tSNE and UMAP, the trained network is a model: it can be saved and applied to other data without training again.

Click Autoencoder to train one. The training dialog comes first:

Setting What it does
Epoch Number Passes over the training events (50).
Batch Size Events per training step (2048). Smaller batches make an epoch slower; larger ones make it no faster.
Redraw Every (epochs) How often the plot is redrawn during the training, so you can watch the layout form (10). 0 draws only the result.
Jacobian coeff The weight of the contraction (0.01). 0 trains a plain autoencoder.
Parameter Name The result's parameters are name-1 and name-2, and a saved model keeps the name. AE is offered.

The first four are remembered. The line at the top of the dialog names the events the network learns from: the events of the node selected in the gating graph (Gate: name), or all events when no node is selected. Either way the trained network then embeds every event. Each parameter is scaled to its 0-1 range first, as for tSNE and UMAP.

A training behaves like a tSNE run:

  • The progress window shows the epoch, and on the right the loss: how badly the network rebuilds the parameters, plus the contraction term. It should fall. The parameters are standardised inside the network, so the loss is on a fixed scale - a reconstruction error of 1 would mean the network had learnt nothing. The log panel lists every epoch's loss, and at the end the reconstruction error and contraction term separately.
  • The plot is redrawn every Redraw Every epochs while the network learns.
  • Cancel, Esc in the window, closing the window, or the button - which reads Stop during a training - stops it within one step. The last redrawn plot stays, but the model and its parameters are not replaced.
  • The result is added to the dataset as two linear parameters named after the model - model name-1 and model name-2 (the name given in the dialog) - which the same model replaces when it is trained or applied again.
  • If you open other data during the training, the result is discarded when it arrives.
  • tSNE, UMAP and the autoencoder share the progress window, so only one of them runs at a time.

The network starts from random weights, so two trainings arrange the populations differently.

It runs on the processor's cores with libtorch, the C++ library behind PyTorch. Measured here, 100000 events of 10 parameters took about 1.5 s an epoch - 74 s for the default 50 epochs - and a million events about 14 s an epoch. The time grows with events times epochs, but large files need fewer epochs: an epoch of a million events is as many training steps as ten epochs of 100000. Watch the loss in the log, and train on previewed data or on a gate first. Applying a trained or saved model is quick - 1.4 s for a million events.

Running it on the graphics card

Use GPU for Autoencoder - on the Options tab, Data sub-tab, under Use GPU for tSNE - trains on an NVIDIA card instead of the processor. The setting is remembered, and the progress window's title says Autoencoder (GPU) when the card is in use.

Unlike tSNE's, this option needs an extra package of files, the GPU package - about 1.5 GB of libtorch's CUDA libraries - copied into the program folder beside the others; the installer does not include it. It replaces nothing: take it away again and the autoencoder trains on the processor. The card must be an NVIDIA GTX 16 or RTX 20 series or newer (compute capability 7.5), with a driver of version 580 or newer. Without them the box disables itself the first time you train, the run goes to the processor, and the box's tooltip says what is missing - the package, the card, a newer driver, or a package that goes with this version of the autoencoder.

It is worth having only with a large batch size. The network is small, so most of a training step is spent starting work on the card rather than doing it; measured here on a million events of 10 parameters, against 8 processor cores:

Batch Size Processor Card
2048 14 s an epoch 9.7 s
8192 13 s an epoch 2.9 s
32768 16 s an epoch 0.7 s

A larger batch is fewer optimisation steps per epoch, though, so the loss falls less per epoch: after two epochs of that million, 0.053 at batch 2048 against 0.081 at 32768. On a card, raise both the batch size and the number of epochs - that is where it wins.

Everything else is unchanged: the same network and settings, the same progress, Stop, snapshots and parameters. A model trained on the card is saved in the same form as any other and can be applied on a machine that has no card.

Saving and reapplying a model

Save AE Model keeps the current model: it asks for its name, as Save Map does, and adds it to the list left of the Autoencoder button. Picking a model from that list applies it to the current data - no training - and its coordinates become model name-1 and model name-2. Saved models are kept with the data - see Where models are kept.

A model knows its parameters by name, and keeps the scale each was plotted through when it was trained. Applied to other data, it finds its parameters there by name, in whatever order they come, and puts them through its own scales, whatever that data's are - so the same events get the same coordinates. If the data lacks any of its parameters, a message names them and nothing is changed: the list goes back to the model it showed. This holds for tSNE and UMAP maps too.

Models saved by earlier versions of PlatAnJr know their parameters' names, but the scales they stored are not the right ones; they are applied through the scales of the data they are applied to, as they always were.

Autoencoders trained by earlier versions of PlatAnJr through Python are not listed: nothing in this version can run them.

Show AE Axes (Options ▸ Data) draws each parameter's axis on the autoencoder plot: the path the embedding takes as that parameter alone sweeps its 0-1 range - where on the map each parameter pulls. With a parameter selected, only its axis is drawn.

Files

The autoencoder needs libae.dll and libtorch's torch_cpu.dll, c10.dll, libiomp5md.dll, uv.dll and cupti64_2025.3.0.dll in the program folder, with the Visual C++ run-time (msvcp140.dll, vcruntime140.dll, vcruntime140_1.dll); the installer puts them there. No Python is needed. If a file is missing, clicking Autoencoder says which. The GPU package adds torch_cuda.dll, c10_cuda.dll, cublas64_13.dll, cublasLt64_13.dll, cufft64_12.dll, cusolver64_12.dll, cusparse64_12.dll, nvJitLink_130_0.dll and cudnn64_9.dll.

Where models are kept

A saved model - an autoencoder, or a map of a tSNE or UMAP result - is kept with the data it was saved from, in the INFO\models folder inside that data's folder, one file each. The model lists show the models of the data that is open, and change when other data is opened.

To take a model to other data: the model last trained or picked stays current when other data is opened, and is applied to it - an autoencoder on its plot, a map on the tSNE or UMAP plot if it was showing there. If the new data lacks any of its parameters it is not applied, and the log says which. Save then puts it into the new data's folder as well. A model file can also simply be copied from one INFO\models folder to another.

A model applied to data adds its coordinates as two parameters named after it: model name-1 and model name-2. A tSNE or UMAP run is not a model, but is named the same way, with the name asked for before it starts. Before version 2.34 the two were named with _ - tSNE_1, tSNE_2.

Gating on an embedding

You can draw polygons on the embedding plots themselves:

  • Shift +click to start a polygon, click per vertex, right-click or click the first vertex to close.
  • Ctrl +click a polygon, or Shift +right-click inside it, to delete it.

These polygons only mark regions on that plot. They are not gates, they do not define a population, and running the method again clears them.

To gate on an embedding, use its parameters: plot PC1 against PC2, tSNE-1 against tSNE-2, UMAP-1 against UMAP-2 - or whatever name the run or model was given, name-1 against name-2 - on an ordinary chart node and draw the gate there. It is then a normal gate and carries into the rest of the analysis like any other. A gate on tSNE, UMAP or autoencoder coordinates drawn before a rerun sits on a different layout afterwards, since every run arranges the clusters differently - except when the coordinates come from the same saved model: an autoencoder, or a saved tSNE or UMAP map.

Judging an embedding

Two tools on the Info page:

Embedding Divergence
How much the projection distorts the original distances - the smaller, the more faithfully the picture reflects the data.
Distance Histogram
The distribution of pairwise distances, before and after. If the two distributions look nothing alike, the projection is telling you more about the algorithm than about the cells.

Do not measure on a tSNE or UMAP plot

Cluster sizes, the distances between clusters and the space between islands are artefacts of the layout. Use an embedding to find populations, then define and quantify them on axes that mean something - which is what carrying the gate back onto real parameters achieves.