In the context of large-scale human system immunology studies, controlling for technical and biological variability is crucial to ensure that experimental data support research conclusions. In this study, we report on a universal workflow to evaluate both technical and biological variation in multiparameter flow cytometry, applied to the development of a 10-color panel to identify all major cell populations and T cell subsets in cryopreserved PBMC. Replicate runs from a control donation and comparison of different gating strategies assessed the technical variability associated with each cell population and permitted the calculation of a quality control score. Applying our panel to a large collection of PBMC samples, we found that most cell populations showed low intraindividual variability over time. In contrast, certain subpopulations such as CD56 T cells and Temra CD4 T cells were associated with high interindividual variability. Age but not gender had a significant effect on the frequency of several populations, with a drastic decrease in naive T cells observed in older donors. Ethnicity also influenced a significant proportion of immune cell population frequencies, emphasizing the need to account for these covariates in immune profiling studies. We also exemplify the usefulness of our workflow by identifying a novel cell-subset signature of latent tuberculosis infection. Thus, our study provides a universal workflow to establish and evaluate any flow cytometry panel in systems immunology studies.

Flow cytometry allows rapid and simultaneous qualitative and quantitative analysis of multiple cell populations within a biological sample at the single-cell level (1). With recent technological advances, it is now routinely possible to perform flow cytometric experiments with 10 or more parameters (i.e., multiparameter flow cytometry) in most research infrastructures (2). Multiparameter flow cytometry has proven successful at identifying disease signatures and prognostic markers in response to infection, immunization, or treatment (3, 4), and has led to the discovery of new cell types that contribute to protective immunity, such as polyfunctional T cells (5). Thus, flow cytometry is a key technique for human cellular immunophenotyping studies.

The systems biology approach is increasingly used in human immunology and often involves the analysis of samples from various human cohorts acquired by different researchers and research centers (6). In this context, experimental data must be quality controlled to ensure that differences reflect the biological variables of interest rather than being due to technical variation or biological covariates. Multiparameter flow cytometry results in particular are known to have low reproducibility when not adequately controlled (7, 8). Possible sources of technical variability in flow cytometry are diverse, ranging from differences in sample handling and staining procedures, to differences in assay reagents, data acquisition settings, cell analyzer performances, and data analysis methods (9). The resolution of each cell population, defined by the expression profile of the markers used for phenotyping and their relative abundance within the sample of interest, can also affect technical variability (7, 10). Approaches aimed at evaluating and reducing technical variability in flow cytometry are therefore crucial to ensure that biological differences can be detected in the systems immunology settings.

Several approaches to assess and reduce technical variability in flow cytometry studies have been developed. Strategies to minimize variation in sample handling and staining procedures include the development of standardized protocols, reagents, and flow cytometry panels (7, 9, 11, 12). The implementation of automated instrument set up templates from data acquisition software (i.e., BD Application Settings on BD Diva software) or the use of calibration beads (13) is helpful to reduce technical variation in data acquisition. Another major source of technical variation in flow cytometry is based on the manual gating of cell populations. This can be improved by performing manual gating following a defined standard operating procedure by a centralized invariant operator, or based on automated gating pipelines. Such computational methods are currently being actively developed and benchmarked by the flow informatics community, for instance through the Flow Cytometry: Critical Assessment of Population Identification Methods project (14).

Although these standardizing efforts have tremendous value for studies that can directly reuse the standardized protocols and reagents that have been developed, they cannot be all encompassing as specific research questions often necessitate the use of custom staining panels for which specific standardization efforts will again be necessary. Furthermore, whereas all the previous standardizing efforts have been interested in the measurement of technical variation, none have developed specific metrics to control for it. Thus, the development of a universal workflow that controls for technical variation and could be easily applied to any flow cytometry panel would be of great interest for the clinical human immunology community.

Controlling and correcting for technical variability allows the teasing out of the true biological variation between samples. This in turn enables examination of another important source of variability when assessing cell populations, which is introduced by biological variations in the donor population introduced by covariates such as different geographical locations, gender, and age of the donors. These factors can contribute to large differences between samples and might mask the immunological differences that were targeted by the study, or lead to false positive results. Recent studies have found a significant association between non-genetic factors such as geographic location with the cellular immune composition of human peripheral blood (15). This means it is important to establish procedures to assess the impact of covariates in the population studied on the cell population under examination to separate the effect of these covariates from the biological factor of interest.

For our studies of immune signatures associated with dengue virus and Mycobacterium tuberculosis infection and vaccination as part of the second round of the Human Immunology Project Consortium (HIPC) program (16), we wanted to design a single flow cytometry panel that could quantify naive and memory T cell subsets, as well as delineate most of the major cell types circulating in the peripheral blood. We thus developed a 10-color flow cytometry panel for the quantification of 17 cell populations from cryopreserved PBMCs. Based on the considerations above, we aimed to establish a reusable approach to quantify technical variability and identify biological covariates associated with cell population frequencies. We devised quality control approaches that flag faulty batches of samples before they are used in resource intensive omics assays. We also show that after correction for technical variability, it is possible to determine if cell population frequencies vary substantially as a function of such demographic variables, and discover which do, and that it is possible to either control for them during donor recruitment or correct for their presence in the final data analysis. Our experience has resulted in detailed specific recommendations for researchers interested in studying cell populations covered in our panel. More generally, our study provides a universal workflow that can be applied to establish and evaluate similar flow cytometry panels for systems immunology studies.

Blood samples were obtained from the University of California, San Diego AntiViral Research Center clinic (San Diego, CA), Universidad Peruana Cayetano Heredia (Lima, Peru), and the National Blood Center, Ministry of Health (Colombo, Sri Lanka) under approval from the respective institutional review boards. All participants, except anonymously recruited blood bank donors in Sri Lanka, provided written informed consent prior to participation in the study. Latent tuberculosis (TB) infection status was confirmed by a positive IFN-γ release assay (IGRA) (QuantiFERON-TB Gold In-Tube, Cellestis or T-SPOT.TB; Oxford Immunotec) but no evidence of clinical symptoms of active TB. TB-negative donors were defined as IGRA negative. Subjects born in areas where bacillus Calmette–Guérin vaccination is prevalent were assumed to have been vaccinated. PBMC were obtained by density gradient centrifugation (Ficoll-Hypaque; Amersham Biosciences) from 100 ml leukapheresis or whole blood samples, according to the manufacturer’s instructions. Cells were resuspended at 50–100 million cells per milliliter in FBS (Gemini Bio-Products) containing 10% dimethyl sulfoxide (Sigma) or in Synth-a-Freeze cryopreservation medium (Life Technologies), and cryopreserved in liquid nitrogen.

Cryopreserved PBMC were quickly thawed by incubating each cryovial at 37°C for 2 min. Cells were then transferred into 9 ml of cold medium (RPMI 1640 with l-Glutamine and 25 mM Hepes (Omega Scientific), supplemented with 5% human AB serum (GemCell), 1% penicillin streptomycin (Life Technologies) and 1% GlutaMAX (Life Technologies) in a 15 ml conical tube. Cells were centrifuged at 1200 rpm for 7 min, and resuspended in medium to determine the cell concentration and viability using Trypan blue and a hematocytometer. Then 10 million cells were transferred into a 15 ml conical tube, centrifuged, resuspended in 200 μl of PBS containing 10% FBS, and incubated for 10 min at 4°C. Cells were stained with 200 μl of PBS containing conjugated Abs at a previously determined optimum dilution (3 μl of anti-human CD3-AF700, anti-human CD4-APCeF780, anti-human CD8a-BV650, anti-human CD19-PE Cy7, anti-human CD14-APC, and anti-human CD45RA eF450; 6 μl of anti-human CD56-PE and anti-human CD25-FITC; 12 μl of anti-human CCR7-PerCp Cy5.5; and 0.3 μl of live/dead eF506 stain; see Supplemental Table I for staining reagent sources) for 30 min at 4°C, protected from light. After two washes in PBS, cells were resuspended in 500 μl of PBS containing 0.5% FBS and 2 mM EDTA (pH 8) (FACS Buffer), transferred into a 5-ml polypropylene FACS tube (BD Biosciences) and stored at 4°C protected from light for up to 4 h until FACS acquisition. For samples requiring fixation, following staining and washes, cells were fixed in 200 μl of 0.5 or 4% paraformaldehyde (PFA) for 15 min at room temperature, washed twice in PBS, and resuspended into 500 μl of PBS containing 0.5% FBS and 2mM EDTA (pH 8) (FACS Buffer), transferred into a 5 ml polypropylene FACS tube (BD Biosciences), and stored at 4°C protected from light for up to 4 h until FACS acquisition. FACS acquisition was performed on a BD LSR-I/II cell analyzer (BD Biosciences) or on a BD FACSAria III/Fusion cell sorter (BD Biosciences). Compensation was realized with single-stained beads (UltraComp eBeads; eBiosciences) in PBS using the same Ab dilution as for the cell staining. FCS files for latent TB donors and healthy controls were deposited in the public database ImmPort (study ID SDY820).

Manual gating was performed with FlowJo version 10.1. The automated gating method used in this manuscript is derived from the FLOCK (FLOw Clustering without K) density-based data clustering method for unsupervised identification of cell populations from polychromatic flow cytometry data (17). The FLOCK method is publicly available at the Immunology Database and Analysis Portal (https://immport.niaid.nih.gov) and its source code is publicly accessible at SourceForge (https://sourceforge.net/projects/immportflock). Major extensions have been added to the FLOCK-based automated gating approach in this manuscript. User-provided constraints about the initial boundaries of the gates have been used to direct the unsupervised clustering of FLOCK as follows. When the centroid of a FLOCK-identified data cluster is outside the user-defined boundary, the whole data cluster will be excluded from the gate; when the centroids of multiple FLOCK-identified data clusters are within the user-defined boundary, the cell populations defined by these centroids will be merged into one single-cell population for comparison. Thus, the user will not need to manually interpret the large number of FLOCK-identified cell populations, which is a common challenge for unsupervised clustering-based automated gating approaches. Further, the original FLOCK approach only outputs statistics for each individual data cluster. The automated gating approach used in this manuscript is able to output population statistics as in sequential manual gating steps by organizing the FLOCK-identified data clusters into the predefined cell type hierarchy. Therefore, the directed unsupervised clustering extension of FLOCK used in this study is specifically designed for scientific use cases when a sequential gating strategy (like Supplemental Fig. 1) is available.

Each provoked fail was performed as indicated in the 4Flow cytometry staining and acquisition section with the following specific changes: 1) staining for 60 min at 37°C; 2) use of cryopreserved PBMC already thawed once and recryopreserved; 3) fixation of stained samples with 0.5% PFA; 4) thawed PBMC were stored in medium supplemented with 0.5% human AB serum for 12 h at 37°C before staining; 5) stained cells were stored at 4°C for 3 d before acquisition; 6) staining for 10 min at 4°C; 7) staining with 3-fold less Ab volumes; 8) stained cells were stored at 4°C for 4 d before acquisition; 9) thawed PBMC were stored in FACS buffer for 12 h at 4°C before staining; 10) stained cells were stored at 4°C for 7 d before acquisition; 11) acquisition on a BD LSR-II cell analyzer with 585/42 and 605/20 bandpasses on the violet laser instead of 525/50 and 660/20 bandpasses for the detection of eF506 and BV650 fluorescence emission, respectively; and 12) fixation of stained samples with 4% PFA. A summary for each provoked fail is available in Fig. 2B.

FIGURE 2.

Development of a QC score to evaluate and correct for technical variation between experiment runs. Cell frequencies were determined with the 10-color flow cytometry panel and the centralized manual gating method on 24 repeat runs or 10 provoked fail runs of cryopreserved PBMC derived from a single blood donation of a control donor. QC score was calculated based on the deviation from average for each of the normally distributed lymphocyte populations and 10% defined as the pass/fail threshold. (A) QC score obtained with normal repeat runs (black) or provoked fail runs (blue). (B) Detailed protocol and associated QC score for each provoked fail run. Effect of fixation with PFA on the frequency of (C) monocytes and (D) CD8 T cells, from three independent repeat runs of the control donation.

FIGURE 2.

Development of a QC score to evaluate and correct for technical variation between experiment runs. Cell frequencies were determined with the 10-color flow cytometry panel and the centralized manual gating method on 24 repeat runs or 10 provoked fail runs of cryopreserved PBMC derived from a single blood donation of a control donor. QC score was calculated based on the deviation from average for each of the normally distributed lymphocyte populations and 10% defined as the pass/fail threshold. (A) QC score obtained with normal repeat runs (black) or provoked fail runs (blue). (B) Detailed protocol and associated QC score for each provoked fail run. Effect of fixation with PFA on the frequency of (C) monocytes and (D) CD8 T cells, from three independent repeat runs of the control donation.

Close modal

Normality of cell populations was determined using the D’Agostino and Pearson omnibus normality test. The technical coefficient of variation associated with each cell population was calculated using the 24 repeat runs of the control donation. For each cell population that followed a normal distribution and a technical coefficient of variation lower than 35% (13 populations in total), a p value was calculated using a Z-score, corrected for multiple testing by the Bonferroni method and expressed as a percentage. For each repeat run, the quality control (QC) score was assigned to the lowest p value from all 13 cell populations (thus reflecting the cell population that deviates the most from its expected coefficient of variation).

The coefficient of variation (CV) of each cell population was defined as the ratio of its SD and its arithmetic mean, expressed as a percentage. The SD of the CV was calculated by a bootstrapping procedure with 10,000 replications using the package boot for R, version 3.2.3. All statistical tests, correlations, and linear regressions were performed using GraphPad Prism, version 6.0. The CV was compared between cell populations using an ordinary one-way ANOVA test. Interindividual and technical variability for each cell population were assessed using multiple t tests. The gender effect on cell population frequencies was determined using either a multiple non-parametric Kruskal-Wallis test or an equivalence test. The equivalence test was based on 90% confidence intervals calculated with a Mann–Whitney non-parametric U test between males and females and defining the zone of statistical indifference as 4-fold of the technical CV. The age effect on cell population frequencies was defined using a Pearson correlation. Multivariate linear regression analysis between age and latent tuberculosis signature was performed using the lm function in R. All p values were adjusted for multiple testing with the Bonferroni method.

Based on previously defined panels from the first round of the HIPC program (9) and our own research goals, we selected six lineage markers, three differentiation markers, and one viability marker to combine into one 10-color flow cytometry panel (Supplemental Table I). Using fluorescence minus one controls, we ensured that the combination of Abs and fluorochromes selected had no detrimental effect on the individual staining for each Ag (data not shown). Using the gating strategy illustrated in Supplemental Fig. 1, we used this panel to identify the 17 lymphocyte and monocyte populations listed in Table I comprising six major cell subsets, five CD4 T cell subsets, four CD8 T cell subsets, and two CD56 T cell subsets. Gates one to four were used as common baseline filters to identify all 17 populations (Supplemental Fig. 1). Throughout this manuscript, we use the population names listed in Table I as shorthand rather than the more exact, but less readable, cell population definitions based on marker patterns. Based on the resolution sensitivity and the level of expression of each marker, cell populations were segregated into clearly defined cell populations (clear distinction between marker pattern positive and negative populations) and poorly resolved cell populations (dim expression and/or poor segregation between positive and negative populations) (Table I).

Table I.
Lymphocyte populations derived from the 10-color flow cytometry panel
CategoryGate IDPopulation NameGating StrategyResolution Sensitivity
Major cell subsets Monocytes CD14+ Clearly defined 
B cells CD14/CD19+CD3 Clearly defined 
NK cells CD14/CD56+CD3 Clearly defined 
11 T cells CD14/CD56CD3+ Clearly defined 
12 CD4 T cells CD14/CD56CD3+/CD4+CD8 Clearly defined 
13 CD8 T cells CD14/CD56CD3+/CD4CD8+ Clearly defined 
CD4 T cell subsets 14 Tregs CD14/CD56CD3+/CD4+CD8/CD25+ Poorly resolved 
15 Naive CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RA+CCR7+ Clearly defined 
16 Tcm CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RACCR7+ Poorly resolved 
17 Tem CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RACCR7 Poorly resolved 
18 Temra CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RA+CCR7 Poorly resolved 
CD8 T cell subsets 19 Naive CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RA+CCR7+ Clearly defined 
20 Tcm CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RACCR7+ Poorly resolved 
21 Tem CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RACCR7 Poorly resolved 
22 Temra CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RA+CCR7 Poorly resolved 
CD56 T cell subsets CD3+ CD56 T cells CD14/CD56+CD3+ Poorly resolved 
10 CD3hi CD56 T cells CD14/CD56+CD3hi Poorly resolved 
CategoryGate IDPopulation NameGating StrategyResolution Sensitivity
Major cell subsets Monocytes CD14+ Clearly defined 
B cells CD14/CD19+CD3 Clearly defined 
NK cells CD14/CD56+CD3 Clearly defined 
11 T cells CD14/CD56CD3+ Clearly defined 
12 CD4 T cells CD14/CD56CD3+/CD4+CD8 Clearly defined 
13 CD8 T cells CD14/CD56CD3+/CD4CD8+ Clearly defined 
CD4 T cell subsets 14 Tregs CD14/CD56CD3+/CD4+CD8/CD25+ Poorly resolved 
15 Naive CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RA+CCR7+ Clearly defined 
16 Tcm CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RACCR7+ Poorly resolved 
17 Tem CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RACCR7 Poorly resolved 
18 Temra CD4 T cells CD14/CD56CD3+/CD4+CD8/CD45RA+CCR7 Poorly resolved 
CD8 T cell subsets 19 Naive CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RA+CCR7+ Clearly defined 
20 Tcm CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RACCR7+ Poorly resolved 
21 Tem CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RACCR7 Poorly resolved 
22 Temra CD8 T cells CD14/CD56CD3+/CD4CD8+/CD45RA+CCR7 Poorly resolved 
CD56 T cell subsets CD3+ CD56 T cells CD14/CD56+CD3+ Poorly resolved 
10 CD3hi CD56 T cells CD14/CD56+CD3hi Poorly resolved 

To estimate the technical variability associated with each cell population, we used multiple vials of cryopreserved PBMC obtained from a single blood donation and repeatedly analyzed them with the 10-color flow cytometry panel. A total of 24 repeat runs were performed on samples from this control donation on various days throughout a 7 mo period by three different operators, and acquired on four different cell analyzers. Technical variability differed greatly between cell populations, ranging from CV = 12–101% (Fig. 1A). Three populations were identified as highly variable (significantly higher coefficient of variation compared with the other cell populations, Fig. 1A): regulatory T cells (Tregs), effector memory cells re-expressing CD45RA (Temra) CD4 T cells and Temra CD8 T cells. All three were previously defined as poorly resolved cell populations (Table I). When separating cell populations according to their resolution sensitivity, poorly resolved cell populations were associated with a significant higher technical variation compared with clearly defined cell populations (Fig. 1B). To assess the extent to which the cell-type resolution explained the increased technical variability, we examined gating of memory CD8 T cells, which can be considered as three distinct poorly resolved cell populations [central memory (Tcm), effector memory (Tem) and Temra], or as one clearly defined cell population (Fig. 1C). Technical variability in total memory CD8 T cells was lower compared with Tcm, Tem, and Temra CD8 T cells, and of a similar level to the clearly defined naive CD8 T cells (Fig. 1D). Thus, the higher technical variability for memory CD8 T cell subpopulations is a direct consequence of them being poorly resolved.

FIGURE 1.

Assessment of technical variability within lymphocyte populations using a control donation. Cell frequencies were determined with the 10-color flow cytometry panel on 24 repeat runs of cryopreserved PBMC derived from a single blood donation of a control donor. (A) Technical variability of each lymphocyte population obtained with the individual manual gating method. (B) Technical variability of clearly defined (black) and poorly resolved (blue) cell populations obtained with the individual manual gating method. (C) Representative staining and (D) technical variability of CD8 T cell subset frequencies obtained with the individual manual gating method. Comparison between technical variability of (E) clearly defined or (F) poorly resolved cell population frequency obtained with individual manual gating, centralized manual gating, or automated gating. Linear regression of cell frequencies obtained with centralized manual gating or automated gating for (G) CD4 T cells, (H) Tem CD8 T cells, (I) clearly defined, or (J) poorly resolved lymphocyte populations. Technical variability between individual cell populations was compared with an ordinary one-way ANOVA test; technical variability between clearly resolved and poorly defined cell populations and between each gating method was compared with the non-parametric Mann–Whitney U test. *p < 0.05, **p < 0.01.

FIGURE 1.

Assessment of technical variability within lymphocyte populations using a control donation. Cell frequencies were determined with the 10-color flow cytometry panel on 24 repeat runs of cryopreserved PBMC derived from a single blood donation of a control donor. (A) Technical variability of each lymphocyte population obtained with the individual manual gating method. (B) Technical variability of clearly defined (black) and poorly resolved (blue) cell populations obtained with the individual manual gating method. (C) Representative staining and (D) technical variability of CD8 T cell subset frequencies obtained with the individual manual gating method. Comparison between technical variability of (E) clearly defined or (F) poorly resolved cell population frequency obtained with individual manual gating, centralized manual gating, or automated gating. Linear regression of cell frequencies obtained with centralized manual gating or automated gating for (G) CD4 T cells, (H) Tem CD8 T cells, (I) clearly defined, or (J) poorly resolved lymphocyte populations. Technical variability between individual cell populations was compared with an ordinary one-way ANOVA test; technical variability between clearly resolved and poorly defined cell populations and between each gating method was compared with the non-parametric Mann–Whitney U test. *p < 0.05, **p < 0.01.

Close modal

It is intuitive that poorly resolved populations may be associated with higher technical variability when gated by different operators as was done in Fig. 1A due to the subjectiveness when visually setting a gate for such populations. We were interested in assessing the impact of the gating method on the technical variability associated with our flow cytometry panel. To do so, we compared the cell frequencies obtained from the same set of repeat runs of the control donation using three different gating methods, ranging from the most to the least subjective approach: 1) individual manual gating (done by different operators that performed the flow cytometry acquisition as for Fig. 1A); 2) centralized manual gating (done by one unique operator); and 3) automated gating using directed unsupervised clustering using FLOCK as described in 2Materials and Methods. Technical variability of clearly defined cell populations was similar between the different gating methods (Fig. 1E). However, technical variability of poorly resolved cell populations was significantly reduced with centralized manual gating (Tregs, Temra CD4 T cells, and Temra CD8 T cells) or automated gating (Tregs and Temra CD8 T cells) compared with individual manual gating (Fig. 1F). Thus, the gating approach significantly impacts the technical variability associated with poorly resolved cell populations, and centralized manual or automated gating can significantly reduce this technical variability.

Next, we wanted to determine to what extent gating approaches explained the remaining variability of cell population frequencies derived from aliquots of the same biological sample, versus other technical factors, such as aliquot sample quality, procedures, reagents, or acquisition parameters (9). We took advantage of having population frequencies generated by two independently performed gating approaches on these samples: centralized manual gating and automated gating. When correlating the frequencies derived by the two gating approaches from the same FCS files, we found high correlations for both clearly defined (Fig. 1G) and poorly resolved (Fig. 1H) populations. This was broadly true for all clearly defined cell populations (r2 = 0.4–0.9, Fig. 1I) and most poorly resolved populations (r2 = 0.2–0.6, Fig. 1J) except for Temra CD4 T cells. Taken together, these results suggest that a significant proportion of the technical variability in flow cytometry is independent of the gating method used, especially for clearly resolved populations. Rather, this variability might be due to differences in samples (i.e., batch-to-batch variation in cryovials of the control donation), in reagents (i.e., batch-to-batch variation in Ab lots), or in cell analyzer daily performances.

Although it is impossible to account and correct for each factor driving technical variability, overall we found that less subjective centralized manual gating can significantly reduce technical variability. Thus, in the following sections, the technical coefficient of variation based on centralized manual gating was chosen as a technical baseline for each cell population against which the effect of biological variables can be assessed.

In our HIPC studies, flow cytometry–based cell sorting is typically the first step in resource intensive experiments such as RNA-Seq or ChIP-Seq. Cell sorts are performed in batches on multiple days to process sufficient numbers of donors. To ensure that flow cytometry was performed equivalently for different batches, we set out to establish a quality score that assesses technical variability between runs using the control donation samples, and provides a pass or fail output. We focused the QC score based on the identification of cell populations that: 1) show a normal distribution (p > 0.05 with the D’Agostino and Pearson omnibus normality test); and 2) have low technical variability (coefficient of variation <35%) between runs. Of the 17 cell populations, 13 met these thresholds when using centralized manual gating (data not shown). Based on the cell frequencies of these 13 cell populations, we calculated a quality control score for each of the repeat runs of the control donation (Fig. 2A). We then provoked fail runs by modifying either the sample preparation or data acquisition parameters. Comparing the QC scores between the regular repeat runs, and the provoked fails, we found that a threshold pass/fail at 10% identified nine out of the 12 provoked fail runs as problematic, whereas all regular runs had scores above this threshold (Fig. 2A, 2B). The three provoked fail runs that were associated with a pass QC score were the refrozen vial, the prolonged staining period at 37°C, and the fixation with 0.5% PFA (Fig. 2A, 2B). Fixation affected cell population frequencies in a PFA dose-dependent manner: the QC score of the control donation dropped to 17 and 0% when samples were fixed with 0.5 or 4% PFA, respectively (Fig. 2A, 2B). Indeed, whereas most cell population frequencies remained unchanged (e.g., monocytes, Fig. 2C), the staining of some cell populations was significantly affected by the fixation process (e.g., CD8 T cells, Fig. 2D). However, repeat runs of fixed or unfixed samples of the control donation generated similar technical variation for each cell population (data not shown).

Interestingly, when using the automated gating method, the provoked fail runs obtained a higher quality control score, often greater than the threshold pass/fail (Fig. 2B). It appears therefore that whereas a human eye can successfully detect and flag missing or fluorescence shifted cell populations, the DAFi algorithm for the automated gating, which aims to be robust to data shifts across samples, is still able to rescue those populations.

In all experiments reported in the following sections, one sample from the control donation was included to ensure technical reproducibility between batches of samples. Runs in which the samples had a QC score higher than 10% were considered acceptable and those with scores below were considered failing our quality control, and were thus excluded.

Following our investigation of technical variation, we assessed interindividual variability of cell populations by analyzing cryopreserved PBMC samples collected from various human cohorts with our 10-color flow cytometry panel [total of 176 subjects from San Diego (124), Peru (8) and Sri Lanka (44)]. For each lymphocyte population, we defined a normal range based on the average and 2.5th–97.5th percentiles of cell frequencies obtained with the manual centralized gating method (Figure 3, Supplemental Table II). Average frequencies varied greatly among all major cell populations as well as within T cell subsets (Fig. 3, Supplemental Table II). The coefficient of variation associated with each cell population also varied greatly, ranging from 28.7 to 157% (interindividual variability, Fig. 4A). Temra CD4 T cells, CD3hi CD56 T cells, and CD3+ CD56 T cells had the highest interindividual variability, significantly higher compared with the other populations. Importantly, for all cell populations except Tregs, the biological interindividual variability was significantly higher compared with the technical variability previously calculated (Fig. 4A).

FIGURE 3.

Normal range of lymphocyte population frequencies within humans. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 176 human subjects. Box plots represent average and normal range (2.5th–97.5th percentiles) of (A) major cell subsets, (B) CD4 T cell subsets, (C) CD8 T cell subsets, and (D) CD56 T cell subsets.

FIGURE 3.

Normal range of lymphocyte population frequencies within humans. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 176 human subjects. Box plots represent average and normal range (2.5th–97.5th percentiles) of (A) major cell subsets, (B) CD4 T cell subsets, (C) CD8 T cell subsets, and (D) CD56 T cell subsets.

Close modal
FIGURE 4.

Interindividual and intraindividual variability of lymphocyte population frequencies within humans. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 176 human subjects. (A) Comparison between technical variability calculated in Fig. 2 (blue) and interindividual variability (black). Linear regression of (B) lymphocyte subset frequencies, (C) naive CD4 T cell frequencies, (D) CD8 T cell frequencies and monocyte frequencies obtained between two longitudinal samples for 13 healthy human subjects. Visit 2 ranged from 114 to 194 d following visit 1. Interindividual variability between cell populations was compared with an ordinary one-way ANOVA test. Interindividual and technical variability for each cell population were compared with multiple comparison t test. *p < 0.05, #p > 0.05.

FIGURE 4.

Interindividual and intraindividual variability of lymphocyte population frequencies within humans. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 176 human subjects. (A) Comparison between technical variability calculated in Fig. 2 (blue) and interindividual variability (black). Linear regression of (B) lymphocyte subset frequencies, (C) naive CD4 T cell frequencies, (D) CD8 T cell frequencies and monocyte frequencies obtained between two longitudinal samples for 13 healthy human subjects. Visit 2 ranged from 114 to 194 d following visit 1. Interindividual variability between cell populations was compared with an ordinary one-way ANOVA test. Interindividual and technical variability for each cell population were compared with multiple comparison t test. *p < 0.05, #p > 0.05.

Close modal

To address whether cell population frequencies derived from our flow cytometry panel were stable over time in a given individual, we applied the 10-color flow cytometry panel on 13 subjects from San Diego for which a second blood sample was collected 3–7 mo later (samples named visit 1 and visit 2, respectively). For six out of the 17 cell populations, the interindividual variability within the longitudinal cohort at visit 1 was too low to be discriminated from the technical variability (fold-change difference between interindividual and technical variability <2); these populations were therefore excluded from the subsequent analysis. For the remaining 11 populations, there was a very high correlation between the frequencies obtained at visit 1 and visit 2 for most cell populations, including both clearly defined and poorly resolved cell populations (Fig. 4B, 4C). CD8 T cells and monocytes were the two populations for which no significant correlation was observed between the two visits (Fig. 4B, 4D). Both CD8 T cells and monocytes are clearly defined cell populations by flow cytometry with two of the three lowest levels of technical variability in the control samples (Fig. 1A). Thus, their lack of stability over time likely reflects immunological perturbations in some of the longitudinal samples between donors. In contrast, the other nine of 11 cell populations that showed differential frequencies across donors largely maintained those differences, indicating that they are relatively stable markers of blood immune cell composition within each donor.

Having observed that for most cell types, interindividual variability in their abundance in the peripheral blood was much greater than either technical variability or longitudinal variability across our human cohorts (Fig. 4A), we were interested to see whether some of this biological variation could be accounted for by demographic variables such as gender and age. Among the 132 samples from which we had demographics data available [San Diego (124) and Peru (8)], there were no significant differences in cell population frequencies between males and females using the multiple comparison non-parametric Kruskal-Wallis test (data not shown). More stringently, the frequencies in males versus females of all the assessed cell populations also passed a test for equivalence, meaning there is statistical support to affirm that differences observed are contained within a range expected based on technical variability (p < 0.05, see 2Materials and Methods).

In contrast, there was a striking significant inverse correlation between age and the frequency of T cells and CD8 T cells (Fig. 5A). Within T cell subsets, age was inversely correlated with the frequency of naive CD4 T cells and naive CD8 T cells, and positively correlated with the frequency of Tcm CD4 T cells, Tcm CD8 T cells, and Temra CD8 T cells (Fig. 5B, 5C). No association between age and CD56 T cell subset frequencies was observed (Fig. 5D). Scatter plots for the two strongest age-frequency correlations are shown in Fig. 5E for naive CD4 T cells and in Fig. 5F for naive CD8 T cells.

FIGURE 5.

Demographic correlates of biological variability of lymphocyte population frequencies. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 132 human subjects. Pearson correlation coefficient between age and the frequency of (A) major cell subsets, (B) CD4 T cell subsets, (C) CD8 T cell subsets, (D) CD56 T cell subsets, (E) naive CD4 T cells, and (F) naive CD8 T cells. The p values were corrected for multiple testing. *p < 0.05.

FIGURE 5.

Demographic correlates of biological variability of lymphocyte population frequencies. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 132 human subjects. Pearson correlation coefficient between age and the frequency of (A) major cell subsets, (B) CD4 T cell subsets, (C) CD8 T cell subsets, (D) CD56 T cell subsets, (E) naive CD4 T cells, and (F) naive CD8 T cells. The p values were corrected for multiple testing. *p < 0.05.

Close modal

Even after correcting for age and gender, a significant proportion of the interindividual variability still remained above technical variability, indicating that other confounding factors are influencing immune cell population frequencies. Across our cohort of samples from San Diego, ethnicity was highly diverse (Supplemental Fig. 2A), thus we were interested to see whether this factor could explain a proportion of the interindividual variability. Although our study was not powered enough to fully dissect the effect of ethnic background on each cell population abundance, we found significant differences between ethnicity groups for the frequency of major cell types such as T cells and B cells (Supplemental Fig. 2B), as well as CD4 and CD8 T cell subsets (Supplemental Fig. 2C, 2D), but not CD56 T cell subsets (Supplemental Fig. 2E). Taken together, these results suggest that ethnicity is an important covariate of blood immune cell composition and should therefore be taken into consideration whenever possible in human immunophenotyping studies, by ensuring that the clinical groups being compared have a balanced ethnic composition.

Finally, we applied our workflow to a cohort of donors with latent tuberculosis infection (LTBI) to check if we could detect a disease signature, even after correction for technical and biological variability. LTBI status is typically defined by positivity to IGRA combined with absence of clinical symptoms of active TB (18). Although IGRAs are associated with a relatively good sensitivity (>75%), they require an in vitro stimulation step that increases the result variability and complicates logistics (19). An ex vivo immune signature specific of LTBI would facilitate diagnosis, but there is no evidence that LTBI and non-TB infected donors can be differentiated solely based on their ex vivo PBMC immune profile. We focused our analysis on the comparison of cell frequencies in PBMC obtained from 29 healthy controls (San Diego) and 31 individuals with LTBI (IGRA+) from two different geographic locations [San Diego (n = 23) and Peru (n = 8)]. First, we confirmed that biological differences could be detected in our study by ensuring that the biological variability within our combined cohorts was above technical variability. Tregs were the only population for which interindividual variability was not significantly higher than technical variability (p > 0.05 with a multiple comparison t test), and was thus excluded from the subsequent analysis. Next, looking at demographics, we found that the LTBI cohort was significantly older than healthy controls [median age (range) of 31 (19–60) and 43 (20–65) for healthy controls and LTBI donors, respectively, p < 0.05 with the non-parametric Mann–Whitney U test], indicating the need to take this cofactor into consideration in our statistical analysis. After age correction, we found that individuals with LTBI were associated with a lower frequency of CD8 T cells, and a higher frequency of B cells compared with healthy controls (Fig. 6A). No significant differences in CD56 T cell subsets and CD8 T cells frequencies were observed between the two cohorts (Fig. 6B, 6D). However, within CD4 T cell subsets, individuals with LTBI had a lower frequency of naive CD4 T cells compared with healthy controls (Fig. 6C).

FIGURE 6.

Identification of a latent tuberculosis signature within PBMCs. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 31 subjects with latent tuberculosis and 29 tuberculosis negative subjects. Differences in the frequency of (A) major lymphocyte subsets, (B) CD56 T cell subsets, (C) CD4 T cell subsets, and (D) CD8 T cell subsets between latent TB donors and healthy subjects, determined by multilinear regression analysis with age as a covariate and multiple testing correction. *p < 0.05, **p < 0.01.

FIGURE 6.

Identification of a latent tuberculosis signature within PBMCs. Cell frequencies were determined with the 10-color flow cytometry panel and centralized manual gating on cryopreserved PBMC samples obtained from 31 subjects with latent tuberculosis and 29 tuberculosis negative subjects. Differences in the frequency of (A) major lymphocyte subsets, (B) CD56 T cell subsets, (C) CD4 T cell subsets, and (D) CD8 T cell subsets between latent TB donors and healthy subjects, determined by multilinear regression analysis with age as a covariate and multiple testing correction. *p < 0.05, **p < 0.01.

Close modal

Standardization of flow cytometry data acquisition and analysis is critical to ensure quality and reproducibility of large-scale human immunology studies. Previous studies have aimed at reducing technical variability by optimizing and standardizing reagents, protocols, and data analysis for predesigned panels (7, 913, 20) but none established a universal workflow that could be applied to any flow cytometry experiment. Furthermore, previous studies did not simultaneously assess the effect of technical variability on cell populations along with biological variability due to demographic cofactors. In this study, we report on the development of a reusable workflow to control for both technical and biological variation in flow cytometry. We applied this workflow to evaluate a customized panel measuring the frequency of 17 immune cell populations within cryopreserved human PBMC of major interest to our studies. We quantified the technical variation associated with each cell population by repeatedly applying this panel to PBMC samples obtained from the same control donation, and comparing different analysis strategies and developing a quality control metric. In addition, we also studied the biological variability associated with our cell populations in different individuals and in a given individual over time, as well as the influence of demographics. Finally, we illustrated the usefulness of our approach by identifying a disease signature in volunteers infected with latent tuberculosis.

Technical variation differed greatly between the cell populations identified in our panel. As expected, poorly resolved cell populations were associated with higher technical variability compared with clearly resolved cell populations. This is explained by the expression profile of our markers used for phenotyping that is either continuous (e.g., Tcm and Tem T cells) or dimly expressed (e.g., Tregs) in poorly resolved cell populations. Technical variability associated with those cell populations can be improved by using less-subjective gating strategies (see the paragraph below) or by choosing alternative markers to define the cell population (e.g., the inclusion of markers such as FOXP3 to more clearly separate the Treg population).

Our standard flow cytometry protocol for HIPC does not include cell fixation, as we are usually sorting cell populations for downstream transcriptomic analyses. However, some clinical immunophenotyping studies might require sample fixation, for instance if no acquisition is possible right away, or if the samples are considered infectious. In this study we show that fixation significantly affects the staining of some cell populations, and thus their frequencies, and that this effect is dependent on the concentration of PFA used. This observation reflects the variable degree of sensitivity of fluorochromes and Ab clones to paraformaldehyde. Thus, the choice of Abs and dilutions needs to be carefully optimized for each given panel and staining protocol. For instance, the normal range reported in this study should only be used as a guideline when samples have been stained under similar conditions. Nevertheless, fixed repeat runs of the control donation generated similar technical variation compared with unfixed repeat runs, validating that our methodological framework can also be applied to flow cytometric experiments requiring sample fixation.

Our study compared three gating approaches: individual manual gating, centralized manual gating, and automated gating. We found that for cell populations that are clearly resolved, all three approaches had a high degree of concordance and all can be applied equally. In contrast, individual manual gating proved to be problematic when applied to cell populations that are poorly resolved. Centralized manual gating and automated gating methods both significantly reduced technical variability of poorly resolved cell populations, which is in concordance with previous reports (7, 10). Comparing the technical variability achieved with centralized manual and automated gating, the two performed comparably except when applied to the poorly resolved and comparatively low-frequency Temra CD4 T cell population. The automated gating pipeline, which requires the detected cell populations to either follow a predefined distribution model or form a data cluster with relatively abundant number of events, could not reliably detect this cell population. This highlights the need to verify robustness of automated gating pipelines to detect cell populations of interest before blindly accepting their output.

Relative abundance and interindividual variability of each cell population differed widely between cell subsets. Our data allowed us to define an expected normal range in human peripheral blood for the 17 immune cell populations derived from our panel. This normal range can be of use for any researcher interested in those same cell populations, to flag any technical issues associated with a given flow cytometry panel or acquisition workflow (e.g., use of wrong Ab clone or poor instrument laser efficiency) as well as to identify biological outliers within a large set of samples.

Remarkably, despite being associated with significant biological interindividual variability, we found that within each individual, most lymphocyte populations have surprisingly stable frequencies over time. This confirms previous findings (4, 15), and suggests that the relative abundance of most lymphocyte subsets in an individual over a period of at least several months is an intrinsic feature of the individual, presumably regulated by invariant factors such as genetic background and infrequent immunological events with major impact that shape the individual’s immune history. The observation that most lymphocyte populations display such low intraindividual variability, including poorly resolved and low-frequency cell populations such as CD3+ CD56 T cells, reinforces the conclusion that important biological differences between individuals can reliably be detected using our flow cytometry panel.

The highest intraindividual variability of cell subsets over time was observed for monocytes and CD8 T cells. Monocytes might be more susceptible to minor immunological perturbations as they are known to be the first line of defense upon Ag exposure, and have a strong ability to migrate from the blood to a site of infection (21). Thus, it is likely that their relative abundance in the peripheral blood fluctuates over time depending on the recent antigenic load being experienced by the individual at a given point of time. For CD8 T cells, the low correlation observed between the two visits was driven by two subjects for which CD8 T cell frequencies dramatically increased at visit 2. It is possible that these two subjects contracted a minor infectious disease or inflammatory process between the two visits that perturbed the CD8 frequency in these individuals specifically. Larger panels of donors with longitudinal samples would be necessary to follow up these findings.

For each immune cell population identified with our panel, the comparison of the technical variability with its biological variability provides an estimate of how many samples will be required to detect significant differences between groups of samples with an effect size similar to the normal biological variation. We found marked differences among cell populations, with some having a high ratio of biological over technical variability (e.g., CD56 T cell subsets) versus others for which a much greater number of samples will be necessary to be sufficiently powered to overcome technical variability (e.g., Tregs, for which our data would alternatively recommend utilizing a different staining panel). Additionally, depending on the study design, different sources of variability have to be taken into account. Because the intraindividual variability is relatively low for most cell populations, in a study comparing paired samples from the same donor cohort pre- and post perturbation, the technical variability of cell population detection becomes the dominant source of noise. In contrast, when samples from different groups of donors are compared, the much larger biological variability between donors has to be considered, which can easily lead to erroneous results if the donor cohort sizes are not sufficiently large.

Genetic heritability accounts for <50% of the immune system variation among humans, suggesting that other non-genetic factors (such as age and exposure) might be critical in explaining the remaining variability (22, 23). Interestingly, we found that although gender had no impact on our immune cell population frequencies, age was strongly associated with changes in the cellular immune composition of human peripheral blood. In particular, older individuals were characterized by a striking loss of naive CD4 and CD8 T cells, which is in concordance with previous reports looking at circulating T cell subsets in young and elderly populations (24, 25). These results also corroborate a recent large-scale study that highlighted the importance of non-genetic factors such as age but not gender on the cellular composition of the circulating human immune system (15).

Additionally, we were able to examine the influence of ethnic background on lymphocyte subset frequencies. Our results strongly suggest that ethnicity can be an important factor driving variation in the cellular immune composition of human peripheral blood. This proposal is also supported by previous studies reporting both qualitative and quantitative differences in the immune response to infection or vaccination in human cohorts of different ethnicities (2628). Although the exact cause underlying those observations remains uncertain, the diversity in genetic background, microbiome communities, environmental exposures, and social differences are likely to be driving human immune variation between ethnicities. Another factor often related to ethnicity that is important to take into consideration is the geographical location of the human subjects. Indeed, distinct geographic locations might indicate exposure to different microorganisms, resulting in a different shaping of the immune system. Hence, two individuals with the same ethnicity but living in different locations might have completely separate immune profiles.

Taken together, our results and those from previous reports highlight the critical importance in clinical immunophenotyping studies to take into consideration non-genetic features that are known or suspected to drive immune variability, by recruiting matching disease and healthy cohorts, or correcting for those differences between groups when performing data analysis.

Finally, our flow cytometry panel allowed us to identify a specific signature of latent tuberculosis infection. Previous T cell immunophenotyping studies have successfully discriminated active TB from latent TB or from non-TB infected individuals (2931). However, the distinction between latent TB and non-TB infected subjects based solely on their PBMC profile has not yet been reported. In this study we show that after controlling for technical variation and age, individuals with latent TB had a lower frequency of CD8 T cells and naive CD4 T cells, and a higher frequency of B cells, compared with TB negative subjects. We thus provide primary evidence that ex vivo immune signatures in PBMC can discriminate between LTBI and non-TB infected donors, which could potentially be applied as an additional tool to IGRAs for improving the specificity of LTBI diagnosis. Recent studies have also shown differences in whole blood cell counts of infants and adults with various TB status (32, 33). Specifically, a high monocyte/lymphocyte ratio was associated with an increased risk of developing active TB (32, 33). Thus, we predict that combining whole blood counts to our newly defined ex vivo PBMC signature of LTBI will improve its discriminatory power.

Overall, our study indicates that controlling for both technical and biological variability is crucial to appropriately design and power human immunology studies. Our findings can be directly applied to any flow cytometry study examining cell populations identified with our panel. In addition, we provide a universal workflow for how to develop and evaluate pre-existing or new flow cytometry panels for large-scale immunology studies.

We thank the National Blood Center, Ministry of Health, Colombo, Sri Lanka for providing buffy coat samples used in this study.

This work was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award numbers U19AI118626 and R24AI108564. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

The flow cytometry data presented in this article have been submitted to the ImmPort database under accession number SDY820.

The online version of this article contains supplemental material.

Abbreviations used in this article:

CV

coefficient of variation

FLOCK

FLOw Clustering without K

HIPC

Human Immunology Project Consortium

IGRA

IFN-γ release assay

LTBI

latent tuberculosis infection

PFA

paraformaldehyde

QC

quality control

TB

tuberculosis

Treg

regulatory T cell.

1
Baumgarth
N.
,
Roederer
M.
.
2000
.
A practical approach to multicolor flow cytometry for immunophenotyping.
J. Immunol. Methods
243
:
77
97
.
2
Chattopadhyay
P. K.
,
Hogerkorp
C. M.
,
Roederer
M.
.
2008
.
A chromatic explosion: the development and future of multiparameter flow cytometry.
Immunology
125
:
441
449
.
3
Betts
M. R.
,
Nason
M. C.
,
West
S. M.
,
De Rosa
S. C.
,
Migueles
S. A.
,
Abraham
J.
,
Lederman
M. M.
,
Benito
J. M.
,
Goepfert
P. A.
,
Connors
M.
, et al
.
2006
.
HIV nonprogressors preferentially maintain highly functional HIV-specific CD8+ T cells.
Blood
107
:
4781
4789
.
4
Tsang
J. S.
,
Schwartzberg
P. L.
,
Kotliarov
Y.
,
Biancotto
A.
,
Xie
Z.
,
Germain
R. N.
,
Wang
E.
,
Olnes
M. J.
,
Narayanan
M.
,
Golding
H.
, et al
Baylor HIPC Center
; 
CHI Consortium
.
2014
.
Global analyses of human immune variation reveal baseline predictors of postvaccination responses. [Published erratum appears in 2014 Cell 158: 226.]
Cell
157
:
499
513
.
5
Darrah
P. A.
,
Patel
D. T.
,
De Luca
P. M.
,
Lindsay
R. W.
,
Davey
D. F.
,
Flynn
B. J.
,
Hoff
S. T.
,
Andersen
P.
,
Reed
S. G.
,
Morris
S. L.
, et al
.
2007
.
Multifunctional TH1 cells define a correlate of vaccine-mediated protection against Leishmania major.
Nat. Med.
13
:
843
850
.
6
Burel
J. G.
,
Apte
S. H.
,
Doolan
D. L.
.
2016
.
Systems approaches towards molecular profiling of human immunity.
Trends Immunol.
37
:
53
67
.
7
Maecker
H. T.
,
Rinfret
A.
,
D’Souza
P.
,
Darden
J.
,
Roig
E.
,
Landry
C.
,
Hayes
P.
,
Birungi
J.
,
Anzala
O.
,
Garcia
M.
, et al
.
2005
.
Standardization of cytokine flow cytometry assays.
BMC Immunol.
6
:
13
.
8
Saeys
Y.
,
Gassen
S. V.
,
Lambrecht
B. N.
.
2016
.
Computational flow cytometry: helping to make sense of high-dimensional immunology data.
Nat. Rev. Immunol.
16
:
449
462
.
9
Maecker
H. T.
,
McCoy
J. P.
,
Nussenblatt
R.
.
2012
.
Standardizing immunophenotyping for the human immunology project. [Published erratum appears in 2012 Nat. Rev. Immunol.12: 471.]
Nat. Rev. Immunol.
12
:
191
200
.
10
Finak
G.
,
Langweiler
M.
,
Jaimes
M.
,
Malek
M.
,
Taghiyar
J.
,
Korin
Y.
,
Raddassi
K.
,
Devine
L.
,
Obermoser
G.
,
Pekalski
M. L.
, et al
.
2016
.
Standardizing flow cytometry immunophenotyping analysis from the Human ImmunoPhenotyping Consortium.
Sci. Rep.
6
:
20686
.
11
Kalina
T.
,
Flores-Montero
J.
,
van der Velden
V. H.
,
Martin-Ayuso
M.
,
Böttcher
S.
,
Ritgen
M.
,
Almeida
J.
,
Lhermitte
L.
,
Asnafi
V.
,
Mendonça
A.
, et al
EuroFlow Consortium (EU-FP6, LSHB-CT-2006-018708)
.
2012
.
EuroFlow standardization of flow cytometer instrument settings and immunophenotyping protocols.
Leukemia
26
:
1986
2010
.
12
Streitz
M.
,
Miloud
T.
,
Kapinsky
M.
,
Reed
M. R.
,
Magari
R.
,
Geissler
E. K.
,
Hutchinson
J. A.
,
Vogt
K.
,
Schlickeiser
S.
,
Kverneland
A. H.
, et al
.
2013
.
Standardization of whole blood immune phenotype monitoring for clinical trials: panels and methods from the ONE study.
Transplant. Res.
2
:
17
.
13
Perfetto
S. P.
,
Ambrozak
D.
,
Nguyen
R.
,
Chattopadhyay
P. K.
,
Roederer
M.
.
2012
.
Quality assurance for polychromatic flow cytometry using a suite of calibration beads.
Nat. Protoc.
7
:
2067
2079
.
14
Aghaeepour
N.
,
Finak
G.
,
Hoos
H.
,
Mosmann
T. R.
,
Brinkman
R.
,
Gottardo
R.
,
Scheuermann
R. H.
FlowCAP Consortium
; 
DREAM Consortium
.
2013
.
Critical assessment of automated flow cytometry data analysis techniques.
Nat. Methods
10
:
228
238
.
15
Carr
E. J.
,
Dooley
J.
,
Garcia-Perez
J. E.
,
Lagou
V.
,
Lee
J. C.
,
Wouters
C.
,
Meyts
I.
,
Goris
A.
,
Boeckxstaens
G.
,
Linterman
M. A.
,
Liston
A.
.
2016
.
The cellular composition of the human immune system is shaped by age and cohabitation.
Nat. Immunol.
17
:
461
468
.
16
Poland
G. A.
,
Quill
H.
,
Togias
A.
.
2013
.
Understanding the human immune system in the 21st century: the Human Immunology Project Consortium.
Vaccine
31
:
2911
2912
.
17
Qian
Y.
,
Wei
C.
,
Eun-Hyung Lee
F.
,
Campbell
J.
,
Halliley
J.
,
Lee
J. A.
,
Cai
J.
,
Kong
Y. M.
,
Sadat
E.
,
Thomson
E.
, et al
.
2010
.
Elucidation of seventeen human peripheral blood B-cell subsets and quantification of the tetanus response using a density-based method for the automated identification of cell populations in multidimensional flow cytometry data.
Cytometry B Clin. Cytom.
78
(
Suppl. 1
):
S69
S82
.
18
Pai
M.
,
Behr
M. A.
,
Dowdy
D.
,
Dheda
K.
,
Divangahi
M.
,
Boehme
C. C.
,
Ginsberg
A.
,
Swaminathan
S.
,
Spigelman
M.
,
Getahun
H.
, et al
.
2016
.
Tuberculosis.
Nat Rev Dis Primers
2
:
16076
.
19
Petruccioli
E.
,
Scriba
T. J.
,
Petrone
L.
,
Hatherill
M.
,
Cirillo
D. M.
,
Joosten
S. A.
,
Ottenhoff
T. H.
,
Denkinger
C. M.
,
Goletti
D.
.
2016
.
Correlates of tuberculosis risk: predictive biomarkers for progression to active tuberculosis.
Eur. Respir. J.
48
:
1751
1763
.
20
Kalina
T.
,
Flores-Montero
J.
,
Lecrevisse
Q.
,
Pedreira
C. E.
,
van der Velden
V. H.
,
Novakova
M.
,
Mejstrikova
E.
,
Hrusak
O.
,
Böttcher
S.
,
Karsch
D.
, et al
.
2015
.
Quality assessment program for EuroFlow protocols: summary results of four-year (2010-2013) quality assurance rounds.
Cytometry A
87
:
145
156
.
21
Shi
C.
,
Pamer
E. G.
.
2011
.
Monocyte recruitment during infection and inflammation.
Nat. Rev. Immunol.
11
:
762
774
.
22
Brodin
P.
,
Jojic
V.
,
Gao
T.
,
Bhattacharya
S.
,
Angel
C. J.
,
Furman
D.
,
Shen-Orr
S.
,
Dekker
C. L.
,
Swan
G. E.
,
Butte
A. J.
, et al
.
2015
.
Variation in the human immune system is largely driven by non-heritable influences.
Cell
160
:
37
47
.
23
Orrù
V.
,
Steri
M.
,
Sole
G.
,
Sidore
C.
,
Virdis
F.
,
Dei
M.
,
Lai
S.
,
Zoledziewska
M.
,
Busonero
F.
,
Mulas
A.
, et al
.
2013
.
Genetic variants regulating immune cell levels in health and disease.
Cell
155
:
242
256
.
24
Fagnoni
F. F.
,
Vescovini
R.
,
Passeri
G.
,
Bologna
G.
,
Pedrazzoni
M.
,
Lavagetto
G.
,
Casti
A.
,
Franceschi
C.
,
Passeri
M.
,
Sansoni
P.
.
2000
.
Shortage of circulating naive CD8(+) T cells provides new insights on immunodeficiency in aging.
Blood
95
:
2860
2868
.
25
Lazuardi
L.
,
Jenewein
B.
,
Wolf
A. M.
,
Pfister
G.
,
Tzankov
A.
,
Grubeck-Loebenstein
B.
.
2005
.
Age-related loss of naïve T cells and dysregulation of T-cell/B-cell interactions in human lymph nodes.
Immunology
114
:
37
43
.
26
Coussens
A. K.
,
Wilkinson
R. J.
,
Nikolayevskyy
V.
,
Elkington
P. T.
,
Hanifa
Y.
,
Islam
K.
,
Timms
P. M.
,
Bothamley
G. H.
,
Claxton
A. P.
,
Packe
G. E.
, et al
.
2013
.
Ethnic variation in inflammatory profile in tuberculosis.
PLoS Pathog.
9
:
e1003468
.
27
Haralambieva
I. H.
,
Ovsyannikova
I. G.
,
Kennedy
R. B.
,
Larrabee
B. R.
,
Shane Pankratz
V.
,
Poland
G. A.
.
2013
.
Race and sex-based differences in cytokine immune responses to smallpox vaccine in healthy individuals.
Hum. Immunol.
74
:
1263
1266
.
28
Haralambieva
I. H.
,
Salk
H. M.
,
Lambert
N. D.
,
Ovsyannikova
I. G.
,
Kennedy
R. B.
,
Warner
N. D.
,
Pankratz
V. S.
,
Poland
G. A.
.
2014
.
Associations between race, sex and immune response variations to rubella vaccination in two independent cohorts.
Vaccine
32
:
1946
1953
.
29
Harari
A.
,
Rozot
V.
,
Bellutti Enders
F.
,
Perreau
M.
,
Stalder
J. M.
,
Nicod
L. P.
,
Cavassini
M.
,
Calandra
T.
,
Blanchet
C. L.
,
Jaton
K.
, et al
.
2011
.
Dominant TNF-α+ Mycobacterium tuberculosis-specific CD4+ T cell responses discriminate between latent infection and active disease.
Nat. Med.
17
:
372
376
.
30
Arlehamn
C. L.
,
Seumois
G.
,
Gerasimova
A.
,
Huang
C.
,
Fu
Z.
,
Yue
X.
,
Sette
A.
,
Vijayanand
P.
,
Peters
B.
.
2014
.
Transcriptional profile of tuberculosis antigen-specific T cells reveals novel multifunctional features.
J. Immunol.
193
:
2931
2940
.
31
Yang
Q.
,
Xu
Q.
,
Chen
Q.
,
Li
J.
,
Zhang
M.
,
Cai
Y.
,
Liu
H.
,
Zhou
Y.
,
Deng
G.
,
Deng
Q.
, et al
.
2015
.
Discriminating active tuberculosis from latent tuberculosis infection by flow cytometric measurement of CD161-expressing T cells.
Sci. Rep.
5
:
17918
.
32
Naranbhai
V.
,
Kim
S.
,
Fletcher
H.
,
Cotton
M. F.
,
Violari
A.
,
Mitchell
C.
,
Nachman
S.
,
McSherry
G.
,
McShane
H.
,
Hill
A. V.
,
Madhi
S. A.
.
2014
.
The association between the ratio of monocytes:lymphocytes at age 3 months and risk of tuberculosis (TB) in the first two years of life.
BMC Med.
12
:
120
.
33
Rakotosamimanana
N.
,
Richard
V.
,
Raharimanga
V.
,
Gicquel
B.
,
Doherty
T. M.
,
Zumla
A.
,
Rasolofo Razanamparany
V.
.
2015
.
Biomarkers for risk of developing active tuberculosis in contacts of TB patients: a prospective cohort study.
Eur. Respir. J.
46
:
1095
1103
.

The authors have no financial conflicts of interest.

Supplementary data