Two approaches for estimating the lower limit of quantitation (LLOQ) of microRNA levels assayed as exploratory biomarkers by RT-qPCR
© The Author(s). 2018
Received: 31 May 2017
Accepted: 4 January 2018
Published: 2 February 2018
Circulating microRNAs are undergoing exploratory use as safety biomarkers in drug development. Reverse transcription quantitative polymerase chain reaction (RT-qPCR) is one common approach used to quantitate levels of microRNAs in samples that includes the use of a standard curve of calibrators fit to a regression model. Guidelines are needed for setting assay quantitation thresholds that are appropriate for this method and to biomarker pre-validation.
In this report, we develop two workflows for determining a lower limit of quantitation (LLOQ) for RT-qPCR assays of microRNAs in exploratory studies. One workflow is based on an error threshold calculated by a logistic model of the calibration curve data. The second workflow is based on a threshold set by the sample blank, which is the no template control for RT-qPCR. The two workflows are used to set lower thresholds of reportable microRNA levels for an example dataset in which miR-208a levels in biofluids are quantitated in a cardiac injury model. LLOQ thresholds set by either workflow are effective in filtering out microRNA values with large uncertainty estimates.
Two workflows for LLOQ determinations are presented in this report that provide methods that are easy to implement in investigational studies of microRNA safety biomarkers and offer choices in levels of conservatism in setting lower limits of acceptable values that facilitate interpretation of results.
MicroRNAs (miRNAs) that have restricted tissue expression and are released into biofluids upon tissue injury are being evaluated as safety biomarkers potentially diagnostic of site of injury. Tissue-selective miRNAs have been identified by profiling the miRNA content of bodily tissues in humans, mice, and rats [1–3]. Reverse transcription quantitative polymerase chain reaction (RT-qPCR) assays have been adapted for the detection of very low levels of miRNAs that are typically found in biofluids, especially in unaffected controls. For RT-qPCR reactions, the relative measure of the concentration of a miRNA target is the quantification cycle (Cq), the fractional cycle number at which a PCR amplification curve crosses a threshold line set within the exponential growth region of the amplification curve. If the amplification curve for an analyte never crosses the threshold line within 40–42 cycles of amplification, the analyte level is considered too low to be quantitated and the Cq is “undetermined.” Cq values are influenced by many factors, so they are not commutable values and cannot be directly compared between PCR reactions run under different conditions. Reporting of miRNA changes by relative quantitation of RT-qPCR data (normalizing Cq values to a control or reference sample Cq) is not an optimal approach because baseline levels of many candidate tissue-selective miRNAs are very low or undetectable in biofluids and because there is no consensus set of small RNAs in serum or plasma that can be widely used for normalization . Although not widely used in the published literature, absolute quantitation of miRNAs in biofluids is a documented method  that has application for reporting results in the exploratory stages of biomarker development. A common approach for measurement of miRNA biomarker candidates by absolute quantitation is the use of RT-qPCR to assay standard curves of synthetic RNA calibrators in parallel to samples for interpolation of unknowns. Analytical validation guidelines recommend the use of at least 6 non-zero calibrators per calibration curve . Calibration curves should cover the dynamic range of the assay, which is a minimum of three orders of magnitude for PCR and ideally five to six . miRNA can also be quantified using droplet digital PCR, an alternate PCR-based approach that doesn’t rely on standard curves [8, 9] and is just beginning to be applied to miRNA safety biomarker assessments.
During the exploratory stage of miRNA biomarker development, guidelines are needed for determining assay thresholds that are appropriate for this method and purpose since circulating levels of tissue-selective miRNA can be at very low to potentially zero baseline levels. The lower limit of quantitation (LLOQ) is the lowest amount of an analyte in a sample that can be quantitatively determined with suitable precision and accuracy. Bioanalytical method validation approaches recommended for pharmacokinetic studies are a good starting point for biomarker measurements but are not necessarily fully applicable . For bioanalytical assays, the LLOQ is typically determined based on the reagent blank response (e.g., at least five times higher) and a threshold for acceptable precision . For miRNA measurements, approaches have been used that define an LLOQ based on the noise around the NTC values, in combination with the slope or linear region of a calibration curve. Kelnar et al. have defined a PCR-specific LLOQ as the NTC Cq value minus a factor (10 times the standard deviation of the NTC replicates divided by the negative slope of the standard curve) . Hindson et al. defined an LLOQ as the lowest concentration tested that remained above or equal to both the lower limit of the linear range and the limit of detection (LOD) . The linear range was determined by runs-testing and the LOD by the mean NTC Cq plus 2.479 times the standard deviation of NTC Cq values. Hughes et al. defined an LLOQ as the lowest calibration curve point with a mean Cq value less than the mean NTC Cq that, if included, does not cause the calibration curve to fall outside a range of 90–110% for PCR efficiency . A simple approach recommended for use with Exiqon miRCURY LNA Universal RT-qPCR assays for miRNA defines the assay background as 5 Cq units below the NTC Cq or below 42 if the NTC signal is undetected .
In response to a current lack of consensus methods for LLOQ determinations that can be used for absolute quantitation of miRNA by RT-qPCR in investigational studies, we evaluated several different methods that are relatively simple and intuitive, and based on principles established for bioanalytical assays. Workflows for each approach were developed using a test set of 65 miR-1 calibration curves that included 10–12 serial dilutions and covered four orders of magnitude at 3-fold intervals (see Additional files 1, 2, 3, 4). This diverse set of calibration curves was generated at six sites following standard protocols that included variations in the degree of multiplexing and incorporated differences in linearity, slope, and detection of background signals in NTC samples. LLOQ determinations were limited to measured points and were calculated on a per-run basis to match projected future workflows. The two workflows that were developed are described further below.
LLOQ determination based on logistic modeling
LLOQ determination based on baseline noise
Quantitation limits for analytical assays have also been based on signal-to-noise ratio . A signal-to-noise ratio of 10 is a typical constant used to define the minimum concentration at which an analyte can be reliably quantified. We tested one approach that is based on defining an LLOQ as the nearest measured point that is ten-fold higher than the baseline noise level, with baseline noise being defined as the signal in the NTC sample in RT-qPCR assays. If there is no measurable background noise in the samples (i.e., Cq values for the NTC sample are assigned an “undetermined” call by the qPCR instrument software), the baseline signal could alternately be defined by the lowest detectable signal determined by the calibrators.
Comparison of LLOQ values determined using two workflows with a multi-site calibration curve dataset
Application of LLOQ workflows for setting thresholds on miRNA values in an RT-qPCR absolute quantitation data set
As part of a study on inter-laboratory variance in the absolute quantitation of miRNAs in biofluids using RT-qPCR, it became evident that there was a lack of clarity on how to establish a LLOQ for this type of data. It is useful to identify approaches that can be retrospectively applied to the types of datasets that were generated for this multi-site study. For example, in the standard protocol for the multi-site study, technical replicates were run at the PCR step but not at the RT step. LLOQ approaches that are based on error around the NTC Cq values would not be applicable to these datasets because both RT and PCR steps need to be included in the error estimate. The two workflows described in this report were developed using principles established for bioanalytical assays. The approaches are based on the logistic modelling of calibration curve data or the NTC Cq values for LLOQ determinations. When applied to a large dataset of calibration curves that were designed to exceed the linear range of the RT-qPCR assays, the two workflows demonstrated general utility, albeit with some differences in conservatism for calculating a LLOQ. These approaches for LLOQ determinations should be applicable to other types of nucleic acid measurements that use calibration curves for quantitation including injury-related miRNAs at sites other than heart and miRNA within tissues and cell lines.
Levels of cardiac-selective miR-208a-3p were measured in biofluids as part of a multi-site study with the expectation based on published literature [16, 17] that low levels would be observed in control plasma and increased levels observed in plasma within 4–24 h of administration of a subcutaneous dose of isoproterenol that induces moderate cardiac injury in male Wistar rats. The levels of miR-208a-3p in the urine of control rats or rats with isoproterenol-induced cardiac injury were an unknown. In this study, we observed that control plasma levels of miR-208a-3p were relatively low and varied by 1000-fold among sites and/or runs, and that the lowest levels (<10 copies/μL biofluid) were associated with large error estimates predicted by the 3-parameter logistic model. When LLOQ thresholds were applied to this dataset that were based on the logistic standard error workflow, it follows that the miR-208a-3p predicted values for unknowns with large confidence intervals in control plasma and in urine are below the LLOQ. The baseline noise workflow for LLOQ determinations that sets a threshold at an interval below the NTC value has a similar effect in filtering out the lower miR-208a-3p estimates with high error estimates. Although there is a rationale for retaining values below the LLOQ to report estimates instead of no results , this use case illustrates why error estimates should be calculated and provided for inverse predictions of miRNA copy number concentrations for exploratory biomarker studies.
We developed and tested two workflows that provide guides for LLOQ determinations and fill a gap in methods that are appropriate for exploratory use of miRNAs as safety biomarkers in drug development. These two workflows are summarized in Additional file 8. We include sample data sets (Additional files 1, 2, 3, 4 and 6) and results tables (Additional files 5 and 7) to allow investigators to train themselves in the use of these guides.
Synthetic microRNA standards used in the test set
Concentrated stocks of rno-miR-1-3p, rno-miR-16-5p, rno-miR-208a-3p, and rno-miR-499-5p RNA were synthesized, HPLC-purified, quantified, and mixed by a contracting lab (IDT, Coralville, IA), and distributed to test sites as a pooled stock solution comprised of equimolar amounts of four miRNAs. A dilution series was prepared by ten 3-fold serial dilutions of a 1 pmol/L stock of the equimolar pool to span an input range of 1 pmol/L to 17 amol/L. The calibration curve dilutions were prepared in water with carrier non-mammalian RNA (MS bacteriophage RNA, Roche Diagnostics) at a concentration of 0.5 ng/μL.
RT-qPCR protocol for the test set
The test set protocol includes three arms that involve three different degrees of primer pool multiplicity in the reverse transcription and preamplification steps (singleplex, 4-plex, and Megaplex). A single RT reaction was run per arm for 12 samples: an 11-point serial dilution curve of synthetic rno-miR-1-3p and a no template control. The RT reactions were preamplified in replicate on 3 separate days, followed by qPCR. The singleplex assay contained TaqMan microRNA assay reagents for rno-mir-1 (Assay No. 002064). Each 15 μL RT reaction included 0.15 μL 100 mmol/L dNTPs, 1.5 μL 10X RT Buffer, 0.19 μL RNase inhibitor (20 U/μL), 6.16 μL H2O, 3 μL 5× RT primer, 1 μL Multiscribe reverse transcriptase, and 3 μL calibrator RNA or water, and reactions were run in a thermal cycler at 16 °C, 30 min; 42 °C, 30 min; 85 °C, 5 min; 4 °C, hold. The singleplex RT reaction was either preamplified and analyzed by qPCR or analyzed directly by qPCR. Preamplification reactions had 2.5 μL RT reaction, 12.5 μL 2× TaqMan Pre-AMP Master Mix, 6.25 μL 0.2× miR-1 TaqMan assay, and 3.75 μL H2O in a total volume of 25 μL. The reactions were run in a thermal cycler for 14 cycles of [95 °C, 15 s; 60 °C, 4 min] and then placed immediately on ice. The preamplification reaction products were diluted 1:20 in 0.1× TE and 5 μL of diluted preamplification product, undiluted RT reaction, or 1:200 diluted RT reaction was added to a 20 μL qPCR reaction containing 1 uL 20× TaqMan miR-1 microRNA assay, 10 μL 2× Universal Master Mix II no UNG, and 4 μL H2O. Reactions were run at 95 °C for 10 min, followed by 40 cycles of [95 °C, 15 s; 60 °C, 60 s] in an Applied Biosystems 7900HT or ViiA7 qPCR instrument. The same qPCR run parameters were used for the singleplex, 4-plex, and Megaplex qPCR reactions.
The 4-plex RT reaction used pooled RT primers from rno-miR-1-3p, rno-miR-208a-3p, rno-miR-499-5p, and rno-miR-192-5p TaqMan microRNA assays. The 15 μL 4-plex RT reactions consisted of 0.15 μL 100 mmol/L dNTPs, 1.5 μL 10× RT Buffer, 0.19 μL RNase inhibitor (20 U/μL), 3.16 μL H2O, 6 μL 1.25× pooled RT primer, 1 μL Multiscribe reverse transcriptase, and 3 μL calibrator RNA, and were run in a thermal cycler at 16 °C, 30 min; 42 °C, 30 min; 85 °C, 5 min; 4 °C, hold. The 4-plex RT reaction products (2.5 μL) were added to a 25 μL preamplification reaction containing 12.5 μL 2× TaqMan Pre-AMP Master Mix, 6.25 μL 0.2× pooled TaqMan assays for rno-miR-1-3p, rno-miR-208a-3p, rno-miR-499-5p, and rno-miR-192-5p, and 3.75 μL H2O in a total volume of 25 μL. The reactions were run in a thermal cycler for 14 cycles of [95 °C, 15 s; 60 °C, 4 min] and placed immediately on ice. The 4-plex preamplification products were diluted 1:20 and 5 μL was combined with 1 uL 20× TaqMan miR-1 microRNA assay, 10 μL 2× Universal Master Mix II no UNG, and 4 μL H2O.
The Megaplex RT reactions combined 0.8 μL 10× Megaplex RT primers for Rodent Pool A (Catalog No 4399970), 0.2 μL 100 mmol/L dNTPs, 1.5 μL Multiscribe reverse transcriptase, 0.8 μL 10X RT Buffer, 0.9 μL 25 mmol/L MgCl2, 0.1 μL RNase inhibitor (20 U/μL), 0.2 μL H2O, and 3 μL calibrator RNA. The RT reactions were incubated on ice for 5 min and run in a thermal cycler with a heated lid for 40 cycles of [16 °C, 2 min; 42 °C, 1 min; 50 °C, 1 s] followed by 85 °C, 5 min and held at 4 °C. Each Megaplex preamplification reaction contained 12.5 μL 2× TaqMan Pre-AMP Master Mix, 2.5 μL 10× Megaplex PreAMP primers for Rodent Pool A (Catalog No 4399203), 7.5 μL H2O, and 2.5 μL Megaplex RT reaction product. The Megaplex preamplification reactions were run in a thermal cycler at 95 °C, 10 min; 55 °C, 2 min; 72 °C, 2 min, followed by 12 cycles at [95 °C, 15 s; 60 °C, 4 min], followed by 99.9 °C, 10 min and held at 4 °C. Each 25 μL reaction was diluted by adding 75 μL 0.1× TE pH 8.0. The Megaplex preamplification products were further diluted 1:20 and 5 μL was assayed by qPCR by adding 1 μL 20X TaqMan miR-1 microRNA assay, 10 μL 2× Universal Master Mix II no UNG, and 4 μL H2O.
Site differences in the test set
Six sites participated in the study used for the test set. All six ran the singleplex and 4-plex arms and four sites also ran the Megaplex arm. Three different brands of thermal cycler were used by the six sites for the RT and preamplification steps. Three sites used the Applied Biosystems 7900HT system and three sites used the Applied Biosystems ViiA7 system for qPCR steps. The one reported variable within a site was the use of three different lots of TaqMan PreAmp MasterMix by Site C.
Calculations were performed in Excel and JMP, and graphics were created in Excel, GraphPad, and JMP. A free JMP add-in called “Calibration Curves” is available at https://community.jmp.com/docs/DOC-6285 to aid in fitting 3PL curves and performing inverse prediction.
We thank the microRNA best practices working group of the HESI Technical Committee on Application of Genomics to Mechanism-based Risk Assessment for providing the independent inter-laboratory study dataset.
This HESI scientific initiative is primarily supported by in-kind contributions (from public and private sector participants) of time, expertise, and experimental effort. These contributions are supplemented by direct funding (that largely supports program infrastructure and management) that was provided by HESI’s corporate sponsors. A list of supporting organizations (public and private) is available at http://hesiglobal.org/application-of-genomics-to-mechanism-based-risk-assessment-technical-committee/.
Availability of data and materials
The datasets supporting the conclusions of this article are included within the article and its additional files.
The statements, opinions, and conclusions expressed herein do not necessarily represent the views or positions of the authors’ institutions. Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by any of the authors’ institutions, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose.
RDW, EB, TC, PC, HEZ, PM, PSP, TS, PSY, and KLT developed the concept of the report. HEZ, SB, TC, JY, TS, PC, GG, CM, and KLT provided data for the test set. RDW developed the logistic model approach and KLT developed the baseline noise method. RDW and KLT drafted the manuscript. ROL provided overall program management. All authors participated in the revision process and provided final approval.
Ethics approval and consent to participate
All animal studies were performed in compliance with the European Directive 2010/63/EU on the Protection of Animals used for Scientific Purposes.
Consent for publication
PM is an employee of Exiqon, which is a company that is a developer and producer of qPCR systems for detection of coding and noncoding RNAs. RDW is an employee of SAS Institute, which develops software for bioinformatics analysis. The other authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Landgraf P, Rusu M, Sheridan R, Sewer A, Iovino N, Aravin A, et al. A mammalian microRNA expression atlas based on small RNA library sequencing. Cell. 2007;129(7):1401–14.View ArticleGoogle Scholar
- Minami K, Uehara T, Morikawa Y, Omura K, Kanki M, Horinouchi A, Ono A, Yamada H, Ohno Y, Urushidani T. miRNA expression atlas in male rat. Sci Data. 2014;1:140005.View ArticleGoogle Scholar
- Smith A, Calley J, Mathur S, Qian H-R, Wu H, Farmen M, Caiment F, Bushel PR, Li J, Fisher C, Kirby P, Koenig E, Hall DG, Watson DE. The rat microRNA body atlas; evaluation of the microRNA content of rat organs through deep sequencing and characterization of pancreas enriched miRNAs as biomarkers of pancreatic toxicity in the rat and dog. BMC Genomics. 2016;17:694.View ArticleGoogle Scholar
- Roberts TC, Coenen-Stass AML, Wood MJA. Assessment of RT-qPCR normalization strategies for accurate quantification of extracellular microRNAs in murine serum. PLoS One. 2014;9(2):e89237.View ArticleGoogle Scholar
- Kroh EM, Parkin RK, Mitchell PS, Tewari M. Analysis of circulating microRNA biomarkers in plasma and serum using quantitative reverse transcription-PCR (qRT-PCR). Methods. 2010;50(4):298–301.View ArticleGoogle Scholar
- Guidance for Industry, Bioanalytical Method validation, U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, September 2013, Rev 1.Google Scholar
- Bustin SA, Benes V, Garson JA, Hellemans J, Huggett J, Kubista M, Mueller R, Nolan T, Pfaffl MW, Shipley GL, Vandesompele J, Wittwer CT. The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments. Clin Chem. 2009;55(4):611–22.View ArticleGoogle Scholar
- Hindson CM, Chevillet JR, Briggs HA, Gallichotte EN, Ruf IK, Hindson BJ, Vessella RL, Tewari M. Absolute quantification by droplet digital PCR versus analog real-time PCR. Nat Methods. 2013;10:1003–5.View ArticleGoogle Scholar
- Miotto E, Saccenti E, Lupini L, Callegari E, Negrini M, Ferracin M. Quantification of circulating miRNAs by droplet digital PCR: comparison of EvaGreen- and TaqMan-based chemistries. Cancer Epidemiol Biomark Prev. 2014;23(12):2638–42.View ArticleGoogle Scholar
- Thompson KL, Boitier E, Chen T, Couttet P, Ellinger-Ziegelbauer H, Goetschy M, Guillemain G, Kanki M, Kelsall J, Mariet C, de La Moureyre–Spire C, Mouritzen P, Nassipour R, O’Lone R, Pine PS, Rosenzweig BA, Sharapova T, Smith A, Uchiyama H, Yan J, Yuen PS, Wolfinger R. Absolute measurement of cardiac injury-induced microRNAs in biofluids across multiple test sites. Toxicol Sci. 2016;154(1):115–25.View ArticleGoogle Scholar
- Kelnar K, Peltier HJ, Leatherbury N, Stoudemire J, Bader AG. Quantification of therapeutic miRNA mimics in whole blood from nonhuman primates. Anal Chem. 2014;86:1534–42.View ArticleGoogle Scholar
- Hughes S, Byrne A, Barfield M, Spooner N, Summerfield S. Preliminary investigation into the use of a real-time PCR method for the quantification of an oligonucleotide in human plasma and the development of novel acceptance criteria. Bioanalysis. 2014;6:127–36.View ArticleGoogle Scholar
- Exiqon miRCURY LNA™ Universal RT microRNA PCR. Instruction manual v5.2, March 2013.Google Scholar
- Gallant AR. Nonlinear statistical models. New York: John Wiley & Sons; 2009.Google Scholar
- Guidance for Industry, Q2B Validation of Analytical Procedures: Methodology, U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, November 1996.Google Scholar
- Glineur SF, De Ron P, Hanon E, Valentin JP, Dremier S, Nogueira da Costa A. Paving the route to plasma miR-208a-3p as an acute cardiac injury biomarker: preclinical rat data supports its use in drug safety assessment. Toxicol Sci. 2016;149:89–97.View ArticleGoogle Scholar
- Wang G-K, Zhu J-Q, Zhang J-T, Li Q, Li Y, He J, Qin Y-W, Jing Q. Circulating microRNA: a novel potential biomarker for early diagnosis of acute myocardial infarction in humans. Eur Heart J. 2010;31(6):659–66.View ArticleGoogle Scholar
- Lee JW, Devanarayan V, Barrett YC, Weiner R, Allinson J, Fountain S, Keller S, Weinryb I, Green M, Duan L, Rogers JA, Millham R, O'Brien PJ, Sailstad J, Khan M, Ray C, Wagner JA. Fit-for-purpose method development and validation for successful biomarker measurement. Pharm Res. 2006;23(2):312–28.View ArticleGoogle Scholar