Skip to main content
  • Methodology article
  • Open access
  • Published:

A simple fluorescence based assay for quantification of human immunodeficiency virus particle release

Abstract

Background

The assembly and release of human immunodeficiency virus (HIV) particles from infected cells represent attractive, but not yet exploited targets for antiretroviral therapy. The availability of simple methods to measure the efficiency of these replication steps in tissue culture would facilitate the identification of host factors essential for these processes as well as the screening for lead compounds acting as specific inhibitors of particle formation. We describe here the development of a rapid cell based assay for quantification of human immunodeficiency virus type 1 (HIV-1) particle assembly and/or release.

Results

Using a fluorescently labelled HIV-derivative, which carries an eYFP domain within the main viral structural protein Gag in the complete viral protein context, the release of virus like particles could be monitored by directly measuring the fluorescence intensity of the tissue culture supernatant. Intracellular Gag was quantitated in parallel by direct fluorescence analysis of cell lysates, allowing us to normalize for Gag expression efficiency. The assay was validated by comparison with p24 capsid ELISA measurements, a standard method for quantifying HIV-1 particles. Optimization of conditions allowed the robust detection of particle amounts corresponding to 50 ng p24/ml in medium by fluorescence spectroscopy. Further adaptation to a multi-well format rendered the assay suitable for medium or high throughput screening of siRNA libraries to identify host cell factors involved in late stages of HIV replication, as well as for random screening approaches to search for potential inhibitors of HIV-1 assembly or release.

Conclusions

The fast and simple fluorescence based quantification of HIV particle release yielded reproducible results which were comparable to the well established ELISA measurements, while in addition allowing the parallel determination of intracellular Gag expression. The protocols described here can be used for screening of siRNA libraries or chemical compounds, respectively, for inhibition of HIV in a 96-well format.

Background

The acquired immunodeficiency syndrome (AIDS), caused by infection with the human immunodeficiency virus (HIV), is a major cause of disease and death worldwide. In the absence of a protective vaccine, prevention of infection and the treatment with antiretroviral drugs remain the only options to prevent the spread of HIV and combat the disease. Current therapy of HIV infection (highly active antiretroviral therapy, HAART) involves treatment with a combination of three or more drugs targeting different steps in HIV replication [reviewed in [1]]. The benefits of HAART are limited by viral resistance development and the transmission and spread of resistant HIV variants. Therefore, additional treatment options targeting alternative steps in the viral replication pathway are needed. The late replication steps of virion assembly and release are not targeted by any of the currently approved antiretroviral drugs and thus present targets for alternative inhibition strategies. The processes of HIV particle assembly and release, as well as the morphological maturation of particles into infectious virions, are orchestrated by the main structural polyprotein Gag. Compounds that do interfere with HIV infectivity in tissue culture by interaction with the Gag polyprotein or its capsid domain (CA), respectively, have been described. The compounds bevirimat [[2]; reviewed in [3]] and CAP-1 [4, 5], do not impair HIV immature particle release, but affect the maturation into infectious particles by binding to a proteolytic processing site in the Gag polyprotein precursor or to the N-terminal domain of the CA protein, respectively. Furthermore, a peptide (CAI), which acts as a bona fide inhibitor of particle assembly from purified HIV structural protein in vitro has been reported [6, 7]. A cell-permeable derivative of this peptide has been shown in a proof of principle study to inhibit HIV replication in tissue culture, albeit with IC50 values in the low to mid μM range and a low selectivity index [8]. Small molecule chemical compounds which specifically inhibit the steps of immature Gag assembly or particle release have not yet been identified. Rational development of such compounds is hindered by the fact that the late stages in the viral life cycle are highly complex processes involving the interaction of viral components with intricate cellular machineries, which are currently only partly understood. It is well established that components of the cellular ESCRT machinery [9], which are recruited by a so-called 'late domain' motif within Gag, are involved in the budding of HIV particles from the host cell [for review see [10]] and a number of additional cellular factors have been implicated to be involved in the late stages of HIV replication [reviewed in [11, 12]]. However, a comprehensive picture of cellular factors and pathways involved in the transport, assembly and release of viral components is currently lacking. Genome-wide siRNA screening approaches have been carried out with the aim of identifying host cell factors essential or inhibitory for HIV replication [13–15]. So far, only two published studies include late stages of virus replication in their analysis, and the results from the different studies showing surprisingly little overlap [16–18]. The availability of simplified cell based assay procedures specifically monitoring the steps of HIV assembly and particle release in tissue culture would greatly facilitate both large-scale siRNA analyses focussing in particular on the late stages of virus replication, as well as the random screening of chemical compound libraries in order to identify lead substances which interfere with HIV particle formation. A well established and widely used method for quantification of HIV particle production is an enzyme linked immunosorbent assay (ELISA) detecting the viral capsid (CA) protein p24 in tissue culture supernatants. However, ELISA measurements display a limited linear range and require many handling steps and expensive reagents. In contrast, measurement of fluorescence intensity is a direct, quantitative readout which can easily be performed in a multi-well format. Thus, we decided to establish a simplified assay based on our previously described fluorescently labelled HIV derivative carrying an eGFP moiety inserted as a separate domain into the main viral structural polyprotein Gag [[19, 20]; Figure 1A]. Cells transfected with the egfp-carrying proviral construct release fluorescent virus like particles in a late domain dependent manner and we have previously used these labelled derivatives to analyze the dynamics of HIV-1 particle formation [21]. Here, we describe a simple and direct assay for monitoring HIV particle release through quantitation of fluorescent VLPs in tissue culture supernatant. Based on this principle, we have established protocols suitable for the screening of either siRNAs or chemical compounds for their interference with HIV assembly and/or release in a multi-well, medium to high throughput format.

Figure 1
figure 1

Characterization of the fluorescently labelled virus derivatives. (A) Map showing the insertion of the eyfp gene within the gag open reading frame of pCHIV. Arrowheads indicate cleavage sites for HIV protease. (B) Fluorescence micrographs of HeLaP4 cells transfected with plasmids peYFP-C1, pCHIVeYFP or pCHIVeYFPlate(-), respectively, recorded at 40 h post transfection. Note that a shorter exposure time was used for cells transfected with peYFP-C1 to adjust for the higher cytoplasmatic fluorescence intensity. (C) Immunoblot analysis of cell lysates and VLPs purified from the supernatant of transfected cells. At 36 h post transfection with peYFP-C1 (lanes 1 and 5), pCHIV (lanes 2 and 6), pCHIVeYFP (lanes 3 and 7) or pCHIVeYFPlate(-) (lanes 4 and 8), respectively, cells were harvested and particles were prepared from the medium by ultracentrifugation through a sucrose cushion. Samples corresponding to equal cell numbers or equal amounts of supernatant were subjected to SDS-PAGE, transferred to a nitrocellulose membrane and proteins were detected by enhanced chemiluminescence using the indicated antisera. Antiserum raised against GFP displayed cross-reactivity against eYFP. (D) Relative release efficiencies of eYFP, HIVeYFP and HIVeYFPlate(-) particles. Relative eYFP fluorescence intensities of tissue culture supernatants and lysates of cells transfected with the indicated plasmids were determined at 44 h post transfection using a TECAN Safire multi-well reader. Values shown here were obtained by subtracting the background derived from untransfected cells and dividing extracellular by total (intracellular plus extracellular) fluorescence. Data shown represent mean values and standard deviations from three parallel transfections.

Results and discussion

Setup of the assay procedure and optimization of assay conditions

The aim of this study was to develop a simple and direct assay for HIV-1 release based on our previously characterized fluorescently labelled HIV derivative [19, 20], which should allow the quantitation of particles in tissue culture supernatants from virus producing cells by direct measurement of fluorescence intensity. Towards this end, we first optimized the detection conditions to increase the sensitivity of the measurements in tissue culture supernatants. Due to intrinsic fluorescence of tissue culture media components at the wavelengths used for detection of fluorescent proteins, optimization of the assay initially required the selection of the fluorophore best suited for detection against medium background. Fluorescence measurements of serial dilutions of bacterially expressed, purified enhanced green, cyan and yellow fluorescent proteins (eGFP, eYFP, eCFP) as well as monomeric red fluorescent protein [22] in PBS and various tissue culture media indicated that enhanced yellow fluorescent protein eYFP was most suitable among these fluorophors for detection against complete medium background. EYFP concentrations above10 ng/ml were detectable over medium background (Additional file 1, Figure S1). Omission of phenol red or fetal calf serum from the medium did not significantly alter the sensitivity of detection.

Based on these findings, we constructed an HIV derivative carrying the eyfp gene within the gag open reading frame, analogous to the previously described construct pCHIVeGFP [20]. The fluorescent protein was inserted between the matrix (MA) and capsid (CA) domains of the Gag polyprotein, upstream of the proteolytic processing site between MA and CA (Figure 1A). Transfection of cells with pCHIV and derivatives results in expression of all viral proteins, except for Nef, under the control of the CMV promoter and to formation and release of entry competent virus like particles (VLPs). The genome encoded by pCHIV is replication incompetent due to deletion of the complete viral long terminal repeat regions essential for reverse transcription and integration.

As shown in Figure 1B, cells transfected with pCHIVeYFP displayed the expected yellow fluorescence. Furthermore, HeLaP4 cells which express the receptor and co-receptors of HIV formed large syncytia upon transfection with pCHIV derived constructs, indicative of HIV Env protein expression. Cell lysates as well as particle preparations from the supernatant of transfected cells were prepared and analyzed by immunoblot (Figure 1C). Analogous to our results obtained with the related construct pKHIVeGFP [19], the Gag.eYFP fusion protein expressed in the viral context was processed upon particle maturation by the viral protease to yield MA.eYFP, and released pelletable particles containing the MA.eYFP fusion protein in a late domain dependent manner (Figure 1C and data not shown). Accordingly, particles should be detectable in the tissue culture supernatant by measurement of eYFP fluorescence intensity.

Specific detection of released particles by fluorescence measurements

The exploitation of fluorescent MA for the quantitation of specific particle release from cells transfected with pCHIVeYFP by fluorescence measurements required appropriate controls. The fluorescence readout - analogous to p24 determination by ELISA - does not necessarily reflect the amount of complete particles released, but rather quantitates the amount of the main particle component Gag present in tissue culture supernatants. Consequently, unspecific release of free Gag, p24 CA or MA.eYFP from damaged or lysed cells could impair the accuracy of results obtained by either ELISA or fluorescence based measurements. To control for unspecific protein release due to cell damage, we used cells expressing unfused eYFP protein, which is not secreted from intact cells, but will be detected in the medium when the sample contains cell debris or cytoplasmic content from lysed cells. Secondly, a positive control for the successful inhibition of release was established. For this purpose, we generated a release deficient variant of the fluorescently labelled HIV derivative by replacing the 'late domain' motif PTAP within the p6 domain of Gag.eYFP with the amino acid sequence LIRL. The late domain is known to mediate interaction with the host cell ESCRT machinery [see [10] for review] and its mutation to LIRL has been previously shown to affect HIV release efficiency [23]. An inhibitory compound which directly blocks viral late domain - ESCRT interaction or an siRNA which downmodulates expression of a late domain interacting host factor can be expected to yield a comparable decrease in release efficiency as the mutation of the late domain, while compounds or siRNAs targeting the Gag assembly process may theoretically accomplish even stronger reduction.

Expression of the Gag.eYFP.late(-) fusion protein and the release deficiency of this construct was confirmed by immunoblot analysis of cell lysate and pelleted tissue culture supernatant (Figure 1C). While comparable amounts of Gag.eYFP and Gag.eYFP.late(-) were detected in cell lysates (compare lanes 3 and 4), the amount of Gag derived products in pelleted particles was significantly decreased in the case of the late-variant (compare lanes 7 and 8). Free eYFP was expressed in transfected cells (lane 1), but not detected in pelleted supernatant (lane 5). We then determined the relative release efficiency of Gag.eYFP.late(-) by fluorescence measurements.

Figure 1D shows the result of a comparison of release efficiencies between 293T cells transfected with plasmids pEYFP-C1, pCHIV and pCHIVeYFPlate(-). At 48 h post transfection tissue culture supernatants were harvested and cells were lysed by detergent treatment as described in methods. Supernatants from particle producing cells displayed typical spectroscopic characteristics of eYFP with excitation and emission maxima at 512 nm and 527 nm (Additional file 2, Figure S2). Relative release efficiencies were calculated by dividing fluorescence intensities measured in tissue culture supernatants by the corresponding total fluorescence intensities measured in supernatants and cell lysates. Cells transfected with plasmid peFYP-C1 displayed high intracellular fluorescence but the relative amount of eYFP released into the supernatant was low in comparison to cells transfected with pCHIVeYFP. Cells expressing the late domain defective variant of the labelled virus reproducibly showed an approximately five-fold reduction of relative release efficiency as compared to cells expressing the wild-type variant.

Sensitivity of measurements

Our previous characterization of eGFP labelled HIV derivatives had revealed that - while bulk release efficiency of labelled virus was comparable to wild-type - cells expressing the labelled HIV derivative displayed electron dense aggregates at the plasma membrane, suggesting altered release kinetics; furthermore released particles displayed reduced infectivity. Full wild-type particle infectivity and budding site morphology was restored by co-transfection of an equimolar amount of wild-type HIV proviral plasmid, leading to the formation of mixed particles carrying 50% of labelled Gag [19]; assuming an average number of approximately 2400 molecules of Gag per particle [24], particles contain thus approximately 1200 molecules of fluorescent protein. For this reason, equimolar mixtures of eYFP labelled proviral derivatives with the corresponding non-labelled provirus were used in all further experiments.

In order to determine the sensitivity of fluorescence based particle quantification, VLPs were prepared from the tissue culture supernatant of cells co-transfected with an equimolar mixture of pCHIV and pCHIVeYFP by ultracentrifugation through a sucrose cushion. The amount of CA p24 present in the particle preparation was determined by ELISA, which represents a currently used standard method for HIV particle quantification. Since both eYFP and p24 CA are expressed from pCHIVeYFP as parts of the Gag polyprotein, they are translated and packaged into VLPs at equimolar amounts; in the case of the mixed particles used here, one molecule of eYFP will thus correspond to two molecules of CA. Serial dilutions of fluorescent particles in complete medium were set up and their relative eYFP fluorescence intensity was determined (Figure 2). The lowest particle concentration detectable above background by fluorescence measurements in complete medium corresponded to approximately 15-30 ng CA/ml (equivalent to ~8-15 ng eYFP/ml), which was in good agreement with the results from measurements using purified fluorescent proteins. Omission of FCS or phenol red from the medium or replacement of the medium by PBS only resulted in a minor increase in sensitivity (data not shown). From these and similar data sets we concluded that robust fluorescence intensity quantitation can be performed for particle concentrations corresponding to 50 ng p24/ml or higher. This sensitivity is significantly lower than that of advanced ELISA based detection systems [25]. However, this does not affect the usefulness of the system for the intended applications, since the amounts of particles released under optimized assay conditions were in a range easily detectable by fluorescence measurements (see below).

Figure 2
figure 2

Titration of eYFP labelled VLPs. Mixed particles were purified from the tissue culture supernatant of 293T cells transfected with an equimolar mixture of pCHIV and pCHIVeYFP by centrifugation through a sucrose cushion and their concentration determined by p24 ELISA. The relative eYFP fluorescence intensity of serial dilutions in DMEM/10% FCS was determined using a TECAN Saphire multi-well reader. Background fluorescence from the medium was subtracted and fluorescence intensities of the samples were plotted as a function of p24 CA concentration. The plot shows mean values and standard deviation from triplicate dilutions; the line represents a linear regression. The inset shows an expansion of the lower part of the same data set. The line represents a linear regression to the data points shown.

Assay validation and time course of particle release

We then analysed the time course of fluorescence release from cells transfected with equimolar mixtures of either wild-type or late domain defective variants of pCHIV/pCHIVeYFP, respectively. Samples of tissue culture supernatant were removed at consecutive time points between 24 and 48 h post transfection and subjected to fluorescence measurements. Cells transfected with peYFP-C1 were used as a negative control. In order to investigate whether budding of HIV from the membrane could promote the release of free eYFP from cells, we included cells transfected with a mixture of pCHIV and peFYP-C1 as an additional control. At 48 h post transfection, cells were lysed by detergent treatment and the fluorescence intensity of the lysate determined in order to normalize for transfection efficiency and Gag or eYFP expression in the different samples. As shown in Figure 3A, fluorescence in the supernatant above background was detectable at 24 h post transfection with pCHIV/pCHIVeYFP (filled circles) and increased over the following hours. Again, release from the late(-) variant (open circles) was found to be diminished. While fluorescence in the supernatant of peYFP-C1 transfected cells (open triangles) remained at background levels, release of low levels of free eYFP was promoted in the presence of HIV expression (filled triangles). This might be explained by unspecific packaging of cytosolic eYFP into budding virus particles, since we observed that in this case release was also influenced by the presence of a functional HIV late domain (data not shown).

Figure 3
figure 3

Time course of labelled VLP release cells determined by fluorescence measurement or ELISA, respectively. 293T cells were transfected with mixtures of pCHIV/pCHIVeYFP (filled circles) or pCHIVlate(-)/pCHIVeYFPlate(-) (open circles), respectively. Cells transfected with peYFP-C1/empty vector (open triangles) or peYFP/pCHIV (filled triangles) served as controls. At the indicated time points after transfection, tissue culture supernatants were sampled and analyzed in parallel by fluorescence measurements (A) or ELISA (B), respectively, as described in methods. To control for transfection efficiency, cell lysates prepared at the end of the experiment were subjected to the same analyses. Normalized release efficiency was determined by subtracting the background derived from untransfected cells and dividing extracellular fluorescence or p24 CA versus intracellular fluorescence or p24 CA determined at 48 h post transfection, respectively. Data represent mean values and standard deviations from triplicate transfections. (C) Increase of unspecific release upon prolonged incubation of cells. Normalized release values for HIVlate(-)/HIVeYFPlate(-) determined in the experiment shown in Figure 3A at the indicated times post transfection were divided by the respective values obtained for HIV/HIVeYFP.

Results obtained from fluorescence measurements were validated by determination of p24 CA in the same samples by ELISA. Again, we calculated the amount of p24 in the supernatant relative to the amount measured in cell lysates at 48 h. Results are shown in Figure 3B. Similar concentrations of p24 were detected in supernatants from cells transfected with pCHIV/pCHIVeYFP (filled circles) and with pCHIV/peYFP-C1 (filled triangles), demonstrating that insertion of eYFP within Gag did not affect release efficiency; p24 concentrations measured corresponded to ~70 ng/ml at 24 h post transfection and ~0.5 μg/ml at the end of the experiment, respectively. Evaluation by ELISA yielded similar results for pCHIV/pCHIVeYFP (filled circles) and pCHIVlate(-)/pCHIVeYFPlate(-) (open circles) as the fluorescence based assay (compare Figure 3A and Figure 3B), confirming the validity of the fluorescence assay method. A slightly larger deviation between parallel samples was observed in the ELISA experiment, presumably because this procedure involved more handling steps. The observation that the relative ratio of intra- to extracellular fluorescence differs somewhat between the two types of measurement is explained by the fact that unprocessed Gag (which represents the predominant form in cell lysates) is recognized less well than processed CA (the predominant form in particles), by immunoprecipitation [26] or ELISA. Even prior heat and detergent treatment of samples, which has been reported to make unprocessed Gag accessible to ELISA detection [27], did not confer full reactivity of the precursor form in our hands. Thus, data derived from the ELISA measurements are less suitable for reliable normalization of CA levels relative to intracellular Gag expression levels. In contrast, in vitro cleavage of immature fluorescent particles using purified PR did not result in a change in fluorescence intensity, demonstrating that the fluorescence based quantitation of Gag was independent of its processing status (data not shown). We concluded that the assay described here is suitable to monitor HIV-1 particle release and can be used to test inhibition of HIV-1 particle formation in tissue culture.

In order to define conditions yielding efficient late domain dependent release accompanied by low levels of unspecific release we compared the ratio between release of the wild-type and late(-) virus variants at the different time points. The ratio of late(-) particles released compared to wild-type particles increased at later time points, presumably due to unspecific Gag release from damaged or dead cells upon prolonged protein over-expression (Figure 3C and data not shown), indicating that harvest times of 33-36 h post transfection were optimal under the conditions used.

Adaptation of the assay to medium throughput formats

Further adaptation of the assay to a semi-automated multi-well format was performed to render the assay suitable for random screening. Two different protocols were established suitable for screening of chemical compound libraries or for siRNA screening approaches, respectively. For the purpose of compound screening we established a protocol for the bulk transfection of 293T cells. Briefly, 293T cells were harvested and transfected in suspension using equimolar mixtures of pCHIV/pCHIVeYFP, pCHIVlate(-)/pCHIVeYFPlate(-), or plasmid peYFP-C1 alone, respectively. Cell suspensions were then distributed into wells of a 96-well plate or 384-well plate, respectively, using an automated reagent dispenser and plates were incubated at 37°C. The addition of chemical compounds dissolved in DMSO was mimicked by the addition of a final concentration of 0.5% DMSO to the medium at 14 h post transfection. At 36 h post transfection, plates were briefly spun to pellet potential cell debris. Aliquots of tissue culture supernatants were transferred to fresh plates using an automated liquid handler, the remaining supernatant was removed from the cell layer and cells were lysed on the plate as described in methods. Relative fluorescence intensities of supernatants and cell lysates were determined using a TECAN Safire multi-well reader. Supernatants and lysates from untransfected cells were used to determine background fluorescence. As shown in Figure 4 and Additional file 3, Figure S3, this procedure yielded robust results in 96-well plates. A limited data set of 60 positive (cells transfected with pCHIV/pCHIVeYFP, 0.5% DMSO) and 36 negative (untransfected cells) control wells from a pilot screen including 6 replicate plates was used to estimate statistical effect size, yielding a Z factor [28] of 0.71 (Additional file 3, Figure S3). Mean fluorescence intensities in the supernatant of 176 wells from 88 plates (Figure 4A, grey bars) showed low plate-to-plate variability. The amount of eYFP labelled protein released into the supernatant by the late domain defective variant was reduced to 16% of the wild-type control. Very similar results were obtained when fluorescence intensities of the supernatant normalized to the total fluorescence measured in the supernatant and cell lysate were compared (Figure 4A, black bars). Thus, normalization for intracellular protein expression was not necessary to improve the accuracy of measurements in a standardized 96-well setup. In the context of a random screening assay, however, the parallel determination of intracellular eYFP intensity in a screening assay can serve to identify compounds which do not specifically interfere with HIV particle formation but rather exert a general effect on protein expression. Comparison of relative release efficiencies determined for labelled wild-type particles across rows 2-11 of 96-well plates in the absence of inhibitory compounds did not reveal any systematic aberration across the plate (Figure 4B). We conclude that the assay allows for medium throughput screening of potential inhibitors of HIV-1 assembly or release.

Figure 4
figure 4

Adaptation of the assay to a multi-well format. (A) Reproducibility of controls. 293T cells transfected with an equimolar mixture of pCHIV and pCHIVeYFP (wild-type) or their late domain defective variants (late(-)) were seeded into 96-well plate. Fluorescence intensities of tissue culture supernatants (grey bars) or release efficiencies (supernatant fluorescence intensities divided by total fluorescence intensity (supernatant and cell lysate; black bars) were determined as described in methods. Bars represent mean value and standard deviation of 176 wells from 88 plates. (B) Variability across a 96-well plate. 293T cells transfected with an equimolar mixture of pCHIV and pCHIVeYFP were seeded into 96-well plate. 0.5% of DMSO was added to the medium at 14 h post transfection to mimic the addition of compounds dissolved in DMSO. Supernatants and cell lysates were harvested at 36 h post transfection and relative release efficiencies were determined by dividing extracellular by total fluorescence and normalized by dividing through the median of release efficiencies from all wells of the respective plate. Mean values and standard deviations from three 96-well plates are shown. (C) Variability across a 384-well plate. 293T cells transfected with a mixture of pCHIV/pCHIVeYFP (black bars) or their late domain defective variants (grey bars), respectively, were seeded into a 384-well plate. Relative fluorescence intensities of the supernatants were measured at 34 h post transfection. Mean values and standard deviations from 24 wells each are shown.

Transfer of the protocol to a 384-well format yielded less robust results (Figure 4C). Due to difficulties in reproducible preparation and measurement of cell lysates in the small well format, normalization of supernatant fluorescence to total fluorescence led to larger variability between wells (data not shown). While the assay in its present form could be used to determine relative amounts of Gag released into the supernatant using a 384-well format, further optimization would be required for a reliable parallel quantification of intracellular Gag.

Finally, a modified protocol was established to be used in screening of siRNA libraries for factors involved in HIV release. For this, 293T cells were distributed onto 96-well plates which had been pre-coated with an siRNA containing transfection mix using a previously described reverse transfection protocol [29, 30]. Dried pre-coated plates could be either used directly for knockdown experiments or be stored for up to 15 months without any loss of efficacy, thus ensuring improved reproducibility and comparability between different plates of the same batch as compared to liquid transfection of siRNA. 293T cells were seeded onto these plates and incubated in order to achieve target gene knockdown, followed by transfection with pCHIV/pCHIVeYFP and further incubation. In order to validate this approach we performed siRNA mediated knockdown of the cellular ESCRT protein Tsg101, which has been previously shown to impair HIV release [31]. At the conditions used, prolonged knock-down of Tsg101 resulted in reduced cell viability and cell death; to avoid this, we chose a relatively short incubation period of 18 h for the initial transfection with siRNA, followed by transfection with pCHIV/pCHIVeYFP. Incubation was then continued for 36 h before read out of fluorescence intensities in supernatant and cell lysates was carried out. As shown in Figure 5A, siRNA mediated knock-down of Tsg101 resulted in an approximately 4-5 fold reduction in release efficiency under these conditions, which was only slightly less pronounced than the reduction observed using the late(-) virus variant.

Figure 5
figure 5

Adaptation of the assay for an siRNA screening approach. (A) Reduction of relative release efficiency by knock-down of Tsg101. 96-well plates were coated with non-silencing control siRNA or siRNA targeting Tsg101, respectively, as described in methods. 293T cells were seeded onto those plates for reverse transfection and incubated for 18 h at 37°C. Subsequently, cells were transfected with a mixture of pCHIV/pCHIVeYFP or their late domain defective variants, respectively. Supernatants and cell lysates were harvested at 36 hours after this second transfection and relative fluorescence intensities were determined. Relative release efficiencies (RFIsup/RFItotal) were normalized to the mean of the non-silencing control. Mean values and standard deviations from triplicate transfections are shown; black bars and gray bars, respectively, represent results from two independent experiments. (B) Reproducibility of results obtained in a pilot siRNA screen. The graph shows the correlation between normalized eYFP intensities (eYFP intensity of supernatant divided by total eYFP intensity; RFIsup/RFItotal) of two replicates of an RNAi screen investigating host cell functions in HIV-1 assembly and release, comprising a total of 480 wells in 96-well plates. Each dot represents an individual siRNA mediated gene knockdown. The Pearson's correlation coefficient was calculated using GraphPad Prism.

We have performed a pilot screen, in which we tested an siRNA library (silencer siRNAs, Ambion, Applied Biosystems) targeting approximately 740 human kinases with one individual siRNA per well and three distinct siRNAs per target gene. Since individual siRNAs from this library have not been tested for their silencing properties and time course of gene silencing, we chose longer incubation times of 30 h after the initial transfection with siRNA and 42 h after the second transfection. The results of this screen and the validation of primary hits will be described elsewhere. Exemplary for the reproducibility of results obtained using the established protocol, Figure 5B shows the correlation of relative release values (RFIsup/RFItotal) obtained for two replicate sets of 480 wells from the siRNA screen. The values obtained for RFIsup/RFItotal were slightly higher than those observed for the compound screen setup due to the modified assay procedure. Pearson's correlation coefficients between replicates of identical siRNA sets were generally found to be in the range of 0.7 - 0.8, indicating that the setup described here allowed us to obtain reproducible results in a 96 well format.

Conclusion

We conclude that direct measurement of fluorescence intensities after transfection of cells with a labelled HIV derivative represents a reliable method for monitoring HIV particle release, which is suitable for use in random screening approaches to search for chemical inhibitors or siRNAs specifically interfering with HIV assembly or release.

Methods

Plasmids

Plasmid pCHIVeYFP was constructed by inserting a PCR fragment comprising the complete eYFP coding sequence from peYFP-C1 (Invitrogen) into an engineered ClaI-site between the codons for amino acids 128 and 129 of MA. The cloning strategy for this construct, as well as for its late domain defective variant was analogous the cloning strategy previously described for eGFP labelled virus variants [19, 20]. Plasmids for the expression of recombinant His-tagged eGFP, eYFP and eCFP were generated by cloning PCR fragments comprising the respective coding sequences into the NdeI and BamHI sites of pET16b (Novagen). Plasmid pmRSETmRFP1 was a kind gift of R. Tsien.

Tissue culture and virus preparation

293T and HeLaP4 cells were cultured at 37°C, 5% CO2 in Dulbecco's modified Eagle's medium (DMEM) supplemented with 10% fetal calf serum (FCS), penicillin, streptomycin, 4 mM glutamine and 10 mM Hepes. For viral particle preparation, 293T cells were transfected with pCHIVeYFP or with an equimolar mixture of pCHIVeYFP and pCHIV as indicated by standard calcium phosphate precipitation. Tissue culture supernatants were harvested at 40-44 h post transfection, filtered through 0.45 μm filters and particles were concentrated by ultracentrifugation through a 20% (w/w) sucrose cushion. Virus preparations and cell lysates were separated by SDS-PAGE (17.5%, acrylamide:bisacrylamide 200:1) transferred to nitrocellulose and analyzed by ECL immunoblot (Pierce) using polyclonal rabbit antisera raised against purified recombinant His-tagged HIV MA and GFP, respectively. The antiserum raised against GFP displays cross-reactivity with the GFP derivative eYFP. For quantitation of virus by ELISA, tissue culture supernatants or particle preparations, respectively, were diluted with SDS-Sample buffer (final concentration 0.1% SDS) and incubated for 5 min at 95°C to enhance antibody recognition of unprocessed Gag. Samples were subsequently diluted in PBS/0.1% Tween 20 before incubation with capture antibody. The concentration of CA p24 in the samples was determined by ELISA as previously described [32].

Fluorescence measurements of particle preparations were carried out at room temperature in phosphate buffered saline or DMEM/10% FCS as indicated using a TECAN Safire multi-well reader set to excitation and emission wavelengths of 512 nm and 528 nm, or an Aminco-Bowman SLM-2 spectrophotometer for cuvette based measurements, respectively.

Release assay

293T cells were transfected with pCHIVeYFP, pCHIVeYFPlate(-) or peYFP-C1, respectively, using calcium phosphate or FuGene 6 (Roche) according to standard procedures. At the indicated time points, samples of tissue culture supernatant were removed, briefly centrifuged to remove cell debris (5 min, 3000 g) and subjected to fluorescence measurement. At the end of the indicated incubation period, the remaining medium was removed, cells were briefly washed with phosphate buffered saline (PBS) and incubated in lysis buffer (PBS/0.5% Igepal) at a volume corresponding to the volume of the tissue culture supernatant. Cells were incubated for 10 min at room temperature and cell debris was removed by brief centrifugation (2 min, 1500 g). Fluorescence measurements of supernatants and cell lysates were carried out using an SLM Aminco 2 spectrofluorometer.

For the semi-automated multi-well screening approach, a protocol for batch transfection of cells in suspension was established. 293T cells grown on 10 cm dishes were harvested by trypsinization and adjusted to 3 × 105cells/ml in DMEM without FCS. Per 96-well plate, 960 μl of DMEM without FCS was mixed with 38.4 μl FuGene6 (Roche). Following 5 min incubation at room temperature 19.2 μg of total plasmid DNA were added and the mixture incubated for further 15 min at room temperature. Subsequently, 9.6 ml of the cell suspension were added and 110 μl of the transfection mixture was seeded per well onto 96-well plates (Corning COSTAR #3603) or 25 μl per well onto 384-well plates (BD FALCON 353962, BD Biosciences) using an automated reagent dispenser (MultiDrop® Combi). Plates were incubated at 37°C. At 36 h post transfection, plates were centrifuged (1500 rpm, 8 min) and 87.5 μl of the supernatant was transferred to new 96-well plates (Corning COSTAR #3916) or 20 μl to 384-well plates (BD FALCON 353962) using the compact automated liquid handler Hydra DT (Matrix, Thermo Fisher Scientific). The residual supernatant was removed and 100 μl (96-well plates) or 25 μl (384-well plates) 0.1% Igepal CA630 (Sigma-Aldrich) in PBS were added to each well, followed by one freeze-thaw cycle to promote cell lysis. Fluorescence intensities of supernatants and cell lysates were determined using a multi-well fluorescence reader (Tecan SAFIRE).

Screening of an siRNA library required several modifications of this procedure. The plates used for seeding the cells were pre-coated with an siRNA transfection mixture using a previously described reverse transfection protocol [29, 30] and liquid handling platform [33]. Briefly, a mixture of the respective siRNA, the transfection reagent Lipofectamine 2000 (Invitrogen), fibronectin (Sigma-Aldrich), sucrose and gelatine were transferred automatically to 96-well plates (Corning COSTAR #3603). Silencer® Select Negative Control #1 siRNA (Ambion Applied Biosystems) was used as a non-silencing control. For the knock-down of Tsg101 we used siRNA with the sequence 5'CCUCCAGUCUUCUCUCGUC3' selected according to Garrus et al. [31]. 293T cells were seeded on top of the dried transfection mixture (7500 cells/well in 100 μL D-MEM 10% FCS) using the reagent dispenser MultiDrop® Combi. Following a 30 hour incubation period at 37°C (18 h for Tsg101, respectively) for target gene knockdown, medium was removed and cells were transfected with an equimolar mixture of plasmids pCHIV and pCHIVeYFP using FuGene6 (Roche) according to the manufacturer's instructions. Per well, 100 ng plasmid DNA in 100 μl medium were added. After an additional 42 h at 37°C (36 h for Tsg101) tissue culture supernatants and cell lysates were harvested and their RFI values were quantitated as described above.

Abbreviations

HIV:

human immunodeficiency virus

ESCRT:

endosomal complex required for transport

eYFP:

enhanced yellow fluorescent protein

MA:

matrix protein

CA:

capsid protein

ELISA:

enzyme linked immunosorbent assay

RFI:

relative fluorescence intensity

PBS:

phosphate buffered saline

siRNA:

short interfering RNA.

References

  1. Hofmann WP, Soriano V, Zeuzem S: Antiviral combination therapy for treatment of chronic hepatitis B, hepatitis C, and human immunodeficiency virus infection. Handbook of experimental pharmacology. 2009, 321-346. full_text. 189

  2. Li F, Goila-Gaur R, Salzwedel K, Kilgore NR, Reddick M, Matallana C, Castillo A, Zoumplis D, Martin DE, Orenstein JM, Allaway GP, Freed EO, Wild CT: PA-457: a potent HIV inhibitor that disrupts core condensation by targeting a late step in Gag processing. Proceedings of the National Academy of Sciences of the United States of Americ. 2003, 100 (23): 13555-13560. 10.1073/pnas.2234683100.

    Article  CAS  Google Scholar 

  3. Salzwedel K, Martin DE, Sakalian M: Maturation inhibitors: a new therapeutic class targets the virus structure. AIDS reviews. 2007, 9 (3): 162-172.

    Google Scholar 

  4. Tang C, Loeliger E, Kinde I, Kyere S, Mayo K, Barklis E, Sun Y, Huang M, Summers MF: Antiviral inhibition of the HIV-1 capsid protein. Journal of molecular biology. 2003, 327 (5): 1013-1020. 10.1016/S0022-2836(03)00289-4.

    Article  CAS  Google Scholar 

  5. Kelly BN, Kyere S, Kinde I, Tang C, Howard BR, Robinson H, Sundquist WI, Summers MF, Hill CP: Structure of the antiviral assembly inhibitor CAP-1 complex with the HIV-1 CA protein. Journal of molecular biology. 2007, 373 (2): 355-366. 10.1016/j.jmb.2007.07.070.

    Article  CAS  Google Scholar 

  6. Sticht J, Humbert M, Findlow S, Bodem J, Müller B, Dietrich U, Werner J, Kräusslich HG: A peptide inhibitor of HIV-1 assembly in vitro. Nature structural & molecular biology. 2005, 12 (8): 671-677.

    Article  CAS  Google Scholar 

  7. Ternois F, Sticht J, Duquerroy S, Kräusslich HG, Rey FA: The HIV-1 capsid protein C-terminal domain in complex with a virus assembly inhibitor. Nature structural & molecular biology. 2005, 12 (8): 678-682.

    Article  CAS  Google Scholar 

  8. Zhang H, Zhao Q, Bhattacharya S, Waheed AA, Tong X, Hong A, Heck S, Curreli F, Goger M, Cowburn D, Freed EO, Debnath AK: A cell-penetrating helical peptide as a potential HIV-1 inhibitor. Journal of molecular biology. 2008, 378 (3): 565-580. 10.1016/j.jmb.2008.02.066.

    Article  CAS  Google Scholar 

  9. Wollert T, Yang D, Ren X, Lee HH, Im YJ, Hurley JH: The ESCRT machinery at a glance. Journal of Cell Science. 2009, 122 (Pt 13): 2163-6. 10.1242/jcs.029884.

    Article  CAS  Google Scholar 

  10. Morita E, Sundquist WI: Retrovirus budding. Annual review of cell and developmental biology. 2004, 20: 395-425. 10.1146/annurev.cellbio.20.010403.102350.

    Article  CAS  Google Scholar 

  11. Klein KC, Reed JC, Lingappa JR: Intracellular destinies: degradation, targeting, assembly, and endocytosis of HIV Gag. AIDS Reviews. 2007, 9 (3): 150-61.

    Google Scholar 

  12. Bieniasz PD: The cell biology of HIV-1 virion genesis. Cell host & microbe. 2009, 5 (6): 550-8.

    Article  CAS  Google Scholar 

  13. Brass AL, Dykxhoorn DM, Benita Y, Yan N, Engelman A, Xavier RJ, Lieberman J, Elledge SJ: Identification of host proteins required for HIV infection through a functional genomic screen. Science. 2008, 319 (5865): 921-926. 10.1126/science.1152725.

    Article  CAS  Google Scholar 

  14. König R, Zhou Y, Elleder D, Diamond TL, Bonamy GM, Irelan JT, Chiang CY, Tu BP, De Jesus PD, Lilley CE, Seidel S, Opaluch AM, Caldwell JS, Weitzman MD, Kuhen KL, Bandyopadhyay S, Ideker T, Orth AP, Miraglia LJ, Bushman FD, Young JA, Chanda SK: Global analysis of host-pathogen interactions that regulate early-stage HIV-1 replication. Cell. 2008, 135 (1): 49-60. 10.1016/j.cell.2008.07.032.

    Article  Google Scholar 

  15. Zhou H, Xu M, Huang Q, Gates AT, Zhang XD, Castle JC, Stec E, Ferrer M, Strulovici B, Hazuda DJ, Espeseth AS: Genome-scale RNAi screen for host factors required for HIV replication. Cell host & microbe. 2008, 4 (5):

  16. Goff SP: Knockdown screens to knockout HIV-1. Cell. 2008, 135 (3): 417-420. 10.1016/j.cell.2008.10.007.

    Article  CAS  Google Scholar 

  17. Bushman FD, Malani N, Fernandes J, D'Orso I, Cagney G, Diamond TL, Zhou H, Hazuda DJ, Espeseth AS, Konig R, Bandyopadhyay S, Ideker T, Goff SP, Krogan NJ, Frankel AD, Young JA, Chanda SK: Host cell factors in HIV replication: meta-analysis of genome-wide studies. PLoS pathogens. 2009, 5 (5): e1000437-10.1371/journal.ppat.1000437.

    Article  Google Scholar 

  18. Prudencio M, Lehmann MJ: Illuminating the host - how RNAi screens shed light on host-pathogen interactions. Biotechnology journal. 2009, 4 (6): 826-837. 10.1002/biot.200900071.

    Article  CAS  Google Scholar 

  19. Müller B, Daecke J, Fackler OT, Dittmar MT, Zentgraf H, Kräusslich HG: Construction and characterization of a fluorescently labeled infectious human immunodeficiency virus type 1 derivative. Journal of virology. 2004, 78 (19): 10803-10813. 10.1128/JVI.78.19.10803-10813.2004.

    Article  Google Scholar 

  20. Lampe M, Briggs JA, Endress T, Glass B, Riegelsberger S, Kräusslich HG, Lamb DC, Bräuchle C, Müller B: Double-labelled HIV-1 particles for study of virus-cell interaction. Virology. 2007, 360 (1): 92-104. 10.1016/j.virol.2006.10.005.

    Article  CAS  Google Scholar 

  21. Ivanchenko S, Godinez WJ, Lampe M, Kräusslich HG, Eils R, Rohr K, Bräuchle C, Müller B, Lamb DC: Dynamics of HIV-1 Assembly and Release. PLoS Pathogens. 2009, 5 (11): e1000652-10.1371/journal.ppat.1000652.

    Article  Google Scholar 

  22. Campbell RE, Tour O, Palmer AE, Steinbach PA, Baird GS, Zacharias DA, Tsien RY: A monomeric red fluorescent protein. Proceedings of the National Academy of Sciences of the United States of America. 2002, 99 (12): 7877-7882. 10.1073/pnas.082243699.

    Article  CAS  Google Scholar 

  23. Huang M, Orenstein JM, Martin MA, Freed EO: p6Gag is required for particle production from full-length human immunodeficiency virus type 1 molecular clones expressing protease. Journal of virology. 1995, 69 (11): 6810-6818.

    CAS  Google Scholar 

  24. Carlson LA, Briggs JA, Glass B, Riches JD, Simon MN, Johnson MC, Müller B, Grunewald K, Kräusslich HG: Three-dimensional analysis of budding sites and released virus suggests a revised model for HIV-1 morphogenesis. Cell host & microbe. 2008, 4 (6): 592-599.

    Article  CAS  Google Scholar 

  25. Ly TD, Ebel A, Faucher V, Fihman V, Laperche S: Could the new HIV combined p24 antigen and antibody assays replace p24 antigen specific assays?. Journal of virological methods. 2007, 143 (1): 86-94. 10.1016/j.jviromet.2007.02.013.

    Article  CAS  Google Scholar 

  26. Ono A, Waheed AA, Joshi A, Freed EO: Association of human immunodeficiency virus type 1 gag with membrane does not require highly basic sequences in the nucleocapsid: use of a novel Gag multimerization assay. Journal of virology. 2005, 79 (22): 14131-14140. 10.1128/JVI.79.22.14131-14140.2005.

    Article  CAS  Google Scholar 

  27. Wyma DJ, Jiang J, Shi J, Zhou J, Lineberger JE, Miller MD, Aiken C: Coupling of human immunodeficiency virus type 1 fusion to virion maturation: a novel role of the gp41 cytoplasmic tail. Jornal of virology. 2004, 78 (7): 3429-35. 10.1128/JVI.78.7.3429-3435.2004.

    Article  CAS  Google Scholar 

  28. Zhang JH, Chung TD, Oldenburg KR: A Simple Statistical Parameter for Use in Evaluation and Validation of High Throughput Screening Assays. Journal of Biomolecular Screening. 1999, 4 (2): 67-73. 10.1177/108705719900400206.

    Article  Google Scholar 

  29. Erfle H, Neumann B, Liebel U, Rogers P, Held M, Walter T, Ellenberg J, Pepperkok R: Reverse transfection on cell arrays for high content screening microscopy. Nature protocols. 2007, 2 (2): 392-399. 10.1038/nprot.2006.483.

    Article  CAS  Google Scholar 

  30. Erfle H, Neumann B, Rogers P, Bulkescher J, Ellenberg J, Pepperkok R: Work flow for multiplexing siRNA assays by solid-phase reverse transfection in multiwell plates. J Biomol Screen. 2008, 13 (7): 575-580. 10.1177/1087057108320133.

    Article  CAS  Google Scholar 

  31. Garrus JE, von Schwedler UK, Pornillos OW, Morham SG, Zavitz KH, Wang HE, Wettstein DA, Stray KM, Côté M, Rich RL, Myszka DG, Sundquist WI: Tsg101 and the vacuolar protein sorting pathway are essential for HIV-1 budding. Cell. 2001, 107 (1): 55-65. 10.1016/S0092-8674(01)00506-2.

    Article  CAS  Google Scholar 

  32. Konvalinka J, Litterst MA, Welker R, Kottler H, Rippmann F, Heuser AM, Kräusslich HG: An active-site mutation in the human immunodeficiency virus type 1 proteinase (PR) causes reduced PR activity and loss of PR-mediated cytotoxicity without apparent effect on virus maturation and infectivity. Journal of virology. 1995, 69 (11): 7180-7186.

    CAS  Google Scholar 

  33. Börner K, Hermle J, Sommer C, Brown NP, Knapp B, Glass B, Kunkel J, Torralba G, Reymann J, Beil N, Beneke J, Pepperkok R, Schneider R, Ludwig T, Hausmann M, Hamprecht F, Erfle H, Kaderali L, Kräusslich HG, Lehmann MJ: From experimental setup to bioinformatics: an RNAi screening platform to identify host factors involved in HIV-1 replication. Biotechnology journal. 2010, 5 (1): 39-49. 10.1002/biot.200900226.

    Article  Google Scholar 

Download references

Acknowledgements

We sincerely thank Hans-Georg Kräusslich for continuous support and critical reading of the manuscript. The help of Holger Erfle and his group for help with the preparations for the siRNA experiments is gratefully acknowledged. We thank Roger Tsien for the gift of plasmid pmRSET.mRFP1. This work is part of the activities of the HIV-ACE research network (HEALTH-F3-2008-201095), which is supported by a grant of the European Commission within the Priority 1 "Health" work programme of the 7th Framework Programme of the EU. The work has also been funded in part by a grant from the Deutsche Forschungsgemeinschaft (MU885/4-2) and by the BMBF funded project Viroquant (0313923).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Barbara Müller.

Additional information

Authors' contributions

JH adapted the protocol to a semi-automated format, established the protocol for the siRNA screening approach and participated in the design of experiments. MA performed the experiments for the initial characterization and validation of the assay procedure. AMH contributed to adaptation and validation of the semi-automated procedure. BM conceived and coordinated the study, participated in its design, performed initial experiments and drafted the manuscript. All authors have read and approved the final manuscript.

Electronic supplementary material

12896_2009_500_MOESM1_ESM.DOC

Additional file 1: Figure S1: Detection of purified fluorescently labelled proteins against medium background. Fluorescent proteins eGFP (open circles), eYFP (filled circles), eCFP (open triangles) and mRFP1 (open squares), respectively, were expressed as recombinant His-tagged proteins in Escherichia coli. Proteins were purified by Ni-chelate affinity chromatography using 1 ml HisTrap columns (GE Healthcare) according to the manufacturer's instructions and dialyzed against PBS. Protein Party was verified by SDS-PAGE and concentrations were calculated from OD280 values determined for GuHCl denatured proteins using extinction coefficients of 21890 (eGFP), 23380 (eYFP), 25900 (eCFP) and 32890 (mRFP), respectively. Serial dilutions were set up in DMEM, 10% FCS, and fluorescence intensity was determined at the appropriate wavelength settings (488/508 nm, 512/528 nm, 433/475 nm and 584/607 nm, respectively) using a TECAN Safire multi-well reader. Background values obtained for medium alone were subtracted. Mean values and standard deviations of triplicate dilutions are shown; lines represent linear regressions. Figure S1 B shows an expansion of the lower part of the graph shown in Figure S1 A. (DOC 72 KB)

12896_2009_500_MOESM2_ESM.DOC

Additional file 2: Figure S2: Fluorescence spectra of tissue culture supernatants. Fluorescence emission spectra of tissue culture supernatants harvested at 36 h post transfection from 293T cells transfected with pCHIVeYFP (black solid line), pCHIVeYFPlate(-) (grey solid line), peYFP-C1 (black dotted line), or from untransfected cells (grey dotted line), respectively, were recorded using an SLM Aminco Spectrofluorometer (excitation wavelength: 512 nm). (DOC 43 KB)

12896_2009_500_MOESM3_ESM.DOC

Additional file 3: Figure S3: Determination of Z factor from control wells of a pilot screen. Fluorescence intensities of supernatants from 60 positive control wells (transfected with pCHIV/pCHIVeYFP, 0.5% DMSO) and 36 negative control wells (untransfected cells) from 6 replicate 96-well plates were determined. The graph shows mean values and standard deviations (SD) derived from these data. The Z factor [28] was calculated from these data using the formula Z = 1 - (3 × (SDpos + SDneg))/(meanpos - meanneg), yielding a value of 0.71. (DOC 32 KB)

Authors’ original submitted files for images

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Hermle, J., Anders, M., Heuser, AM. et al. A simple fluorescence based assay for quantification of human immunodeficiency virus particle release. BMC Biotechnol 10, 32 (2010). https://doi.org/10.1186/1472-6750-10-32

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1472-6750-10-32

Keywords