GoIFISH: a system for the quantification of single cell heterogeneity from IFISH images
© Trinh et al.; licensee BioMed Central Ltd. 2014
Received: 6 May 2014
Accepted: 15 August 2014
Published: 26 August 2014
Molecular analysis has revealed extensive intra-tumor heterogeneity in human cancer samples, but cannot identify cell-to-cell variations within the tissue microenvironment. In contrast, in situ analysis can identify genetic aberrations in phenotypically defined cell subpopulations while preserving tissue-context specificity. GoIFISHGoIFISH is a widely applicable, user-friendly system tailored for the objective and semi-automated visualization, detection and quantification of genomic alterations and protein expression obtained from fluorescence in situ analysis. In a sample set of HER2-positive breast cancers GoIFISHGoIFISH is highly robust in visual analysis and its accuracy compares favorably to other leading image analysis methods. GoIFISHGoIFISH is freely available at www.sourceforge.net/projects/goifish/.
Quantifying cell-to-cell heterogeneity in the tissue context
Intra-tumor heterogeneity is currently accepted as a hallmark of cancer, being present in virtually all tumor traits . Sensitive molecular techniques developed in the last few years have allowed a detailed genetic and phenotypic deconvolution of intra-tumor heterogeneity. These include genome-wide analysis of bulk tumor samples to describe evolutionary trajectories in relapsed tumors and genomic divergence between primary tumors and metastases -, as well as single-cell genomic profiling ,. However, despite methodological improvements in the molecular characterization of single cells, the accurate interpretation of intra-tumor heterogeneity requires the inference of cell-to-cell variability within a particular tissue context, which can only be directly assessed by in situ analysis. Microenvironmental constraints within spatially restricted areas of a tumor can exert differential selective pressures, leading to the manifestation and the selection of different phenotypes and particular genotypes. For instance, different oxygen levels, the presence of inflammatory cells, or the physical interaction with extracellular components in different parts of a tumor - can influence cellular phenotypes and contribute to different trajectories in the evolution of a tumor . Therefore, the accurate interpretation of cellular phenotypic and genomic heterogeneity requires tissue-context specificity -.
IFISH: Immunofluorescence in situ hybridization In situ fluorescence-based detection of proteins, DNA, and RNA enables the simultaneous detection of multiple markers in single cells by epifluorescence, confocal, or multispectral imaging technology. Combining both immunofluorescence and fluorescence in situ hybridization (IFISH) allows multiplexing to detect both genomic and phenotypic traits at the single cell level . This approach captures cell-to-cell variations missed in cell population analyses while preserving specific microenvironmental contexts. As an in situ analysis, IFISH allows the spatial mapping of individual cells to measure topological heterogeneity. Visualizing topological heterogeneity can have important implications in predicting treatment response, as well as tailoring treatment to suit the diverse cell populations observed within a tumor . However, these in situ studies require the analysis of multiple markers in thousands of cells, are very time-consuming, and their reproducibility could be influenced by variability between users . Therefore, there is an urgent need for the development of objective analytical tools that minimize scoring subjectivity and facilitate the quantification of multiple traits in single cells while preserving context specificity. The implementation of these tools in both basic and translational research will advance our understanding of tumor biology and will facilitate biomarker discovery and validation.
GoIFISH GoIFISH:quantifying tumor heterogeneity in IFISH images For application to IFISH, accurate segmentation at the nuclear, membrane and spot level are critical for subsequent analysis, which often interrogates clonal populations or evaluates relationships between protein and genomic expression. Objective integration of protein expression and copy number requires not only accurate segmentation, but also the separation of normal cells from tumor cells, and appropriate background subtraction associated with auto-fluorescence. Very few existing softwares allow manual alterations of small inaccuracies in cell segmentation and often incorrect cell classification results cannot be changed. Visual scoring by a trained observer (e.g. pathologist) is the gold standard for detecting cells and automated image analysis systems developed to complement pathologist scoring require user validation .
To address these challenges, we have developed a semi-automated system which provides users with an automated starting point in segmentation and can readily accept user input to improve the segmentation result. GoIFISHGoIFISH is able to segment nuclei, locate and estimate spots, detect membranes, measure morphological and intensity properties and classify cells. GoIFISHGoIFISH is a versatile method that allows researchers to determine and quantify, for instance, the amplification status of single locus within cells, together with the detection of phenotypic markers present in different subcellular locations. It preserves the tissue context specificity and provides coordinates for the topological mapping of each cell. Simple topology maps can be displayed to illustrate spatial variations within an image with respect to two given stains. GoIFISHGoIFISH allows users to analyze FISH, IF or IFISH images containing a maximum of 5 markers, of which one must be the nuclear marker DAPI. We validated our software in a pilot HER2+ breast cancer cohort of 10 samples and compared its performance with existing softwares.
Comparison between softwares available for cell segmentation
Open-Source on request
Background Intensity Subtraction
Tuning but no subtraction
Correct Illumination Calculate
A-Priori knowledge required (eg. cell size or segmentation methods)
Visualise Batch Segmentation Results
Cell Specific Information
Topology or summary Maps
OMERO is a platform for the storage and annotation of microscopy images , and Icy and ImageJ have been developed as general platforms for image analysis ,. All three are dependent on the development of plug-ins for specific applications from its user-base. OMERO currently does not have automated algorithms for image segmentation, and is dependent on user input for the delineation of cell boundaries or other regions of interest. ImageJ and Icy have a series of plugins for nuclear segmentation, membrane segmentation and spot detection, however these three processes are often disjoint and will require user effort to collate these results. We have used MATLAB as the platform for developing GoIFISHGoIFISH due to its wide user-base, strong image analysis capabilities and comprehensive data analysis features.
Softwares with specific cell segmentation capabilities include Columbus, CellProfiler , ImageM  and CellTracker . CellProfiler and Columbus are programs which specialize in the segmentation of cells from the cytoplasmic and nuclear level, down to the subcellular or genomic level. These have been developed primarily for high-throughput analysis of cells in culture, and often are based on assumptions about the regularity of size and morphology within the cell population of interest. These softwares are optimized to have minimal segmentation errors in cell-culture images, however, may not be directly applicable to real tissue.
ImageM  is a software developed for detection and counting of RNA signals using a semi-automated approach. Users can refine results, however, to extract features on a per nucleus or cell basis, manual delineation of regions of interest is required. CellTracker  has been developed primarily for the live-tracking of cells, and a semi-automated approach to nuclear and cytoplasmic segmentation is also applied.
Availability of GoIFISH
GoIFISHGoIFISH is freely available at www.sourceforge.net/projects/goifish/ under the GNU General Public License version 2. GoIFISHGoIFISH is written in MATLAB and all source code is provided to allow analysis on both command line and through the Graphical User Interface (GUI) (Additional file 1). It is dependent on OMERO Bio-formats for the conversion of images to the correct file format for loading . All images used in this study are available at the given link. The GUI has been created into a stand-alone program operable on Windows and Mac OS systems following the installation of the appropriate version of MATLAB Compiler Runtime (v7.14), provided at the given link.
Optimal properties of images for analysis in GoIFISH GoIFISH
GoIFISHWrapper (Command Line)
.tiff.zvi Use bio-formats to convert other formats
12 megapixels for comfortable use
Number of stains
Up to 5. Must include DAPI
Unlimited but must include DAPI
Number of Cells
Under 1000 for comfortable use
Cell Size (px)
Optimally 60x magnification (2500-10000 px), 20x and 40x magnification
also available (250-3000 px)
The GoIFISH workflow
The following paragraphs give a detailed overview of the GoIFISHGoIFISH workflow. An overview of the capabilities of GoIFISHGoIFISH, including the user interface is shown in Figure 1.
Step 1: Loading and preprocessing dataImages can be loaded as a.mat file containing a cell array or a.tiff file into the GUI. After successful loading, the first image in the series will be presented (Figure 1A). These are automatically adjusted to ensure 1% of the image is saturated at lower and higher intensities, which is more suited for nuclear or cytoplasmic images but may saturate spots. The ‘RangeScale’ option is recommended in these scenarios. Brightness and contrast of each image can be adjusted for auto-fluorescence, to ensure an optimal dynamic range and to prevent saturation of the image. This is critical for good segmentation results. Note that the image adjustment will improve the user experience and segmentation results, however, the intensities can be measured from the raw image for comparative quantitation.
GoIFISHGoIFISH allows background intensity adjustment, both using a single global intensity value, or on a cell specific level for nuclei. The user draws ‘background’ regions using the paintbrush tool, from which the mean background intensity is calculated. This is subtracted from either raw intensities, which is recommended for comparisons between samples, or from an adjusted image. A per cell nuclear background adjustment is also available, whereby a background intensity is calculated at a margin of 2-6 pixels from the edge of each segmented nucleus. This will be computed automatically for all nuclear stains, and will account for local variation in background intensity.
To finalize the preprocessing stage the user will need to indicate the stain type in each channel, of which one must be DAPI, and the magnification of the image. 60x, 40x and 20x magnifications are permitted, however higher resolutions are recommended for the accurate detection of spots. Following this, a quick preprocessing step is applied to assign the stain type to each image. After this, segmentations on either the DAPI channel alone or all channels using default parameters can be performed (Figure 1B).
Step 2: Nuclear segmentation The foreground or cellular portion of the image is automatically detected by Otsu thresholding  on a combined image of entropy and intensity, which is used as a mask in cell segmentation. Nuclei are segmented using an iterative H-minima transformed watershed , where the local intensity depth of pixels under a given threshold is suppressed, and a watershed is applied. Fragments attained from each step are classified as either optimal, undersegmented, or oversegmented. Cells with optimal properties are selected, and the remaining image is subjected to segmentation at a lower threshold. We have developed an approach to mitigate oversegmentation by joining neighboring fragments according to their morphological features (see Material and methods). There is also an option of performing a seeded-watershed  for images with a small number of cells or poor contrast at cell boundaries, whereby the user indicates the cell locations with a series of spots. A H-minima watershed is then applied using this information.
Step 3: Membrane segmentation Following segmentation, the user has the option of narrowing down the population of interest using a minimum size threshold, or edit the borders manually (see Step 5). It is recommended that the user checks the output of the nuclear segmentation before proceeding as membrane segmentation and spot detection are dependent on the nuclear map generated.
Membrane segmentation is performed by combining a Voronoi segmentation of nuclei with the intensity information of the image. High intensity edges within the image are set as local maxima to ensure segmentation occurs along these edges. Fragments are then merged based on their location with respect to the nuclear segmentation. This result can be further refined using active contours, such as Chan-Vese Segmentation  or Localized Segmentation .
Step 4: FISH detection For single spot detection, a Laplacian of Gaussian filter is applied to the image to determine candidate spots. Spots which are also local maxima in the gradient image are selected. The user has an option of entering an expected size threshold (Default minimum spot size of 15 pixels for 60x images), and a preferred intensity threshold. An optional morphology classifier (see Material and methods) can be applied to differentiate between spots and artefacts in images with low contrast signal, such as signals from centromere 17.
Step 5: Manually editing segmentationsGoIFISHGoIFISH provides a toolbox to manually edit segmentations if needed. Manual editing of the segmentation output can be easily achieved by drawing a border between cells with the ‘scissors’ tool, oversegmented cells can be ‘glued’ together, and artefacts can be ‘trashed’. Regions can be ‘painted’ or ‘erased’. All operations are terminated by right clicking, and pressing the ‘escape’ exits a particular editing mode.
Step 6: Post segmentation processingSegments from each channel need to be mapped to the DAPI channel in order to construct a matrix of features, using the ‘Update’ function. An error will appear if there are inconsistencies in the segmentation, such as two nuclei mapping to one membrane. Following successful mapping, the user can generate heatmaps and topology maps to visualize staining variations within the image (Figure 1B) and perform cell classification (Figure 1C).
Cells in the image are classified by support vector machine into 4 possible cell types. The user labels candidate cells for each class (for example, to separate fibroblasts from tumor cells), and applies the classification. This classification is based on morphological parameters but can also include intensity information. The classification result can be manually corrected if inconsistencies appear.
Step 7: Output from GoIFISHThe output from GoIFISHGoIFISH can be saved as a series of images and a.csv file with cell specific measurements. These include intensity measurements, including raw and background adjusted intensities, morphologial parameters such as area, perimeters, axis lengths, the location of the centroid of each nuclei, the cell label if classification is performed and copy numbers in the case of spot detection. Data can be downloaded into the MATLAB environment, or saved as a progress.mat file where processing can be resumed in another session.
GoIFISHWrapper: Combining Steps 1-4GoIFISHGoIFISH provides a wrapper for batch analysis, which is implemented via command line. The user simply provides the filepath of interest and edits a Parameter File which contains information such as the stains used, the magnification of the image and the segmentation parameters. The results are automatically saved as progress.mat files which can be loaded into the GUI for segmentation editing, background selection and cell labelling. Other benefits of running the wrapper include unlimited number of stains and the analysis of larger images. However, it is recommended that each image does not exceed a resolution of 12 megapixels if the user wishes to edit cells in the GUI. In this circumstance, it is recommended that the image is sectioned into a number of smaller constituent images which are analyzed independently.
GoIFISH performance and benchmarking
The performance of GoIFISHGoIFISH was compared to two state-of-the-art image analysis systems, the proprietary Columbus software from Perkin-Elmer, and the open-source CellProfiler from the Broad Institute . All three image analysis softwares can detect nuclei, FISH signals, cytoplasm staining, and report morphological properties including size, and intensities. Details of all statistical analyses including code to reproduce plots are contained in the Additional file 3.
Comparing GoIFISH GoIFISHto existing automated methods
Columbus has an intuitive interface, real-time feedback, automatic detection of approximate cell sizes, with very little image processing knowledge required to operate the system (Table 1). On the other hand, CellProfiler has the benefits of wider functionality as its open-source nature allows its user base to develop and maintain specialized functions. However, it requires a-priori knowledge about the images and different segmentation methods, which may take the user a long time to develop an optimal segmentation pipeline. The parameters used to segment images in these two programs are described in Material and methods.
GoIFISHGoIFISH was tested in two scenarios. In order to perform fair benchmarking, images from 10 samples were run in GoIFISHGoIFISH on the default settings (see Material and methods). In addition, we tested its capabilities for improvement with user input.
In Cent17 and HER2 detection GoIFISHGoIFISH with default parameters surpassed the two existing methods (Mean F-Scores: 0.69, 0.83 for cent17 and HER2 respectively). Columbus has high recall but poor precision, reflecting a higher false positive rate. It should be noted that the manually curated samples had variable F-score, which may be a subsequent propagation error from nuclear segmentation.
While precision and recall assess the presence of an object, they provide no information on how accurate the morphology of the segmentation is. For instance, encroachment errors with slight misplacement of cell boundaries will have no effect on the F-Score. Therefore, the perimeter-area ratio was measured for each cell and compared to the gold standard as an assessment of whether the correct shape was detected. Each individual spot in Figure 2C represents the average difference in perimeter-area ratio for one particular image. Points centred around 1 indicate very little variation in shape compared to the gold standard. In both nuclear and membrane segmentation, GoIFISHGoIFISH values were closer to 1 compared to Columbus and CellProfiler, indicating that the morphology was better represented.
Timing comparisons between GoIFISHGoIFISH and CellProfiler
Approximate Number of Cells
CellProfiler Time (s)
GoIFISHGoIFISH Time (s)
Correlating GoIFISH GoIFISHoutput with visual interpretation
While precision-recall testing allows reliable assessment of segmentation accuracy, the obtained data must be reflective of the biology to draw valid conclusions. Automated scoring of protein intensities and spot areas were compared with visual pathologist scoring on a single-cell level, with a total of 355 cells scored for ER staining, membrane HER2 intensity and cent17 and HER2 copy number.
ANOVA in conjunction with Tukey’s range test was performed on samples with ‘negative’ expression to determine whether the baseline means are directly comparable to each other. Out of the 45 possible pairwise comparisons, Columbus had 26 pairwise comparisons, CellProfiler had 13 and GoIFISHGoIFISH had only 10 pairwise comparisons which showed a significant difference in baseline mean (p<0.05). Sample 7360 had a negative intensity after background correction using GoIFISHGoIFISH, but in practice would be assigned a value of 0 which would further lower the number of significant differences. Using a per nucleus specific background subtraction method in GoIFISHGoIFISH, ‘positive’ samples become comparable to each other across all images (Figures 3A, Additional file 5: Figure S2A). Figure 3B illustrates the right ordering and statistical difference between each class, demonstrating that the method can reproduce quantitatively the visual scoring (T-test, p<0.01 between all categories).
The same analysis was applied to HER2 membrane staining to determine whether the intensity could recapitulate the membrane completeness in cells. HER2 protein assessment in a clinical setting often uses patterns of staining to guide subsequent treatment . Cells are classified as having ‘negative’ membrane staining, ‘complete’ positive staining or ‘incomplete’ positive staining. Both GoIFISHGoIFISH after background subtraction and Columbus demonstrated a step-wise increase in intensity with highest intensity observed in complete membranes (Figure 3C,D). Combining all cells, a statistical difference at the correct order of classes was observed in only the GoIFISHGoIFISH background adjusted and manually edited samples (Figure 3D, Additional file 5: Figure S2D).
The coefficient of variation was also computed as a second metric of differentiating between ‘complete’ and ‘incomplete’ membranes. It is expected that ‘complete’ membranes have a lower variation than ‘incomplete’ membranes. In both GoIFISHGoIFISH and Columbus, an increased coefficient was observed in the samples with broken membranes compared to the samples with complete membranes, this was however only statistically significant in GoIFISHGoIFISH but not in Columbus (Figure 3E,F, Additional file 5: Figure S2C,D).
Effects of user variability on GoIFISH GoIFISHoutputs
User input may influence background intensity correction, cell segmentation results and cell classification. GoIFISHGoIFISH has been developed with a number of strategies to minimize the effects of inter-user variation.
To address the issue of background heterogeneity within an image, a trained observer selected four different background regions within each image to compare intensities. These values are reported as coefficients of variation (Figure 5B), where the size of each box proportional to the mean reported intensity. In most images, a low variation of 10% or less was observed, with the exception of 6361 and 7916 which had high auto-fluorescence and overexposure respectively. The greatest variation was observed in the DAPI channel, which is a general DNA marker used to assess the quality of the sample. In practice, DAPI intensities are rarely measured for quantitative analysis. The other stains are more selective and specific for a particular protein or locus of interest, and have demonstrated greater stability in background intensities.
Manual editing of segmentation results is also prone to user subjectivity. To reduce both this effect and manual labor, GoIFISHGoIFISH was designed with a toolbox which minimizes the amount of clicks or mouse-drawing performed by the user. For example, the merging of cells requires two clicks of the mouse, and the segmentation of overlapping cells requires one line to be drawn. These features ensure that the morphology and boundary of cells are consistent in each image irrespective of the user.
To determine the effectiveness of these tools, two independent scorers manually edited 50 missegmented cells across the 10 test images. The nuclear and cytoplasmic areas were measured, alongside nuclear ER intensity and HER2 membrane intensity. Nuclear segmentation was consistent between the two scorers (r=0.86, Figure 5C), however a number of cells were considered to be larger by Scorer 1 than Scorer 2. The discrepant cells were determined to be mitotic, phenotypically characterised by the appearance of two nuclei in the DAPI channel yet sharing the same membrane in the HER2 channel, and considered as one cell by Scorer 1 but as two cells by Scorer 2. The cyotplasmic areas had lower correlation between the two observers (r=0.79), which can be attributed to the HER2 status of the cell. In the absence of a well defined HER2 membrane, the shape is open to interpretation, accounting for the greater variation in the HER2- cells than the HER2+ cells (HER2+ only: r=0.83).
Despite the differences in morphology of the segmented cells, 99% correlation was observed in the raw recorded intensities (r=0.99 for both ER and HER2, Figure 5D), demonstrating that intensity measurements are robust to differences in cell segmentation between users.
Finally, we tested how the performance of the cell classifier depends on the size of the training set and cellular features in two different cell classification scenarios: (1) to differentiate myoepithelial from luminal cells and (2) to differentiate lymphocytes and stroma from tumor (Figure 5E). Two images representing these two scenarios were labelled by a trained observer. Training sets of increasing size (starting from 2 cells) were created by randomly sampling the number of required cells, and where possible an equal number of cells from each class were selected. To determine the 95% confidence interval for classifier accuracy, 500 permutations of the training set for each size were used to predict the labels within an image. Our results demonstrate that using morphological parameters alone, the accuracy approaches 70% for myoepithalial-luminal discrimination, and 80% for lymphocyte-stromal-tumor discrimination. With the addition of stain information, the accuracy approached 95% and 100% accuracy respectively. The average accuracy of the classifier increases with a larger number of labelled cells, however, if well-chosen, high classification accuracy can still be attained with a training set of under 10 cells.
Visualizing the cellular diversity within an image
Both the training and test set showed resemblance, however samples 6361 and 7619 differed from the gold standard distribution. These differences may be due to the arbitrary setting of one intensity threshold, and the subjectivity in visual scoring where background intensities between samples are seldom taken into consideration.
Figure 6B illustrates the spatial distribution of the classified cells in sample 6370 and the corresponding topology map which displays the relative ratio between HER2 and ER. Most of the cells which were considered as double negative are identified morphologically as stroma, and display low intensity in both proteins. Cells which were classified as double positive demonstrated subtle cell-to-cell variations which would not otherwise be observable with a strict threshold. The same analysis was applied to HER2 protein and HER2 spots, with a cutoff placed at HER2 spot area of 60 pixels which is roughly equivalent to 3 spots. Most cells exhibited a HER2 protein intensity increase with a spot area increase (shown in green), however, a subset with high expression of HER2 but a relatively lower spot area was also present.
The scoring of cells is also dependent on the tumor region selected for analysis (Figure 6D). Sample 6361 was considered to be clinically HER2+, however a ductal carcinoma in situ rather than an invasive component of the tumor was selected for this analysis. As a result, weak HER2 and ER intensities were attained, as shown in the topology map, compared to sample 6370.
Segmentation of complex tissue components improves with manual correctionThere are many challenges with in situ analysis of molecular features in tissue sections. Tissues display complex compositions of organic structures such as epithelial elements, vasculature, lymphatic components, nerves and supportive tissue including different types of fibers. There is morphological diversity between cell types and their organization, and in cancer this is even more pronounced. For automatized image analysis, accurate segmentation of cellular components is crucial to avoid misleading estimates of markers. Overlapping or closely spaced nuclei can easily be interpreted as one, and cells with major sectioning artefacts need to be discarded.
Many automatized approaches have been developed to address these issues, and Columbus and CellProfiler are two state of the art softwares used to benchmark the performance of GoIFISHGoIFISH. These have been developed for general applicability to a number of biological scenarios, of which cell-based culture is their main strength. As a result, the application to tissue-sections with the heterogeneous morphology may have resulted in the poorer results observed. In particular, CellProfiler did not perform at a similar level as Columbus and GoIFISHGoIFISH: this may be attributed to the need to optimize parameters in a segmentation pipeline before batch processing, whereas both Columbus and GoIFISHGoIFISH automatically calculates parameters for each image. In addition, GoIFISHGoIFISH provides user-friendly options to manually correct inaccurate segmentations and remove artefacts. As shown in Figures 2A and Additional file 4: Figure S1B, this correction step is crucial for improving precision and recall.
Inter-user subjectivity is a large issue in the field of pathology and there is potential of introducing user bias in the background selection and segmentation editing steps. To mitigate this, our editing toolbox is designed to minimize the amount of manual cell-outlining required, and guidelines have been included in the user-manual for background selection. We have demonstrated that these measures are effective in minimizing inter-user variation, with similar intensity measurements for both background and cell staining reported by different scorers (Figure 5).
Image analysis is dependent on image qualityThe quality of the segmentation and marker recognition is highly dependent on the quality of the samples attained. For formalin-fixated paraffin-embedded tissue sections, there are variables including fixation type, fixation duration and tissue processing that differ from patient to patient and between laboratories . For fluorescence analysis in general, some tissue composites induce more autofluorescence than others making “true" staining difficult to quantify. Tissue sections from patient samples have preprocessing steps which cannot be controlled for at the same level of precision as fixation of cell lines can. This explains the variation in sample-to-sample fluorescence intensities as illustrated in Figure 3, 5B, 6B, despite imaging all sections under similar conditions.
The images used in this study were of high quality but still exhibited artefacts that confounded segmentation results, which is reflective of the challenges faced in image acquisition and analysis. Low intensity of a spot marker compared to the background was observed in sample 6361, resulting in poor detection using all three methods. Overexposure of a channel will increase the false positive rate, as seen in image 7619, and the presence of background artefacts in the DAPI channel will affect segmentation accuracy, as seen in sample 7350. Background adjustment, manual editing and the application of classifiers are strategies GoIFISHGoIFISH uses to address these issues. GoIFISHGoIFISH allows users to select background regions to ensure baseline intensities are comparable across samples, and performs per nucleus background adjustment to remove local variations in auto-fluorescence. This is necessary for comparability of cells across samples. To assist in accurate segmentation, a morphological classifier was applied to centromere 17 detection to remove confounding effects. In addition, manual user input in GoIFISHGoIFISH rectified most difficulties encountered during segmentation.
Contribution of the software on measuring intra-tumor heterogeneityAccurate segmentation on the nuclear, membrane and spot level are essential for the extraction of biologically meaningful features from cells. GoIFISHGoIFISH has demonstrated comparable segmentation to CellProfiler and Columbus in nuclear segmentation, and has outperformed them in membrane and spot detection. GoIFISHGoIFISH is capable of segmenting membranes when weakly positive or incomplete, allowing for subsequent objective analysis of intensity-based features.
To address the complexity of tissue composition and its impact on prognosis  we have also included a cell-type classifier based on morphology and intensity. By marking a few segmented cells, all cells with a similar morphology are identified with high accuracy, particularly if intensity information is also included. To illustrate the importance of quantitative analysis in the correct cellular context, we have included a pre-invasive part of a tumor in our analysis (Sample 6361). As shown in Figure 6, the clinically reported HER2 positivity was not detected. These cells are luminal epithelial and myoepithelial cells, rather than invasive neoplastic cells. In downstream analyses, the added categorical knowledge will ensure these would not be directly compared to invasive cells. The extraction of features from a sample can be multi-dimensional, making visualization of heterogeneity a difficult task. We have included simple topology maps that overlap two stains of interest, allowing visualization of both heterogeneity across cells and their spatial relationships.
SummaryGoIFISHGoIFISH has been developed to segment high magnification images with combined genomic and phenotypic traits, combining the analysis of nuclei, membranes and spots into a single easy-to-operate system. Thus, GoIFISHGoIFISH allows the objective quantification of the morphological, genomic and phenotypic heterogeneity often observed in tumor IFISH images. Application of quantitative approaches like GoIFISHGoIFISH on large sample collections will lead to profound insights into the impact tumor heterogeneity has on disease progression, and may uncover evolutionary pathways explaining the development of resistance.
Material and methods
A sample set of HER2 positive breast cancers
This study was conducted in compliance with the Declaration of Helsinki and was approved by the regional ethics committee (REK S-06495b).
Human tissue samples were collected following protocols approved by the institutional review board of Oslo University Hospital Radiumhospitalet (IRB 2006-53). We used 10 formalin-fixed paraffin-embedded (FFPE) primary tumors from HER2+ breast cancer patients. In this work, we performed IFISH by combining the immunodetection of HER2 protein (expressed in the cell membrane) and Estrogen Receptor α (ER α, located in the nuclei) with the detection of HER2 and centromere 17 (cent17) copy number, following a protocol previously described . FFPE samples were dewaxed and hydrated in series of ethanol. Heat-induced antigen retrieval was performed in citrate buffer (pH 6) followed by pepsin digestion. After the immunostaining of HER2 and ER at room temperature in a humidifier, tissue slides were hybridized with HER2 and centromere 17 probes at 37C°. overnight. Post wash was carried out in SSC (saline-sodium citrate) buffers with different stringency, before air drying and mounting media with DAPI was added. Image acquisition was carried out in an epifluorescence microscope. One randomly selected area per tumor was photographed in a Zeiss Axioplan 2 microscope equipped with an Axio Cam MRM CCD camera and Axio Vision software. The experimental methods are explained in greater detail in the Additional file 3.
Analysis pipelines for CellProfiler, Columbus andGoIFISH GoIFISH
Columbus provides 4 nuclear, 4 cytoplasmic and 3 spot detection methods. These were first tested visually to determine the best candidate methods, which were then quantitatively compared with CellProfiler and GoIFISHGoIFISH in terms of precision and recall. The best results from each image were then used for direct comparison (see Additional file 3).
A CellProfiler analysis pipeline was constructed with the following parameters: Nuclei Segmentation was performed using two class Otsu Global Thresholding, and diameter of objects restricted to 20-120 pixels. Clumped cells were separated using the propagation method. Membrane detection was performed based on the propagation method, using the combination of the distance to the nuclei and intensity gradient to select the membrane. The spot signals were enhanced and masked to nuclear regions. Spots were detected using ‘RobustBackgroundPerObject’, limited to a diameter between 5 and 40 pixels, with clumped objects separated based on intensity.
The default GoIFISHGoIFISH pipeline performed shape optimised nuclear segmentation with intensity suppression between 10 to 30% and fragments of size less than 500 pixels discarded. The output nuclear map was applied to HER2 membrane detection and spot detection. Detected Centromere 17 spots were run through a morphological classifier to minimise effects of autofluorescence.
Metrics for performance evaluation
We compared computational approaches to a manually segmented ‘gold standard’. For each of the 10 images, nuclei, membranes and spots were outlined manually in the maximum projected image. Spots were counted through 15-21 z-stacks. In total, 355 individual cells were scored for membrane completeness, nuclear positivity and copy number. We then benchmarked the computational outputs of GoIFISHGoIFISH with the gold standard using the following panel of quality criteria. We define N t as the number of correctly segmented cells, N under and N over as the number of under and over segmented cells, N FP as the number of false positives which do not appear in the gold standard image, and N FN as the number of false negatives (See Additional file 4: Figure S1A).
Precision is defined as P=N t /(N t +N FP +N over )
Recall: R=N t /(N t +N FN +N under )
the F-Score is the harmonic mean of precision and recall: F=2P·R/(P+R), which is a measure of how well a cell can be detected, and whether the number of cells present closely resemble the true value.
Currently we use two morphological features for scoring fragments: Solidity and deviation from theoretical area. If g=∅ or if there is no positive maximum of the score, then c∗=f.
Classifiers have been implemented in a number of segmentation steps, including nuclei detection and spot detection, to minimise errors. GoIFISHGoIFISH only uses linear classifiers to reduce overfitting to data.
Nuclear detectionNuclei detection performs linear discriminant analysis based on a training set of 153 fragments from 5 images attained from nuclear segmentation performed at different depths. A total of 10 morphological properties including solidity, area, perimeter, axis lengths, axis ratios, circularity, area-perimeter ratio, and deviation from theoretical area and perimeter were measured. Each fragment was scored for oversegmentation, undersegmentation or optimal shape.
Spot detectionA spot classifier was constructed using segmentation output from 2 training images containing 67 spot candidates and placed into a linear discriminant analysis. Features extracted for the classifier include solidity, area, perimeter, axis ratios, circularity, area-perimeter ratio deviation from theoretical area and perimeter, mean intensity and minimum intensity. Three classes were assigned to each ‘spot’: optimal, too small or too large.
Cell classificationClassification of cells after segmentation is performed using a one vs all Support Vector Machine with linear kernel. Training data is generated on the spot from information supplied from the user. Features can either be morphological (area, perimeter, solidity, axis lengths and eccentricity) or contain extra information from the other channels, such as spot area or intensity.
We would like to thank Prof. Kornelia Polyak and Prof. Anne-Lise Børresen-Dale for support and insightful discussions. AT and FM would like to acknowledge the support of The University of Cambridge, Cancer Research UK and Hutchison Whampoa Limited. IHR and HGR would like to acknowledge support from Helse Sør-Øst, The Norwegian Cancer Association, the K. G. Jebsen Foundation and Radiumhospitalets legater. VA would like to thank the Breast Cancer Research Foundation and the National Cancer Institute (grant U54CA143798).
- Almendro V, Marusyk A, Polyak K: Cellular heterogeneity and molecular evolution in cancer. Annu Rev Pathol. 2013, 8: 277-302. 10.1146/annurev-pathol-020712-163923. [http://0-dx.doi.org.brum.beds.ac.uk/10.1146/annurev-pathol-020712-163923], [http://0-dx.doi.org.brum.beds.ac.uk/10.1146/annurev-pathol-020712-163923]PubMedView ArticleGoogle Scholar
- Navin N, Kendall J, Troge J, Andrews P, Rodgers L, McIndoo J, Cook K, Stepansky A, Levy D, Esposito D, Muthuswamy L, Krasnitz A, McCombie WR, Hicks J, Wigler M: Tumour evolution inferred by single-cell sequencing. Nature. 2011, 472: 90-94. 10.1038/nature09807. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nature09807], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nature09807]PubMedPubMed CentralView ArticleGoogle Scholar
- Nik-Zainal S, Van Loo P, Wedge DC, Alexandrov LB, Greenman CD, Lau KW, Raine K, Jones D, Marshall J, Ramakrishna M, Shlien A, Cooke SL, Hinton J, Menzies A, Stebbings LA, Leroy C, Jia M, Rance R, Mudie LJ, Gamble SJ, Stephens PJ, McLaren S, Tarpey PS, Papaemmanuil E, Davies HR, Varela I, McBride DJ, Bignell GR, Leung K, Butler AP, et al: The life history of 21 breast cancers. Cell. 2012, 149: 994-1007. 10.1016/j.cell.2012.04.023.PubMedPubMed CentralView ArticleGoogle Scholar
- Ding L, Ley TJ, Larson DE, Miller CA, Koboldt DC, Welch JS, Ritchey JK, Young MA, Lamprecht T, McLellan MD, McMichael JF, Wallis JW, Lu C, Shen D, Harris CC, Dooling DJ, Fulton RS, Fulton LL, Chen K, Schmidt H, Kalicki-Veizer J, Magrini VJ, Cook L, McGrath SD, Vickery TL, Wendl MC, Heath S, Watson MA, Link DC, Tomasson MH, et al: Clonal evolution in relapsed acute myeloid leukaemia revealed by whole-genome sequencing. Nature. 2012, 481: 506-510. 10.1038/nature10738. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nature10738], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nature10738]PubMedPubMed CentralView ArticleGoogle Scholar
- Junker JP, van Oudenaarden A: Every cell is special: genome-wide studies add a new dimension to single-cell biology. Cell. 2014, 157: 8-11. 10.1016/j.cell.2014.02.010. [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.cell.2014.02.010], [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.cell.2014.02.010]PubMedView ArticleGoogle Scholar
- Widmer DS, Hoek KS, Cheng PF, Eichhoff OM, Biedermann T, Raaijmakers MIG, Hemmi S, Dummer R, Levesque MP: Hypoxia contributes to melanoma heterogeneity by triggering HIF1a-dependent phenotype switching. J Invest Dermatol. 2013, 133: 2436-2443. 10.1038/jid.2013.115. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/jid.2013.115], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/jid.2013.115]PubMedView ArticleGoogle Scholar
- Anderson ARA, Weaver AM, Cummings PT, Quaranta V: Tumor morphology and phenotypic evolution driven by selective pressure from the microenvironment. Cell. 2006, 127: 905-915. 10.1016/j.cell.2006.09.042. [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.cell.2006.09.042], [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.cell.2006.09.042]PubMedView ArticleGoogle Scholar
- Castaño Z, Marsh T, Tadipatri R, Kuznetsov HS, Al-Shahrour F, Paktinat M, Greene-Colozzi A, Nilsson B, Richardson AL, McAllister SS: Stromal EGF and igf-I together modulate plasticity of disseminated triple-negative breast tumors. Cancer Discov. 2013, 3: 922-935. 10.1158/2159-8290.CD-13-0041. [http://0-dx.doi.org.brum.beds.ac.uk/10.1158/2159-8290.CD-13-0041], [http://0-dx.doi.org.brum.beds.ac.uk/10.1158/2159-8290.CD-13-0041]PubMedPubMed CentralView ArticleGoogle Scholar
- Merlo LMF, Pepper JW, Reid BJ, Maley CC: Cancer as an evolutionary and ecological process. Nat Rev Cancer. 2006, 6: 924-935. 10.1038/nrc2013. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nrc2013], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nrc2013]PubMedView ArticleGoogle Scholar
- Almendro V, Cheng YK, Randles A, Itzkovitz S, Marusyk A, Ametller E, Gonzalez-Farre X, Muñoz M, Russnes HG, Helland A, Rye IH, Borresen-Dale AL, Maruyama R, van Oudenaarden A, Dowsett M, Jones RL, Reis-Filho J, Gascon P, Gönen M, Michor F, Polyak K: Inference of tumor evolution during chemotherapy by computational modeling and in situ analysis of genetic and phenotypic cellular diversity. Cell Rep. 2014, 6: 514-527. 10.1016/j.celrep.2013.12.041. [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.celrep.2013.12.041], [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.celrep.2013.12.041]PubMedPubMed CentralView ArticleGoogle Scholar
- Almendro V, Kim HJ, Cheng YK, Gönen M, Itzkovitz S, Argani P, van Oudenaarden A, Sukumar S, Michor F, Polyak K: Genetic and phenotypic diversity in breast tumor metastases. Cancer Res. 2014, 74: 1338-1348. 10.1158/0008-5472.CAN-13-2357-T. [http://0-dx.doi.org.brum.beds.ac.uk/10.1158/0008-5472.CAN-13-2357-T], [http://0-dx.doi.org.brum.beds.ac.uk/10.1158/0008-5472.CAN-13-2357-T]PubMedPubMed CentralView ArticleGoogle Scholar
- Fuchs TJ, Buhmann JM: Computational pathology: challenges and promises for tissue analysis. Comput Med Imaging Graph. 2011, 35: 515-530. 10.1016/j.compmedimag.2011.02.006. [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.compmedimag.2011.02.006], [http://0-dx.doi.org.brum.beds.ac.uk/10.1016/j.compmedimag.2011.02.006]PubMedView ArticleGoogle Scholar
- Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, Guertin DA, Chang JH, Lindquist RA, Moffat J, Golland P, Sabatini DM: CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 2006, 7: R100-10.1186/gb-2006-7-10-r100. [http://0-dx.doi.org.brum.beds.ac.uk/10.1186/gb-2006-7-10-r100], [http://0-dx.doi.org.brum.beds.ac.uk/10.1186/gb-2006-7-10-r100]PubMedPubMed CentralView ArticleGoogle Scholar
- de Chaumont F, Dallongeville S, Chenouard N, Pop S, Provoost T, Meas-Yedid V, Pankajakshan P, Lecomte T, Le Montagner Y, Lagache T, Dufour A, Olivo-Marin JC: Icy: an open bioimage informatics platform for extended reproducible research. Nat Methods. 2012, 9: 690-696. 10.1038/nmeth.2075. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nmeth.2075], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nmeth.2075]PubMedView ArticleGoogle Scholar
- Johnston J, Nagaraja A, Hochheiser H, Goldberg I: A flexible framework for Web interfaces to image databases: supporting user-defined ontologies and links to external databases. ISIB: IEEE2006:1380–1383.Google Scholar
- Schneider CA, Rasband WS, Eliceiri KW: NIH Image to ImageJ: 25 years of image analysis. Nat Methods. 2012, 9: 671-675. 10.1038/nmeth.2089.PubMedView ArticleGoogle Scholar
- Wang Q, Niemi J, Tan CM, You L, West M: Image segmentation and dynamic lineage analysis in single-cell fluorescence microscopy. Cytometry A. 2010, 77: 101-110. [http://0-dx.doi.org.brum.beds.ac.uk/10.1002/cyto.a.20812], [http://0-dx.doi.org.brum.beds.ac.uk/10.1002/cyto.a.20812]PubMedPubMed CentralGoogle Scholar
- Lyubimova A, Itzkovitz S, Junker JP, Fan ZP, Wu X, van Oudenaarden A: Single-molecule mRNA detection and counting in mammalian tissue. Nat Protoc. 2013, 8: 1743-1758. 10.1038/nprot.2013.109. [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nprot.2013.109], [http://0-dx.doi.org.brum.beds.ac.uk/10.1038/nprot.2013.109]PubMedView ArticleGoogle Scholar
- Linkert M, Rueden CT, Allan C, Burel JM, Moore W, Patterson A, Loranger B, Moore J, Neves C, Macdonald D, Tarkowska A, Sticco C, Hill E, Rossner M, Eliceiri KW, Swedlow JR: Metadata matters: access to image data in the real world. J Cell Biol. 2010, 189: 777-782. 10.1083/jcb.201004104. [http://0-dx.doi.org.brum.beds.ac.uk/10.1083/jcb.201004104], [http://0-dx.doi.org.brum.beds.ac.uk/10.1083/jcb.201004104]PubMedPubMed CentralView ArticleGoogle Scholar
- Otsu N: A Threshold Selection Method from Gray-Level Histograms. IEEE Trans Syst Man Cybernet. 1979, 9: 62-66. 10.1109/TSMC.1979.4310076.View ArticleGoogle Scholar
- Soille P: Morphological image analysis: principles and applications. 1999, Inc., Springer-Verlag New YorkView ArticleGoogle Scholar
- Meyer F, Beucher S: Morphological segmentation. J Vis Commun Image Represent. 1990, 1: 21-46. 10.1016/1047-3203(90)90014-M. [http://0-www.sciencedirect.com.brum.beds.ac.uk/science/article/pii/104732039090014M], [http://0-www.sciencedirect.com.brum.beds.ac.uk/science/article/pii/104732039090014M]View ArticleGoogle Scholar
- Chan TF, Vese LA: Active contours without edges. IEEE Trans Image Process. 2001, 10: 266-277. 10.1109/83.902291.PubMedView ArticleGoogle Scholar
- Lankton S, Tannenbaum A: Localizing region-based active contours. IEEE Trans Image Process. 2008, 17: 2029-2039. 10.1109/TIP.2008.2004611. [http://0-dx.doi.org.brum.beds.ac.uk/10.1109/TIP.2008.2004611], [http://0-dx.doi.org.brum.beds.ac.uk/10.1109/TIP.2008.2004611]PubMedPubMed CentralView ArticleGoogle Scholar
- Wolff AC, Hammond MEH, Hicks DG, Dowsett M, McShane LM, Allison KH, Allred DC, Bartlett JMS, Bilous M, Fitzgibbons P, Hanna W, Jenkins RB, Mangu PB, Paik S, Perez EA, Press MF, Spears PA, Vance GH, Viale G, Hayes DF: Recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical oncology/College of American pathologists clinical practice guideline update. J Clin OncolL: Official J Am Soc Clin Oncol. 2013, 31: 3997-4013. 10.1200/JCO.2013.50.9984. doi:10.1200/JCO.2013.50.9984, http://0-dx.doi.org.brum.beds.ac.uk/10.1200/JCO.2013.50.9984., [http://0-dx.doi.org.brum.beds.ac.uk/10.1200/JCO.2013.50.9984]View ArticleGoogle Scholar
- Gown AM: Current issues in ER and HER2 testing by IHC in breast cancer. Mod Pathol. 2008, 21 Suppl 2: S8-S15. 10.1038/modpathol.2008.34. [http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/18437174],PubMedView ArticleGoogle Scholar
- Yuan Y, Failmezger H, Rueda OM, Ali HR, Gräf S, Chin SF, Schwarz RF, Curtis C, Dunning MJ, Bardwell H, Johnson N, Doyle S, Turashvili G, Provenzano E, Aparicio S, Caldas C, Markowetz F: Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Sci Transl Med. 2012, 4: 157ra143-10.1126/scitranslmed.3004330. [http://0-dx.doi.org.brum.beds.ac.uk/10.1126/scitranslmed.3004330], [http://0-dx.doi.org.brum.beds.ac.uk/10.1126/scitranslmed.3004330]PubMedView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.