User Guide

PiNGO

PiNGO Installation Instructions

1. PiNGO is a plugin for the Cytoscape platform. PiNGO 1.11 is compatible with Cytoscape version 2.8, not with earlier versions. Cytoscape and PiNGO require Java. If not already installed on your computer, download and install the Java 2 Runtime Environment, version 1.6.0 or higher.

2. After installing Cytoscape, install BiNGO (2.42 or higher) and PiNGO using the Cytoscape Plugins Manager (under Plugins menu). PiNGO requires BiNGO to work.

PiNGO is now ready to use !

PiNGO Manual

0. Introduction

PiNGO is a Java-based tool to find unknown genes in a molecular network that are significantly associated with user-defined target Gene Ontology (GO) categories. PiNGO is implemented as a plugin for Cytoscape, a popular open source software platform for visualizing and integrating molecular interaction networks. PiNGO predicts the categorization of a gene based on the annotations of its neighbors, using the enrichment statistics of its sister tool BiNGO. Networks can either be selected from the Cytoscape interface or uploaded from file. The main advantage of PiNGO is its flexibility. PiNGO also takes full advantage of Cytoscape's versatile visualization environment.

Features include :

•Cytoscape graph or network file input
•Limit your candidate gene search to genes already annotated to specific GO categories, or to genes not annotated to certain GO categories, or both
•Customizable annotations, ontologies and reference sets, not limited to GO
•Evidence code filtering
•Hypergeometric or binomial test for enrichment analysis
•Multiple testing correction using Bonferroni (FWER) or Benjamini&Hochberg (FDR) correction
•Interactive visualization of results
•extensive results in tab-delimited text file format
•open source

This manual aims to explain the inner workings, and potential pitfalls, of PiNGO in more detail. To get a taste of the basic user interface, please take a look at the tutorial section.

1.Custom input network format

PiNGO accepts custom networks in a tab-delimited text file format with at least two columns, e.g.

#GENE_A GENE_B

gene1 gene2

gene1 gene3

gene1 gene4

gene1 gene5

gene2 gene3

gene2 gene4

gene2 gene5

...

where the whitespaces are tabs. Each line represents an edge in the network going from gene x to gene y (edge direction does not matter). Separating genes with regular spaces will not work. Extra columns (e.g. edge weights) can be present but are not taken into account in the current version of PiNGO. Header lines are not necessary, but if they’re there, they should start with ‘#’.

2.Start, Filter and Target GO categories and use cases

Start GO categories: the existing GO annotations a candidate gene should have

Filter GO categories: existing GO annotations a candidate gene should not have

Target GO categories: GO categories for which you want to predict candidate genes

The GO category ID for a term can be found with tools like AMIGO. You should only paste numbers in the GO category boxes, not the GO:00... part of the IDs.

Use case 1: You want to find candidate genes in your network that are associated with some process, say ‘response to cold stress’ (70417). You’re only interested in transcription factors (3700) and signal transducers (4871) that are not already annotated to the ‘response to cold stress’ category. Additionally, you’re not interested in genes that are known to be involved in ‘response to other organism’ (51707).

Start GO categories: 3700 4871

Filter GO categories: 70417 51707

Target GO categories: 70417

Use case 2: You want to check how many known ‘response to cold stress’ genes are predicted to be involved in ‘response to cold stress’ on the basis of an input network. This may serve as an indicator of the usefulness of the network for studying cold stress (note that you only assess the recall or sensitivity of the network-based predictions here, not the precision).

Start GO categories: 70417

Filter GO categories:

Target GO categories: 70417

3. Statistical tests

PiNGO currently provides two statistical tests, borrowed from BiNGO, to assess functional enrichment in a set of genes. The basic question answered by these tests is the following :

'When sampling X genes (test set) out of N genes (reference set ; graph or annotation), what is the probability that x or more of these genes belong to a functional category C shared by n of the N genes in the reference set.' In PiNGO’s case, the test set is the set of network neighbors of a candidate candidate gene.

The hypergeometric test (test without replacement) provides an accurate answer to this question in the form of a p-value. Its counterpart with replacement, the binomial test, provides only an approximate p-value but requires less calculation time. You should only consider using the binomial test if your test set contains several thousand genes...

4. Multiple testing corrections

Because PiNGO tests multiple genes and possibly multiple GO labels, the number of statistical tests performed in a single analysis may amount to several hundreds. When testing randomly connected genes for association with some GO category at a significance level of say 0.05, you would expect 5 out of each 100 tested genes to be positively identified just by chance. Suppose you have one gene genuinely associated with that GO category, this would imply you would have identified 5 times more false positive than true positive candidate genes. Multiple testing corrections are designed to provide better control over the false positive rate at a given significance level. Multiple testing corrections in PiNGO are always performed on all test results for all target categories simultaneously, regardless of the visualization mode (tabbed output or not).

One of the most basic corrections is the Bonferroni correction. The Bonferroni correction provides strong control over the Family-Wise Error Rate (FWER), which is defined as the probability of making at least one type I (false positive) error. E.g., when performing Bonferroni control of the FWER at level a = 0.05, you would be 95% certain that the over-represented categories that you identified contain no false positives. The Bonferroni correction is generally assumed to be rather conservative, although there have been reports (Boyle et al. 2004) that the Bonferroni correction would actually be rather liberal (at least for FWER control) when used for correcting tests that are not mutually independent, as is the case when testing GO categories (see further).

An alternative to using FWER controlling corrections is to control the False Discovery Rate (FDR), i.e. the expected proportion of false positives among the positively identified tests. Generally, this type of correction is more appropriate for our purposes, since we would typically rather have more power (less false negatives) at the cost of a few more false positives. One of the most popular FDR-corrections is the Benjamini & Hochberg correction, which provides strong control over the FDR under positive regression dependency of the null hypotheses.

In fact, it's not sure whether the PiNGO test structure (testing several genes for several GO categories) fulfills this positive regression dependency requirement. Nevertheless, the Benjamini & Hochberg correction is used widely. Alternatives include the Benjamini & Yekutieli procedure, which controls the FDR under arbitrary dependency, or resampling based procedures to control either the FWER (e.g. Westfall & Young step-down minP procedure) or the FDR (e.g. Storey & Tibshirani ST-q procedure, which calculates adjusted q-values instead of p-values). The latter procedures are rather computationally intensive, which is why they have not yet been implemented in PiNGO. The Benjamini & Yekutieli procedure exhibits severely decreased power compared to the Benjamini & Hochberg correction, which is a large price to pay for allowing arbitrary dependence.

For a more thorough discussion of this topic, see e.g. Ge et al. (2003).

Boyle, E.I., Weng, S., Gollub, J., Jin, H., Botstein, D., Cherry, J.M., Sherlock, G. (2004) GO::TermFinder--open source software for accessing Gene Ontology information and finding significantly enriched Gene Ontology terms associated with a list of genes, Bioinformatics 20, 3710-3715.

Ge, Y., Dudoit, S. and Speed, T.P. (2003) Resampling-based multiple testing for microarray data analysis, Technical Report 633, Dept. of Statistics, UC Berkeley

available at : http://stat-www.berkeley.edu/tech-reports/

5. Using Standard and Custom Annotations and Ontologies

The goal of the Gene Ontology (GO) project is to provide a structured description of known biological information at different levels of granularity. GO consists of three structured, controlled vocabularies that describe gene products in terms of their associated biological processes, molecular functions and cellular components in a species-independent manner.

The default annotations and ontologies in PiNGO are obtained from BiNGO. BiNGO provides several default GO ontologies and annotations for a wide range of organisms. The GO ontologies and annotations in BiNGO are parsed from information available at the NCBI (see version_information.txt in the BiNGO.jar file). The identifiers supported in the default annotation files usually include UniProt IDs, LocusTags, Official Gene Symbols and Unigene IDs, and some other identifiers dependent on the organism. Default identifiers supported for each organism can be found in the table below. For the identifiers in the other IDs column, you have to use the database as prefix to the identifier, e.g. TAIR:AT5G45880, HGNC:19074, MIM:612733, Ensembl:ENSG00000100296. We cannot guarantee that all identifier types cover all genes, as this depends on the completeness of the NCBI GO annotation files, which is beyond our control. In case of doubt, please check if the identifiers used in the default annotations, which can be found in the BiNGO.jar archive (usually hidden in .cytoscape/2.7/plugins/ in your home folder), correspond to the ones you use in your network/test set. If not, you should either change the identifiers in your network/test set or create a custom annotation file.

IMPORTANT: the default annotations and ontologies, obtained from BiNGO, are being phased out. They will not be updated regularly. We recommend you use custom annotation and ontology .obo files available on the GO website (www.geneontology.org). Download the files from your species of interest and specify them in the PiNGO annotation and ontology choice panels under ‘Custom...’.

One way to avoid the issues related to multiple testing is to test fewer categories. This option is especially attractive if you're only interested in more general functional profiling anyway. To this end, we provide several GOSlim ontologies in PiNGO, which are (organism-specific) slimmed-down versions of the full GO hierarchy. When using these default GOSlims in combination with either standard or custom annotations, the provided annotation is automatically remapped by PiNGO onto the chosen GOSlim, using the default full GO ontology as a remapping guide. Similar automatic remapping also occurs when you use .obo ontology files downloaded from GO. However, when you would envision building and using a custom GOSlim of your own, annotation files will NOT be remapped in the same way. When using custom ontologies, remapping will only occur within your custom ontology (i.e. from specific nodes in the custom ontology to its parents along the custom hierarchy). Please make sure that you specify an appropriate custom annotation file accordingly. You can no longer use the default organism/annotation options, since these provide annotation on the full GO, typically at more specific levels in the GO hierarchy. These annotations cannot be remapped appropriately onto your custom ontology. To make sure that you're using an appropriate annotation file, PiNGO will issue a warning message when some of the GO labels in your annotation don't match with those of the chosen ontology. If you're not working with GOSlims, this might also indicate that you're using annotation and ontology files of different versions (GO is still very much under development, with new labels being added regularly).

6. Which reference set to use ?

Choosing the appropriate reference set against which your genes of interest will be tested depends very much on the problem under study. When you wish to assess the enrichment of functional categories in a gene’s neighborhood relative to the genes in the network, you can select the Use network as reference set option in the Select Reference Set dropdown box of the PiNGO Settings Panel. You can then choose any of the organism/annotation and ontology options provided, and the appropriate reference set annotation will be parsed automatically. However, you typically want to test enrichment against a bigger reference set, e.g. all genes in the genome, all genes on a microarray or all genes screened in a Y2H screen. You can Use whole annotation as reference set in some cases, but be aware that some annotations, e.g. some annotation files from the GO consortium, may contain many more entries than you care to use as a reference. The annotation files for some organisms are protein-centric, while others are gene-centric. The annotation file for Arabidopsis, for example, contains splice variants of genes, resulting in approx. 50,000 annotated entities, although there are only approx. 25,000 ORFs. If you do not want to include splice variants in your analysis, you'll have to use a custom reference set containing all Arabidopsis loci without splice variants. In case of doubt, it’s always better to use a custom reference set. Custom reference set files simply contain all gene identifiers you want to include in the reference set, separated by newlines. E.g.

AT5G67110

AT5G67120

AT5G67130

AT5G67140

AT5G67150

AT5G67160

AT5G67170

AT5G67180

AT5G67190

AT5G67200

AT5G67210

AT5G67220

AT5G67230

AT5G67240

...

7. Interpretation of PiNGO results

PiNGO provides the user with a set of candidate genes for the chosen target categories, with restrictions set by start and filter categories. Subcategories of the target categories are also screened. The p-values associated with candidate genes in combination with frequency counts give a good indication of the prominence of a given target category in a candidate gene’s neighborhood and may be useful in the prioritization of candidate genes. But p-values do not tell everything. If available, gene descriptions and known annotations of the candidate genes are also reported to facilitate interpretation of the results. In many cases, you may spot some ‘not-so-novel’ genes in the resulting candidate gene list, in the sense that their involvement in the chosen target process may already be known, but not annotated in GO.

For interpretation of PiNGO figures, see tutorial.

8. Manipulating and saving PiNGO graphs

The controls and options for visualizing and modifying PiNGO graphs are basically the same as for other Cytoscape graphs. You can modify colors, labels, node form and size... through the Set Visual Properties option under the Cytoscape Visualization menu. A visual styles menu should pop up that allows you to modify the current Visual Style. Please note that each PiNGO graph has its own visual style, identified as PiNGO_<cluster name>. The basic reason for this is that the mapping of data attributes (e.g. p-values) to visual attributes (e.g. color) is different for every analysis you perform. Because you can switch back and forth between several PiNGO networks, all these attributes need to be stored separately in order to avoid confusion. DON'T change the visual style of one PiNGO network to another. This will cause all visual attributes of the nodes in the second network, such as node size (~ # of genes annotated to that node) and color (~ p-value), to change to the values calculated for those nodes in the first network. Overall, you should avoid renaming networks (also when saving them) or visual styles, since this can give rise to similar phenomena on rare occasions.

You can map various attributes to the node labels. A few of them need further explanation :

•x_<cluster name> : the number of genes in a gene’s neighborhood annotated to a certain GO class
•X_<cluster name> : the total number of genes in a gene’s neighborhood.
•n_<cluster name> : the number of genes in the reference set (graph or annotation) annotated to a certain GO class
•N_<cluster name> : the total number of genes in your reference set.

You can save PiNGO figures just as any other Cytoscape figures in a number of formats, available under File>Export>Network As Graphics... . If you want to store your PiNGO graph for future use in Cytoscape, save your Cytoscape session as a .cys file. Alternatively, you can choose to Export>Network and attributes as XGMML... (preferrably with the same name as the cluster name you used in PiNGO, to keep things coherent). However, then you should also Export>Vizmap Property File to save the visual style. The visual style will be necessary when restoring your graph at a later time. The PiNGO visual styles DO NOT get saved automatically upon exiting Cytoscape (otherwise you would get a flood of PiNGO visual styles after a while), in contrast to visual styles you create from within Cytoscape.

When you would, at some point in time, load in two or more PiNGO graphs from saved files, and you chose not to follow the naming guidelines outlined above, you have to make sure that the names of the attributes in your two PiNGO networks are different. Otherwise, one of them will definitely get the wrong attributes mapped to its nodes. Likewise, if you did follow the naming guidelines, never load 2 networks with the same name at the same time.