Conditional random field approach to prediction of protein-protein interactions using domain information

Understanding of protein functions and protein-protein interactions is one of important topics in the field of molecular biology and bioinformatics. Recently, many researchers have focused on the investigation of amino acid residues of proteins to reveal interactions and contacts between residues [1–4]. If residues at important sites for interactions between proteins are substituted in one protein, the corresponding residues in interacting partner proteins are expected to be also substituted by selection pressure. Otherwise, such mutated proteins may lose the interactions. Fraser et al. confirmed that interacting proteins evolve at similar evolutionary rates by comparing putatively orthologous protein sequences between S. cerevisiae and C. elegans[5]. It means that substitutions for contact residues occur in both interacting proteins as long as the proteins keep interacting with each other. Therefore, mutual information (MI) between residues is useful for predicting protein-protein interactions for proteins of unknown function. MI is calculated from multiple sequence alignments for homologous protein sequences. Weigt et al. identified direct residue contacts between sensor kinase and response regulator proteins by message passing, which is an improvement of MI [4]. Burger and van Nimwegen used a dependence tree where a node corresponds to a position of amino acid sequences, and predicted interactions using a Bayesian network method [2]. On the other hand, Markov random field and conditional random field models have been well studied in fields of natural language processing [6, 7]. Also in bioinformatics, protein function prediction methods from protein-protein interaction network and other biological networks were developed using Markov random fields [8, 9]. On the other hand, several prediction methods have been developed based on domain-domain interactions. Deng et al. proposed a domain-based probabilistic model of protein-protein interactions, and developed EM (Expectation Maximization) method [10]. Based on this probabilistic model, LP (Linear Programming)-based methods were developed [11], and Chen et al. improved the accuracy of interaction strength prediction by APM (Association Probabilistic Method) [12]. In this paper, we propose prediction methods based on domain-domain interactions using conditional random fields with and without mutual information. Furthermore, we perform computational experiments for several protein-protein interaction datasets, compare the methods with the EM method proposed by Deng et al. [10], which is one of the best predictors based on domain-domain interactions, and the association method proposed by Sprinzak and Margalit [13] (the APM method for binary interaction data is equivalent to the association method), and show that our methods outperform the EM method and the association method.

In order to investigate the relationship between two positions of proteins, MI for distributions of amino acids at the positions is used. Such distributions can be obtained from multiple alignments of protein sequences and domain sequences. In this section, we briefly review MI for distributions of amino acids, and explain MI between domains.

We assume that multiple sequence alignments for domains D _m and D _n are obtained, respectively (see Figure 1). In order to calculate MI, we need joint appearance frequencies. However, we cannot see which sequence in the multiple alignment of domain D _m corresponds to a specified sequence in that of D _n . Therefore, we assume that sequences contained in the same organism can be paired. In the example of Figure 1, the second sequence of D _m is paired with the first one of D _n , the third one of D _m is paired with the second one of D _n , and so on. The first sequence of D _m is not counted into the appearance frequencies because it is not paired with any sequence of D _n although it may be paired with sequences of other domains than D _n .

Let A be a set of amino acids, f _i (A) be the appearance frequency of amino acid A at position i in domains D _m and D _n , and f _ij (A, B) be the joint appearance frequency of a pair of amino acids A at position i in D _m and B at position j in D _n , where each frequency is divided by the number of paired sequences M in the multiple alignments such that ∑_A∈Af _i (A) = ∑_A,B∈Af _ij (A,B) = 1.

Multiple alignments often include some gaps. Weigt et al. counted the frequencies of gaps as well as amino acids [4]. Therefore, we also consider gaps to be a kind of amino acids, that is, the number of distinct amino acids is |A| = 21. Then, mutual information for positions i in D _m and j in D _n is defined as the Kullback-Leibler divergence between the multiplication of appearance frequencies, f _i (A)f _j (B), and the joint appearance frequencies, f _ij (A,B), as follows.

(1)

If frequency distributions of amino acids at positions i and j are independent from each other, f _ij (A,B) ≈ f _i (A)f _j (B), and MI _ij approaches to zero. This means that the two positions are not related with each other in the evolutionary process. If domains D _m and D _n interact at the positions, it is considered that MI _ij becomes high because the positions have coevolved through the evolutionary process in order to keep the interaction. It should be noted that two positions i and j do not always directly interact even if MI _ij is high [4]. However, such proteins with high values of MI have a possibility to directly interact with each other at other positions in the proteins.

However, we need to reduce MI _ij because it can be unnecessarily high depending on distributions of f _i (A) and f _j (B). For that purpose, we make use of , which is the mutual information MI _ij from the joint frequency, f _ij (A, B), obtained by shuffling at random the combinations of sequences in multiple alignments. In this paper, we repeat the procedure 400 times according to [4], and take the average. For practical uses of MI, f _i (A), f _j (B) and f _ij (A,B) should be positive values. Otherwise, we cannot calculate MI _ij by using computers. Therefore, we use the following pseudocount as in [4],

(2)

(3)

where η is a constant value, in this paper we use η = 1. It should be noted that the sum over all amino acids A, and because ∑_A∈Af _i (A) = ∑_A,B∈Af _ij (A,B) = 1.

In order to investigate interactions between proteins, we need MI between domains included in the proteins. Thus, we define MI between domains D _m and D _n , M _mn , to be the maximum of MI over all positions i and j as follows.

(4)

where 〈v〉 means the average of v, i and j are positions of D _m and D _n , respectively. Since MI _ij is calculated to be high for the positions i and j that include many gaps, we exclude positions that include more than 20% gaps as in [14].

In this section, we propose a probabilistic model for protein-protein and domain-domain interactions using conditional random fields [6, 7] because it can be considered that two domains D _m and D _n do not always interact even if the mutual information M _mn is large. For example, Weigt et al. improved MI and proposed direct information (DI) because residues do not always contact with each other even if the MI is large [4]. Most proteins contain domains as is well known. If two proteins do not interact with each other, any two domains contained in the proteins must not interact with each other. In the left example of Figure 2, protein P _i consists of domains D₁ and D₂ and protein P _j consists of domain D₃ respectively. If P _i and P _j do not interact, any pair of (D₁, D₃) and (D₃, D₃) does not interact. Deng et al. proposed a probabilistic model for a pair of proteins as follows [10]. By assuming that proteins P _i and P _j interact if and only if at least a pair of domains included in the proteins interacts, and events that domains interact are independent from each other, they defined

(5)

where P _ij = 1 means that proteins P _i and P _j interact, D _mn = 1 means that domains D _m and D _n interact, D _mn ∈ P _ij means that domain D _m is included in protein P _i and D _n is included in P _j and the product in the right hand side is calculated for all domain pairs (D _m , D _n ) included in the protein pair (P _i , P _j ). By transforming equation (5), we have

(6)

(7)

where λ^(mn) = log(1 – Pr(D _mn = 1)).

From this equation, we can consider the following Markov random field model for protein pair (P _i , P _j ) (see Figure 2).

(8)

where p _ij ∈ {0, 1}, d means a set of events on domain-domain interactions, D _mn = d _mn (d _mn ∈ {0, 1}), denotes a local feature, is the corresponding weight parameter and related to the joint probability Pr(P _ij = s, D _mn = t), and Z _ij denotes the normalization constant. For instance, equation (8) for p _ij = 0 is equivalent to equation (7) in the case that for all protein pairs (P _i , P _j ) and if s = t = 0, otherwise 0.

In Markov random fields, random variables have Markov properties represented as an undirected graph [15]. The factor graph for our model is represented to be a bipartite graph G(U, V, E) with a set of vertices U corresponding to protein-protein interactions P _ij , a set of vertices V corresponding to domain-domain interactions D _mn , and a set of edges E between U and V as the right figure of Figure 2. There exists an edge between P _ij ∈ U and D _mn ∈ V if and only if D _mn ∈ P _ij . For the left example of Figure 2, protein pair (P _i , P _j ) includes domain pairs (D₁, D₃) and (D₂, D₃). Then, in the factor graph, the vertex of P _ij is connected with vertices of D₁₃ and D₂₃, respectively. Although the vertex of P _ij does not have other adjacent vertices than the vertices of D₁₃ and D₂₃, those of D₁₃ and D₂₃ can be connected with other vertices than that of P _ij

Since Pr(P _ij = 0|D _mn =t) = 1 – Pr(P _ij = 1|D _mn = t), it is redundant to consider both s = 0, 1, and it is sufficient to consider only s = 1. Therefore, in order to simplify the model, we substitute , , and for all protein pairs (P _i , P _j ). Then, we have the following joint probability,

(9)

where p means a set of events on protein-protein interactions, P _ij = p _ij .

We here introduce mutual information between domains M = {M _mn } as given conditional data in order to combine it with the probabilistic model. Then, equation (9) can be written as

(10)

where

(11)

(12)

σ(x) = 1/(1 + e^–x) is an increasing function, and c is a positive constant. It should be noted that a negative value, –1, is given to because it is undesired that a pair of domains interact although proteins having the pair do not interact. In this way, the local feature correlates protein-protein interactions P _ij with domain-domain interactions D _mn (see Figure 2).

For a conditional random field model without MI, we use the following local feature instead of .

(13)

Parameter estimation

In this section, we discuss how to estimate the parameters . We assume that protein-protein interaction data p = {p _ij } are given. Then, the likelihood function is represented by

(14)

where Z(M) = ∏ _{p ij∈p}Z _ij (M). By taking the logarithm, we have

(15)

We estimate the parameters by maximizing the log-likelihood function, l(λ). Since log(e ^x + e ^y ) is a convex function for variables x and y, that is, l(λ) is a concave function, we are able to obtain a global maximum. For maximizing such functions, various methods such as the steepest descent method, Newton’s method, and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) [16] method have been developed. Newton’s method calculates the inverse of the Hessian matrix for the objective function. However, the computational cost is high. Therefore, the quasi-Newton method approximates the matrix by some efficient method using the first derivatives, the gradient. In this paper, we use the BFGS method, which is one of the quasi-Newton methods. By differentiating equation (15) partially with respect to each parameter , we have

(16)

In the BFGS method, this equation is repeatedly applied for updating a solution.

We proposed novel methods which combine conditional random fields with the domain-based model of protein-protein interactions. In order to give better performance, we introduced mutual information to the probabilistic model. In the improved model, mutual information between domains is given as conditions, where MI between domains is defined as the maximum of MIs between residues in the domains. This method was developed based on the fact that amino acid residues at important sites for interactions have coevolved with each other, and MI has been used for identifying contact residues in interactions. We performed five-fold cross-validation experiments, and calculated AUC for probabilities that two proteins interact. The results suggested that our proposed methods, especially the CRF method with mutual information, are useful. However, the results of AUC for training datasets implied that estimated parameters were overfitting to training datasets. For avoiding that problem, we can improve the methods, for instance, by adding regularization terms, l₁-norm of parameters to the log-likelihood function. Since CRF has an advantage to be able to incorporate large number of features, it remains as a future work to improve the model itself to obtain better accuracy by, for instance, modifying the local feature and adding new features.

JS proposed the use of mutual information for predicting protein-protein interactions. Methods were developed and implemented by MH. MK and TA participated in the discussion during development of the methods. The manuscript was prepared by MH, JS, and TA.