1. Introduction
1.6. Network analysis techniques
1.6.4. Methods for graph kernel computation
Computing a kernel matrix of an extremely large graph is a computationally expensive task (the time complexity is , where is the number of nodes in the graph). The above presented graph kernels can be classified into two groups, one is kernels based on von Neumann series, and the others are kernels based on matrix exponential. It is well known that computing the inverse or matrix exponential of a general, large matrix is a computationally challenging problem. The size of a human protein-protein interaction network is approximately 20000 x 20000 (this is considered to be large). Since the computational time is considerable, finding the optimal kernel parameters for an application has also become challenging. To overcome this limitation, approximation algorithm can be used, such as Arnoldi algorithm [209], or Cholesky decomposition [211]. Further advantage of that algorithms is that they exploit the graph sparsity, which is an inherent property of the biological networks [212].
It is a frequent case in practice, that not the whole kernel matrix is needed, but the product of the kernel with a vector, and in that case, Arnoldi algorithm can be extremely useful for computing that product [207, 209, 213].
The computation of von Neumann series is also possible using series approximation, )
(n3
O n
Cholesky decomposition technique [211]. Although, this decomposition technique approach is
“direct”, thus it does not leverage the sparsity property directly, it is still possible to reduce computation time by transforming the adjacency matrix by using the reverse Cuthill–McKee algorithm [214].
1.6.4.1. Krylov space methods
The evaluation of matrix functions, such as matrix exponential or inverse, is often challenging in practice. In a number of such cases, Krylov subspace methods are easily applicable, especially in the time evolution of a complex system, but we can use them in several other fields, such as solving linear equations, finding the eigenvalues of a matrix, and evaluation of square root or trigonometric functions [205].
Let matrix be large and sparse. Then Krylov space methods are useful, especially if we want to evaluate the following matrix vector product:
( ) , where N N, N
f A b A b
: is a function for which ( ) is defined
f D f A
(24)
In most applications, only the product ( )f A b, and not ( )f A is needed. For example, when we solve the linear equationAxb, the solution isA b1 , thus the solution is the result of the multiplication of the inverse matrix withb. Another example of solving a system of ordinary differential equations given: ( )
dx t ( )
dt Ax t , and (0)x as the initial condition. Then the solution is ( ) At (0)
x t e x . In that case, the solution is also a vector, given by a matrix vector product.
Krylov space algorithms are iterative methods, and they give an approximation of the solution with an acceptable error within a reasonable computational time. They have some further advantages over other iterative methods, namely the matrixA is required only for computing matrix vector products. It is a very advantageous property while working with large sparse matrices.
Usually, the computation cost of a matrix vector product is O N( N), but it is easy to construct
data structures and procedures, for which the cost of evaluating of Ab is only O(N), where is the number of nonzero elements per row. Another important benefit is that the approximation of
( )
f A b for a smooth function f ( f is continuously differentiable, such as the exponential function) converges super linearly [207].
The family of Krylov space methods has many members. The Generalized minimal residual method (GMRES) [215], Conjugate gradient method (CGN) [216], Biconjugate gradient method (BiCG) [217] solve linear equation (you can decide whether your matrix is hermitian or not). Based on Arnoldi and Lánczos algorithm, it is possible to find the extreme eigenvalues of A as well.
1.6.4.2. Arnoldi algorithm
Most of the Krylov space methods are based on the outcome of the Arnoldi algorithm. The main idea is to project matrix A to a smaller space, called Krylov subspace, and looking for a solution in this smaller space. Arnoldi algorithm computes the projection of A, and builds an orthogonal basis on the Krylov subspace, and this projection can be seen as the compressed version of A[209].
Krylov subspace is a vector space spanned by the vectors of Krylov sequence, Ks which is a set of vectors generated by A and b:
, , 2 , , m 1
Ks b Ab A b A b (22)
The mth Krylov subspace of A N N and b0,b N is:
, , 2 , , m1
( ) : 1
m m
K span b Ab A b A b q A b qP (23)
Pm are the polynomials with degree m. If A is sparse, we can generate Ks very easily since in every iteration we just multiply A with the result vector of the previous iterations.
The approximation of the matrix function ( )f A b with the help of Krylov space is done by
where A N N , v is a nonzero vector and pm1 is a polynomial with degree m-1. The approximation is also the element of the Krylov subspace (Km). The goal is to find an element in
Km that approximates ( )f A b.
The Arnoldi approximation of ( )f A b is defined by [209]:
( ) ,1 where =
m m m
f Q f H e b (25)
The rationality behind this approximation is that Hm represents the compression of A into ( , )
Km A b with respect to the basis Qm and bQ em 1, where em denotes the mth unit coordinate vector. The matrix values of Qm and, Hm as calculated by a modified Gramm-Schmidt process [209] that generates a m orthonormal vectors in Km.
1.6.4.3. Approximating the matrix exponential
The exponential of a matrix plays an important role in many applications such as in the analysis of networks. Although formula 25 computes a matrix vector product, it is possible to get the full exp(A) column by column. One gets back the ith column of eA by choosing b as ei, where ei N N and ei is the unit coordinate vector. The approximation formula for ith column of matrix exponential of A is: