biocore / gemelli / 125

pending completion

Build # 125

Build Type

Pull #36

travis-ci

Committed by

web-flow

Commit Message

<a href="https://github.com/biocore/gemelli/commit/<a class=hub.com/biocore/gemelli/commit/<a class="double-link" href="https://git"><a class=hub.com/biocore/gemelli/commit/<a class="double-link" href="https://git"><a class=hub.com/biocore/gemelli/commit/<a class="double-link" href="https://git"><a class=hub.com/biocore/gemelli/commit/f2dbb535e6abe434e3f55196eae525b90e60a6fa">f2dbb535e">&lt;a href=&quot;https://github.com/biocore/gemelli/commit/</a><a class="double-link" href="https://github.com/biocore/gemelli/commit/&lt;a class=&quot;double-link&quot; href=&quot;https://git">&lt;a class=</a>hub.com/biocore/gemelli/commit/&lt;a class=&quot;double-link&quot; href=&quot;https://git">&lt;a class=</a>hub.com/biocore/gemelli/commit/&lt;a class=&quot;double-link&quot; href=&quot;https://git">&lt;a class=</a>hub.com/biocore/gemelli/commit/&lt;a class=&quot;double-link&quot; href=&quot;https://git">&lt;a class=</a>hub.com/biocore/gemelli/commit/f2dbb535e6abe434e3f55196eae525b90e60a6fa">f2dbb535e</a><a href="https://github.com/biocore/gemelli/commit/f2dbb535e6abe434e3f55196eae525b90e60a6fa">&quot;&gt;&amp;lt;a href=&amp;quot;https://github.com/biocore/gemelli/commit/&lt;/a&gt;&lt;a class=&quot;double-link&quot; href=&quot;https://github.com/biocore/gemelli/commit/&amp;lt;a class=&amp;quot;double-link&amp;quot; href=&amp;quot;https://git&quot;&gt;&amp;lt;a class=&lt;/a&gt;hub.com/biocore/gemelli/commit/&amp;lt;a class=&amp;quot;double-link&amp;quot; href=&amp;quot;https://git&quot;&gt;&amp;lt;a class=&lt;/a&gt;hub.com/biocore/gemelli/commit/&amp;lt;a class=&amp;quot;double-link&amp;quot; href=&amp;quot;https://git&quot;&gt;&amp;lt;a class=&lt;/a&gt;hub.com/biocore/gemelli/commit/&amp;lt;a class=&amp;quot;double-link&amp;quot; href=&amp;quot;https://git&quot;&gt;&amp;lt;a class=&lt;/a&gt;hub.com/biocore/gemelli/commit/f2dbb535e6abe434e3f55196eae525b90e60a6fa&quot;&gt;f2dbb535e&lt;/a&gt;&lt;a href=&quot;https://github.com/biocore/gemelli/commit/f2dbb535e6abe434e3f55196eae525b90e60a6fa&quot;&gt;&amp;quot;&amp;gt;&amp;amp;lt;a href=&amp;amp;quot;https://github.com/biocore/gemelli/commit/&amp;lt;/a&amp;gt;&amp;lt;a class=&amp;quot;double-link&amp;quot; href=&amp;quot;https://github.com/biocore/gemelli/commit/&amp;amp;lt;a class=&amp;amp;quot;double-link&am... (continued)

Pull Request Pull Request #36: Add RPCA and standalone RCLR functionality

Run Details

338 of 384 branches covered (88.02%)

Branch coverage included in aggregate %.

344 of 344 new or added lines in 11 files covered. (100.0%)

860 of 905 relevant lines covered (95.03%)

0.95 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

96.36

/gemelli/matrix_completion.py

# ----------------------------------------------------------------------------
# Copyright (c) 2019--, gemelli development team.
#
# Distributed under the terms of the Modified BSD License.
#
# The full license is in the file COPYING.txt, distributed with this software.
# ----------------------------------------------------------------------------

import numpy as np
from gemelli.optspace import OptSpace
from .base import _BaseImpute
from scipy.spatial import distance


class MatrixCompletion(_BaseImpute):

    def __init__(self, n_components=2, max_iterations=5, tol=1e-5):
        """

        This form of matrix completion uses OptSpace (1). Furthermore,
        here we directly interpret the loadings generated from matrix
        completion as a dimensionality reduction.

        Parameters
        ----------

        X: numpy.ndarray - a rclr preprocessed matrix of shape (M,N)
        N = Features (i.e. OTUs, metabolites)
        M = Samples

        n_components: int, optional : Default is 2
        The underlying rank of the default set
        to 2 as the default to prevent overfitting.

        max_iterations: int, optional : Default is 5
        The number of convex iterations to optimize the solution
        If iteration is not specified, then the default iteration is 5.
        Which redcues to a satisfactory error threshold.

        tol: float, optional : Default is 1e-5
        Error reduction break, if the error reduced is
        less than this value it will return the solution

        Returns
        -------
        U: numpy.ndarray - "Sample Loadings" or the unitary matrix
        having left singular vectors as columns. Of shape (M,n_components)

        s: numpy.ndarray - The singular values,
        sorted in non-increasing order. Of shape (n_components,n_components).

        V: numpy.ndarray - "Feature Loadings" or Unitary matrix
        having right singular vectors as rows. Of shape (N,n_components)

        solution: numpy.ndarray - (U*S*V.transpose()) of shape (M,N)

        distance: numpy.ndarray - Distance between each
        pair of the two collections of inputs. Of shape (M,M)

        Raises
        ------
        ValueError

        ValueError
            `ValueError: n_components must be at least 2`.

        ValueError
            `ValueError: max_iterations must be at least 1`.

        ValueError
            `ValueError: Data-table contains either np.inf or -np.inf`.

        ValueError
            `ValueError: The n_components must be less
             than the minimum shape of the input table`.

        References
        ----------
        .. [1] Keshavan RH, Oh S, Montanari A. 2009. Matrix completion
                from a few entries (2009_ IEEE International
                Symposium on Information Theory

        Examples
        --------
        TODO

        """

        self.n_components = n_components
        self.max_iterations = max_iterations
        self.tol = tol

        return

    def fit(self, X):
        """
        Fit the model to X_sparse
        """

        X_sparse = X.copy().astype(np.float64)
        self.X_sparse = X_sparse
        self._fit()
        return self

    def _fit(self):

        # make copy for imputation, check type
        X_sparse = self.X_sparse
        n, m = X_sparse.shape

        # make sure the data is sparse (otherwise why?)
        if np.count_nonzero(np.isinf(X_sparse)) != 0:
            raise ValueError('Contains either np.inf or -np.inf')

        # test n-iter
        if self.max_iterations < 1:
            raise ValueError("max_iterations must be at least 1")

        # check the settings for n_components
        if isinstance(self.n_components, str) and \
           self.n_components.lower() == 'auto':
            # estimate the rank of the matrix
            self.n_components = 'auto'
        # check hardset values
        elif isinstance(self.n_components, int):
            if self.n_components > (min(n, m) - 1):
                raise ValueError("n-components must be at most"
                                 " 1 minus the min. shape of the"
                                 " input matrix.")
            if self.n_components < 2:
                raise ValueError("n-components must "
                                 "be at least 2")
        # otherwise rase an error.
        else:
            raise ValueError("n-components must be "
                             "an interger or 'auto'")

        # return solved matrix
        self.U, self.s, self.V = OptSpace(n_components=self.n_components,
                                          max_iterations=self.max_iterations,
                                          tol=self.tol).solve(X_sparse)
        # save the solution (of the imputation)
        self.solution = self.U.dot(self.s).dot(self.V.T)
        self.eigenvalues = np.diag(self.s)
        self.explained_variance_ratio = list(
            self.eigenvalues / self.eigenvalues.sum())
        self.distance = distance.cdist(self.U, self.U)
        self.feature_weights = self.V
        self.sample_weights = self.U

    def fit_transform(self, X):
        """
        Returns the final SVD of

        U: numpy.ndarray - "Sample Loadings" or the
        unitary matrix having left singular
        vectors as columns. Of shape (M,n_components)

        s: numpy.ndarray - The singular values,
        sorted in non-increasing order. Of shape (n_components,n_components).

        V: numpy.ndarray - "Feature Loadings" or Unitary matrix
        having right singular vectors as rows. Of shape (N,n_components)

        """
        X_sparse = X.copy().astype(np.float64)
        self.X_sparse = X_sparse
        self._fit()
        return self.sample_weights, self.s, self.feature_weights

1	# ----------------------------------------------------------------------------
2	# Copyright (c) 2019--, gemelli development team.
3	#
4	# Distributed under the terms of the Modified BSD License.
5	#
6	# The full license is in the file COPYING.txt, distributed with this software.
7	# ----------------------------------------------------------------------------
8
9	import numpy as np	1✔
10	from gemelli.optspace import OptSpace	1✔
11	from .base import _BaseImpute	1✔
12	from scipy.spatial import distance	1✔
13
14
15	class MatrixCompletion(_BaseImpute):	1✔
16
17	def __init__(self, n_components=2, max_iterations=5, tol=1e-5):	1✔
18	"""
19
20	This form of matrix completion uses OptSpace (1). Furthermore,
21	here we directly interpret the loadings generated from matrix
22	completion as a dimensionality reduction.
23
24	Parameters
25	----------
26
27	X: numpy.ndarray - a rclr preprocessed matrix of shape (M,N)
28	N = Features (i.e. OTUs, metabolites)
29	M = Samples
30
31	n_components: int, optional : Default is 2
32	The underlying rank of the default set
33	to 2 as the default to prevent overfitting.
34
35	max_iterations: int, optional : Default is 5
36	The number of convex iterations to optimize the solution
37	If iteration is not specified, then the default iteration is 5.
38	Which redcues to a satisfactory error threshold.
39
40	tol: float, optional : Default is 1e-5
41	Error reduction break, if the error reduced is
42	less than this value it will return the solution
43
44	Returns
45	-------
46	U: numpy.ndarray - "Sample Loadings" or the unitary matrix
47	having left singular vectors as columns. Of shape (M,n_components)
48
49	s: numpy.ndarray - The singular values,
50	sorted in non-increasing order. Of shape (n_components,n_components).
51
52	V: numpy.ndarray - "Feature Loadings" or Unitary matrix
53	having right singular vectors as rows. Of shape (N,n_components)
54
55	solution: numpy.ndarray - (USV.transpose()) of shape (M,N)
56
57	distance: numpy.ndarray - Distance between each
58	pair of the two collections of inputs. Of shape (M,M)
59
60	Raises
61	------
62	ValueError
63
64	ValueError
65	`ValueError: n_components must be at least 2`.
66
67	ValueError
68	`ValueError: max_iterations must be at least 1`.
69
70	ValueError
71	`ValueError: Data-table contains either np.inf or -np.inf`.
72
73	ValueError
74	`ValueError: The n_components must be less
75	than the minimum shape of the input table`.
76
77	References
78	----------
79	.. [1] Keshavan RH, Oh S, Montanari A. 2009. Matrix completion
80	from a few entries (2009_ IEEE International
81	Symposium on Information Theory
82
83	Examples
84	--------
85	TODO
86
87	"""
88
89	self.n_components = n_components	1✔
90	self.max_iterations = max_iterations	1✔
91	self.tol = tol	1✔
92
93	return	1✔
94
95	def fit(self, X):	1✔
96	"""
97	Fit the model to X_sparse
98	"""
99
100	X_sparse = X.copy().astype(np.float64)	1✔
101	self.X_sparse = X_sparse	1✔
102	self._fit()	1✔
103	return self	1✔
104
105	def _fit(self):	1✔
106
107	# make copy for imputation, check type
108	X_sparse = self.X_sparse	1✔
109	n, m = X_sparse.shape	1✔
110
111	# make sure the data is sparse (otherwise why?)
112	if np.count_nonzero(np.isinf(X_sparse)) != 0:	1✔
113	raise ValueError('Contains either np.inf or -np.inf')	1✔
114
115	# test n-iter
116	if self.max_iterations < 1:	1✔
117	raise ValueError("max_iterations must be at least 1")	1✔
118
119	# check the settings for n_components
120	if isinstance(self.n_components, str) and \	1✔
121	self.n_components.lower() == 'auto':
122	# estimate the rank of the matrix
123	self.n_components = 'auto'	1✔
124	# check hardset values
125	elif isinstance(self.n_components, int):	1!
126	if self.n_components > (min(n, m) - 1):	1✔
127	raise ValueError("n-components must be at most"	1✔
128	" 1 minus the min. shape of the"
129	" input matrix.")
130	if self.n_components < 2:	1✔
131	raise ValueError("n-components must "	1✔
132	"be at least 2")
133	# otherwise rase an error.
134	else:
135	raise ValueError("n-components must be "	×
136	"an interger or 'auto'")
137
138	# return solved matrix
139	self.U, self.s, self.V = OptSpace(n_components=self.n_components,	1✔
140	max_iterations=self.max_iterations,
141	tol=self.tol).solve(X_sparse)
142	# save the solution (of the imputation)
143	self.solution = self.U.dot(self.s).dot(self.V.T)	1✔
144	self.eigenvalues = np.diag(self.s)	1✔
145	self.explained_variance_ratio = list(	1✔
146	self.eigenvalues / self.eigenvalues.sum())
147	self.distance = distance.cdist(self.U, self.U)	1✔
148	self.feature_weights = self.V	1✔
149	self.sample_weights = self.U	1✔
150
151	def fit_transform(self, X):	1✔
152	"""
153	Returns the final SVD of
154
155	U: numpy.ndarray - "Sample Loadings" or the
156	unitary matrix having left singular
157	vectors as columns. Of shape (M,n_components)
158
159	s: numpy.ndarray - The singular values,
160	sorted in non-increasing order. Of shape (n_components,n_components).
161
162	V: numpy.ndarray - "Feature Loadings" or Unitary matrix
163	having right singular vectors as rows. Of shape (N,n_components)
164
165	"""
166	X_sparse = X.copy().astype(np.float64)	1✔
167	self.X_sparse = X_sparse	1✔
168	self._fit()	1✔
169	return self.sample_weights, self.s, self.feature_weights	1✔

biocore / gemelli / 125

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous