dynpy.markov - Markov chain module

Module implementing Markov chains.

class dynpy.markov.MarkovChain(transition_matrix, base_sys=None, discrete_time=True)

Bases: LinearDynamicalSystem

This class implements Markov chains.

There is some potential for confusion regarding the term ‘Markov chain’. It may be used to indicate a stochastic dynamical system, which transitions from state to state with different probabilities. Alternatively, and in the sense used in dynpy, a Markov chain refers to a deterministic, multivariate dynamical that transforms probability distributions over some underlying to distributions into other probability distributions.

Parameters:

transition_matrixnumpy array or scipy.sparse matrix

Matrix defining the evolution of the dynamical system, i.e. the \(\mathbf{A}\) in \(\mathbf{x_{t+1}} = \mathbf{x_{t}}\mathbf{A}\) (in the discrete-time case) or \(\dot{\mathbf{x}} = \mathbf{x}\mathbf{A}\) (in the continuous-time case)

base_sysDiscreteStateDynamicalSystem, optional

Specifies an underlying dynamical system, possibly vector-valued, to whose states the individual outcomes of the Markov chain refer.

discrete_timebool, optional

Whether updating should be done using discrete (default) or continuous time dynamics.

get_equilibrium_distribution()

Get equilibrium state (i.e. the stable, equilibrium distribution) for this dynamical system. Uses eigen-decomposition.

Returns:

numpy array or scipy.sparse matrix

Equilibrium distribution

property num_states

get_uniform_distribution()

Return uniform starting distribution over all system states.

classmethod from_deterministic_system(base_sys, issparse=True)

Alternative constructor creates a a Markov Chain from the transitions of an underlying deterministic system. It maintains properties of the underlying system, such as the sparsity of the state transition matrix, and whether the system is discrete or continuous-time. The underlying system must be an instance of dynpy.dynsys.DeterministicDynamicalSystem and dynpy.dynsys.DiscreteStateDynamicalSystem.

For example, for a Boolean network:

>>> import dynpy
>>> yeast = dynpy.sample_nets.budding_yeast_bn
>>> bn = dynpy.bn.BooleanNetwork(rules=yeast)
>>> bnEnsemble = dynpy.markov.MarkovChain.from_deterministic_system(bn, issparse=True)
>>> init = bnEnsemble.get_uniform_distribution()
>>> trajectory = bnEnsemble.get_trajectory(init, max_time=80)

If we wish to project the state of the Markov chain back onto the activations of the variables in the underlying system, we can use the ndx2state_mx matrix. For example:

>>> import dynpy
>>> import numpy as np
>>> yeast = dynpy.sample_nets.budding_yeast_bn
>>> bn = dynpy.bn.BooleanNetwork(rules=yeast)
>>> bn_ensemble = dynpy.markov.MarkovChain.from_deterministic_system(bn, issparse=True)
>>> init = bn_ensemble.get_uniform_distribution()
>>> final_state = bn_ensemble.iterate(init, max_time=80)
>>> print(np.ravel(final_state.dot(bn.get_ndx2state_mx()))) 
[0.          0.05664062  0.07373047  0.07373047  0.92236328  0.          0. 
 0.          0.91503906  0.          0.        ]

Parameters:

base_sysobject

Dynamical system over whose states the Markov chain will be defined

issparsebool, optional

Whether transition matrix should be in sparse or dense matrix format

project(keep_vars, initial_dist=None)

Create a Markov chain by projecting an existing Markov chain over a multivariate dynamical system onto a subset of those variables.

For example:

>>> import dynpy
>>> r = [
...     ['x1', ['x1','x2'], lambda x1,x2: (x1 and x2) ],
...     ['x2', ['x1','x2'], lambda x1,x2: (x1 or  x2) ],
... ]
>>> bn = dynpy.bn.BooleanNetwork(rules=r, mode='FUNCS')
>>> bnensemble = dynpy.markov.MarkovChain.from_deterministic_system(bn, issparse=False)
>>> proj = dynpy.markov.MarkovChain.project(bnensemble, [0])
>>> print(proj.transition_matrix) 
[[1.   0. ]
 [0.5  0.5]]

Parameters:

keep_varslist

List of variables to keep

initial_distoptional

Marginalize using this distribution for starting conditions

discrete_time = True

Whether the dynamical system obeys discrete- or continuous-time dynamics

get_trajectory(start_state, max_time, num_points=None, logscale=False)

This method get a trajectory of a dynamical system starting from a particular starting state.

Parameters:

start_stateobject

Which state to start from

max_timefloat

Until which point to run the dynamical system (number of iterations for discrete-time systems or time limit for continuous-time systems)

num_pointsint, optional

How many timepoints to sample the trajectory at. This determines how big each ‘step size’ is. By default, equal to int(max_time)

logscalebool, optional

Whether to sample the timepoints on a logscale or not (default)

Returns:

trajectory: numpy array

Array of states corresponding to trajectory

iterate(start_state, max_time)

This method runs the dynamical system for max_time starting from start_state and returns the result. In fact, this method is set at run-time by the constructor to either _iterateDiscrete or _iterateContinuous depending on whether the dynamical system object is initialized with discrete_time=True or discrete_time=False. Thus, sub-classes should override _iterateDiscrete and _iterateContinuous instead of this method. See also dynpy.dynsys.DynamicalSystem.iterateOneStep()

Parameters:

start_statenumpy array or scipy.sparse matrix

Which state to start from

max_timefloat

Until which point to run the dynamical system (number of iterations for discrete-time systems or time limit for continuous-time systems)

Returns:

numpy array or scipy.sparse matrix

End state

iterate_1step(start_state)

This method runs a discrete-time dynamical system for 1 timestep. At run-time, the construct either repoints this method to _iterateOneStep for discrete-time systems, or removes it for continuous time systems.

Parameters:

start_statenumpy array or scipy.sparse matrix

Which state to start from

Returns:

numpy array or scipy.sparse matrix

Iterated state

num_vars = None

The number of variables in the dynamical system

transition_matrix = None

Transition matrix for linear system.

property var_name_ndxs

A mapping from variables names to their indexes

var_names

The names of the variables in the dynamical system

class dynpy.markov.MarkovChainSampler(markov_chain)

Bases: StochasticDynamicalSystem

This class implements a stochastic dynamical system whose trajectory represents a sample from a provided Markov chain.

Parameters:

selfdynpy.markov.MarkovChain

Markov chain from which to sample from. For now, only discrete time is supported.

discrete_time = True

Whether the dynamical system obeys discrete- or continuous-time dynamics

get_trajectory(start_state, max_time, num_points=None, logscale=False)

This method get a trajectory of a dynamical system starting from a particular starting state.

Parameters:

start_stateobject

Which state to start from

max_timefloat

Until which point to run the dynamical system (number of iterations for discrete-time systems or time limit for continuous-time systems)

num_pointsint, optional

How many timepoints to sample the trajectory at. This determines how big each ‘step size’ is. By default, equal to int(max_time)

logscalebool, optional

Whether to sample the timepoints on a logscale or not (default)

Returns:

trajectory: numpy array

Array of states corresponding to trajectory

iterate(start_state, max_time)

This method runs the dynamical system for max_time starting from start_state and returns the result. In fact, this method is set at run-time by the constructor to either _iterateDiscrete or _iterateContinuous depending on whether the dynamical system object is initialized with discrete_time=True or discrete_time=False. Thus, sub-classes should override _iterateDiscrete and _iterateContinuous instead of this method. See also dynpy.dynsys.DynamicalSystem.iterateOneStep()

Parameters:

start_statenumpy array or scipy.sparse matrix

Which state to start from

max_timefloat

Until which point to run the dynamical system (number of iterations for discrete-time systems or time limit for continuous-time systems)

Returns:

numpy array or scipy.sparse matrix

End state

iterate_1step(start_state)

This method runs a discrete-time dynamical system for 1 timestep. At run-time, the construct either repoints this method to _iterateOneStep for discrete-time systems, or removes it for continuous time systems.

Parameters:

start_statenumpy array or scipy.sparse matrix

Which state to start from

Returns:

numpy array or scipy.sparse matrix

Iterated state