mxnet.kvstore.BytePS¶
-
class
BytePS
[source]¶ Bases:
mxnet.kvstore.base.KVStoreBase
BytePS backend for MXNet KVStore interface.
Methods
__init__
()Initializes a new KVStore.
broadcast
(key, value, out[, priority])Broadcast the value NDArray at rank 0 to all ranks’ out.
is_capable
(capability)Queries if the KVStore type supports certain capability, such as optimizer algorithm, gradient compression, sparsity, etc.
load_optimizer_states
(fname)Not Implement yet.
pushpull
(key, value[, out, priority])Performs push and pull a single value from the store.
register
(klass)Registers a new KVStore.
save_optimizer_states
(fname[, dump_optimizer])Not Implement yet.
set_optimizer
(optimizer)Not Implement yet.
Attributes
OPTIMIZER
kv_registry
Returns the local rank of this worker on the node.
Returns the number of worker nodes.
Returns the rank of this worker node.
Returns the type of this kvstore.
-
broadcast
(key, value, out, priority=0)[source]¶ Broadcast the value NDArray at rank 0 to all ranks’ out. If out is None, the result is stored in value.
- Parameters
Examples
>>> # broadcast a single key-value pair >>> shape = (2,3) >>> kv = mx.kv.create('byteps') >>> a = mx.nd.zeros(shape) >>> kv.broadcast('3', mx.nd.ones(shape)*2, out=a) >>> print a.asnumpy() [[ 2. 2. 2.] [ 2. 2. 2.]]
-
static
is_capable
(capability)[source]¶ Queries if the KVStore type supports certain capability, such as optimizer algorithm, gradient compression, sparsity, etc. As byteps server does not store weight, this function will return false for any capabilities.
- Parameters
capability (str) – The capability to query
- Returns
result – Whether the capability is supported or not.
- Return type
bool
-
load_optimizer_states
(fname)[source]¶ Not Implement yet.
- Parameters
fname (str) – Path to input states file.
-
property
local_rank
¶ Returns the local rank of this worker on the node.
- Returns
rank – The local rank of this node, which is in range [0, num_workers_on_current_node())
- Return type
int
-
property
num_workers
¶ Returns the number of worker nodes.
- Returns
size – The number of worker nodes.
- Return type
int
-
pushpull
(key, value, out=None, priority=0)[source]¶ Performs push and pull a single value from the store. This function is coalesced form of push and pull operations. value is pushed to the kvstore server for the specified keys and the aggregated values are pulled from the server to out. If out is not specified the pulled values are written to value.
- Parameters
key (str, or int) – The key.
value (NDArray, or list of NDArray) – Values corresponding to the key.
out (NDArray, or list of NDArray) – Values corresponding to the key.
priority (int, optional) – The priority of the operation. Higher priority operations are likely to be executed before other actions.
Examples
>>> # pushpull a single key-value pair >>> kv.pushpull('3', mx.nd.ones(shape)*8, out=a) >>> print a.asnumpy() [[ 8. 8. 8.] [ 8. 8. 8.]]
-
property
rank
¶ Returns the rank of this worker node.
- Returns
rank – The rank of this node, which is in range [0, num_workers())
- Return type
int
-
save_optimizer_states
(fname, dump_optimizer=False)[source]¶ Not Implement yet.
- Parameters
fname (str) – Path to the output states file.
dump_optimizer (bool, default False) – Whether to also save the optimizer itself. This would also save optimizer information such as learning rate and weight decay schedules.
-
set_optimizer
(optimizer)[source]¶ Not Implement yet.
- Parameters
optimizer (KVStoreBase) – The new optimizer for the store
-
property
type
¶ Returns the type of this kvstore.
- Returns
type – the string type
- Return type
str
-