Parallel in Variables (Vertexes):
General huge, undirected graph: each vertex is a variable (parallel sampling on a high dimension).
Single Node with multiple cores:
Split into two groups, with non-connected vertexes.
Conditionally independent. [1]
1. Parallel sample in Blue group.
2. Parallel sample in Green group.
(Order can be changed.)
Time:
O(blue) + O(green)
Multiple nodes with multiple cores: Split the dataset
Problems:
How to deal with the edges(borders)?
Is it reasonable for GS? Conditionally Independent? (I think this part need to be proved.)
Parallel on Neural Networks:
Same idea with the Neural Networks.
Parallel design for a requital network training, to improve efficiency.
Need to read more papers and then decide if it is a nice topic. I can find few papers, but not too many.
References:
IreneLi 