Why do Markov Blankets include children (and parents of these children) of a target node? Why is this important? I understand why parents are important as they directly affect the target. But what does adding children bring? What is the practical application?
The Markov Blanket of a (Target) node consists of all nodes that make this Target conditionally independent of all the other nodes in the model:
- The Parents (red nodes in the example below) are used for cutting the information coming from their ascendants (pink nodes),
- The Children (yellow nodes) are used for cutting the information coming from their descendants (light green nodes),
- The Spouses (or co-parents, dark green) are used for cutting the information coming from the ascendants of the Children (blue nodes). The Target node is marginally independent of the Spouses, but becomes conditionally dependent, i.e. when some evidence is available on the Children.
The Markov Blanket learning algorithm is a supervised algorithm that is used to find a Bayesian Network that characterizes the Target node. Regression, Decision Trees, Support Vector Machines, Neural Networks are alternative supervised approaches, but they are all Discriminative models whereas the Markov Blanket Learning algorithm returns a Generative model (this explains the redundancy of some features, but also allows to have more robust models, specifically when some values are missing).
When learned from data, without any expert knowledge or temporal information, the direction of the arcs cannot be interpreted as causal. The Parents are just nodes that bring more information when used in conjunction with their co-parents than alone (as the children). Also note that the Children that are interconnected (and then form a triangle with the Target) may also be Parents. These triangles can indeed be a way to get a more compact representation than adding these pseudo-Children directly as Parents (the size of the Conditional Probability Tables grows exponentially with the number of Parents).
The Markov Blanket learning algorithm can also be used as a powerful feature selection algorithm. Optionally, you can run the Markov Blanket algorithm ahead of the Naïve, Augmented Naïve and Tree Augmented Naïve algorithms.