Bayesian decision theory on three layered neural networks

We treat the Bayesian decision problem, mainly the two-category case. A three layered neural network, having a logistic output unit and a small number of hidden layer units, can approximate the a posteriori probability in L²-norm, without knowing the type of the probability distribution before learning, if the log ratio of the a posteriori probabilities is a polynomial of low degree as in the case of most familiar probability distributions. This is because the log ratio itself can be well approximated by a linear sum of outputs of the hidden layer units in L²-norm.