This paper proposes density estimation as a feasible approach to the wide class of learning problems where traditional function approximation methods fail. These problems generally involve learning the inverse of causal systems, specifically when the inverse is a non-convex mapping. We demonstrate the approach through three case studies: the inverse kinematics of a three-joint planar arm, the acoustics of a four-tube articulatory model, and the localization of multiple objects from sensor data. The learning algorithm presented differs from regression-based algorithms in that no distinction is made between input and output variables; the joint density is estimated via the EM algorithm and can be used to represent any input/output map by forming the conditional density of the output given the input.