The ability to hide private data and confidential patterns from potential adversaries while still maintaining data mining value is an important aspect in privacy preserving data mining. In this paper, we study a nonnegative matrix factorization technique, where we show how to define objective functions and derive corresponding multiplicative update functions. We then use that knowledge to propose a data value perturbation scheme that hides data values but still keeps the data pattern to a large degree. Based on the proposed data value perturbation scheme, we develop a dual data hiding scheme which not only hides data but also hides individual sample's class membership. The essential idea is to use an indicator matrix as a guide for the update process. The performance of the proposed schemes are examined on benchmark datasets for both utility value and data perturbation degree. The empirical results show that the data values are well perturbed and our schemes are capable of hiding a data sample's class membership without side effects. At the end, we draw some interesting conclusions and layout potential future work.