Infrared wavelengths hold the key to understanding the evolution of galaxies. Thanks to unprecedented depth and the crucial wavelength range of Herschel a much better census of the hidden AGN activity and dust-obscured star-forming galaxies can now be obtained. I will present the new Herschel IR luminosity function of galaxies and AGN obtained from the cosmological PEP and HERMES surveys up to z~4. Thanks to a detailed SED-decomposition, I will also present the first estime of the black hole accretion history obtained with IR data and compare the results with X-ray based estimates.