Non-thermal radiation from Pulsar Wind Nebulae and from jets in Active Galactic Nuclei and Gamma- Ray Bursts is usually modeled as synchrotron or inverse Compton emission from a power-law population of particles, presumably accelerated in relativistic collisionless shocks. We explore, by means of particle-in-cell numerical simulations, the conditions in the upstream flow that allow efficient particle acceleration. We find that, if the upstream region is initially unmagnetized, the Weibel instability generates sub-equipartition fields which isotropize and thermalize the incoming particles. A fraction of them (~1% by number, ~10% by energy) is accelerated by the Fermi process to supra-thermal energies and populates a power-law tail. We compute from first principles the radiative signature expected from such shocks and show that it is consistent with synchrotron radiation in the Weibel-generated fields. We discuss the viability of deviations from synchrotron radiation (such as "jitter" radiation) in view of these results. If the upstream flow is seeded with a background laminar field, efficient acceleration occurs only for subluminal magnetic configurations, where relativistic particles following the magnetic field can escape ahead of the shock. However, most of magnetized astrophysical shocks should be superluminal, and in this case we do not observe significant particle acceleration. This may place challenging constraints on the standard models of astrophysical non- thermal emission.