I’d guess rough or unpolished silver just looks grey-white. An unpolished metal surface won't give a mirror image.
It’s the flatness of the surface that lets the reflection be specular and show an image, as opposed to diffusely scattering the light back. It’s a chemical quirk of silver that enables it to reflect almost all the light energy, and to reflect all visible wavelengths about equally so the reflected light is about the same color as the incident light. Why does light reflect off a silvered surface but scatter off a painted wall? Is the explanation geometrical (depending on the flatness of the mirror surface relative to the wall) or is it chemical (depending on a quirk of silver)?