Does anyone know if there are any downsides to using UV spectrophotometry to determine the concentration of a purified protein? The only one I can really think of is that it would also measure any protein impurities the sample might contain in addition to the desired protein and add them into the final concentration it gives. Does anyone know if that is indeed a problem, and if there are any others?

There are several issues and depends on what kind of results you need (i.e. the actual analytical application) as well as the nature of the sample. As you already mentioned, contamination (not only proteins, but everything else that absorbs at near UV) will confound results. The less purified your sample, the larger the error. Especially if separation is not ideal beforehand, this is a big issue. The next is sensitivity. Depending on the aromatic components of your protein, it will absorb more or less. In some cases, where you only got minute amounts, it will be below detection limit. As a general rule of thumb, protein quantification via UV/Vis is quite ok, if you are within the dynamic range of the assay, and have a very pure sample (including buffer/solvents). For other things, colorimetric assays often have better sensitivity and/or robustness, or even MS (which requires a bit more work).