In a typical textbook electrochemical cell (e.g. Copper electrode in CuSO4, zinc electrode in ZnSO4), are the two electrolyte solutions at the same voltage?

Good question.

They are in contact so there should be no voltage between them, but depending on what electrodes you use, you might measure a voltage.

The simple answer is they are at the same voltage as far as most exam questions are concerned.