There are two approaches here, the formal one and the brain-compatible one.
1) Formally: Realize that there are hidden coordinate dependencies. You are probably looking to construct a function p(y). Since y is the coordinate at the lower side, you have p(y) at the lower edge and p(y+dy) at the upper. This is (possibly) slightly different from p(y) (because of the displacement dy). If you call the difference dp, then p(y+dy) = p(y) + dp(y) = p + dp (note that dp can and will be negative).
2) Brain compatible: Put the equation first and then define the variables: The forces on top and bottom should cancel out. There force up is the pressure force p*A from below. The force from up is the pressure force p2*A from above the small fluid element plus the weight dw of the small fluid element. Since we are talking about infinitesimal coordinates, and since p(y) should be a function, it makes sense to say that p2 = p+dp, which you can then integrate over.