You need to pay attention to the following points:
a) If one wife fulfills your sexual needs, it is better to remain faithful to that one person, as you put it. However, in the case of necessity, what you said will not create a problem in re-marrying.
b) Having sexual relations with others must be Islamically legitimate (through temporary or permanent marriage), and one should be careful not to fall into the trap of sin.
One thing you must keep in mind is that you should strive to be committed to your promises and vows in your marital life so that the family doesn’t fall apart. You should try and do everything necessary and set the grounds to take the utmost pleasure in your own wife, because if you fail to do so, you may end up with the same problem in your second marriage as well.
For further information, you can take a look at the following links of this same website:
2- The Ruling on Unchastity (i.e. Adultery), Question 2569 (website: 2723).
3- The Reason for Marriage in Islam, Question 1300 (website: 1283).
4- Sexual Relationships before Marriage, Question 2373 (website: 2516).
5- The Conditions for Re-Marriage and Polygamy, Question 3103 (website: 3368).