I was also puzzled by the statement that soap should not be used for the inner sheath (material: TPE).
I then asked Lovense myself yesterday (in German and English straight away), and within less than 24 hours this morning I got a nice, detailed and useful answer in English! :-)

So: you are allowed to use MILDE soaps, which are usually "without" perfume. I write "without" in inverted commas because soaps are always perfumed, just more or less. (Perfumes are essential oils, and oil of any kind can attack some plastics).

Here is an unchanged (i.e. original, in English) part of the very detailed answer I received from Lovense:
"If you need deeper cleaning, you can clean the sleeve and the main unit with mild soap and warm water. Mild soaps are those with less chemical content like Dove and Cetaphil. These kinds of soaps are usually unscented as well. Do not use harsh soap on the inner sleeve as it may degrade the material."