If you get into a little more than average detail concerning strings, you'll come across blanks or spaces in your string. The CONDENSE statement can be used to clean up the string. Can more be said about blanks ? Or rather: White-space ?
CONDENSE can remove double (triple, quadruple, ..) spaces and transform them into a single space. There is also a utility class CL_ABAP_STRING_UTILITIES that can help you remove spaces from the end of a string (method DEL_TRAILING_BLANKS).
Blanks are blanks, but they can also be referred to as "White spaces". To be platform independent, the "White space" can be a different character accross the universe of systems/platforms. Check out method GET_SIMPLE_SPACES_FOR_CUR_CP on class CL_ABAP_CHAR_UTILITIES: this will return a string with all possible white-space characters.
Have you tried to replace blanks with something else using the REPLACE statement ? Or the other way around, replace the ; with a space in "I;wish;this;worked!". The REPLACE statement will die with a REPLACE_INFINITE_LOOP dump (replace space with something else) or simply return "Iwishthisworked". The space or blank is a special character which is best avoided with REPLACE. Use TRANSLATE instead: TRANSLATE yourwords USINE ' -'. will change all spaces into a dash.
lv_tokensize = STRLEN( lv_token ). IF lv_tokensize > 0. TRANSLATE lv_token USING ' #'. lv_token = lv_token(lv_tokensize). ENDIF.
The above example also tackles the issue that a char 20 fiels has 20 spaces, even when it's empty.