Basis Errors and Solutions

Symptom

You are running the preparations for an MDMP -> Unicode conversion
with transaction SPUMG (release SAP_BASIS 620 or newer) or with
transaction SPUM4 (SAP_BASIS 46C).
In the so called “Reprocess Scan” you get short dumps containing messages like:
a) SYSTEM_IMODE_TOO_LARGE or SYSTEM_NO_ROLL or other messages showing a shortage of memory
b) EXPORT_TOO_MUCH_DATA
c) TSV_TNEW_OCCURS_NO_ROLL_MEMORY

Other terms

SPUMG, SPUM4, Unicode conversion, Reprocess Scan, SPUMG:$WA, SPUM4:$WA
Reason and Prerequisites

Depending on the data in the system the amount of entries in the
“Reprocess Log” can become very large.
Solution

Background
Transaction SPUMG/SPUM4 searches for all texts on the database which contain special national characters and which have no codepage assigned. These words are inserted into the so called system vocabulary. A language has to be assigned to the words which then defines the code page used for the Unicode conversion during database export.
The Reprocess Scan does a conversion prediction which uses the language assignments in the system vocabulary. Records which cannot be converted correctly are logged in the so called Reprocess Log for further review together with more context which helps to identify the language of a record. It is possible that the Reprocess Log can become very large if the system vocabulary is not maintained completely or if words are missing in the system vocabulary.
In general the size of a Reprocess Log is defined mainly by two factors:
1. the SPUMG/SPUM4 option “Minimum Word Length”
This option defines the minimum word length in bytes at which a word is inserted into the system vocabulary. The default setting is 3, i.e. all words with 1 or 2 bytes are not inserted in the system vocabulary by default.
2. the language combination in the system and specific text data and text patterns stored by applications in the database
Memory problems can occur if the “Minimum Word Length” is too high for the specific data situation.
Example:
Some languages have words which consist of 1 byte only, e.g. the french word “?. It is possible that the word is used in many records but is not inserted into the system vocabulary because the “Minimum Word Length” option is set to 3. All records containing the “? will be logged in the Reprocess Log for the affected tables.

In general it is desirable to have small Reprocess Log to decrease the downtime as much as possible. Please note that
all records in the Reprocess Logs have to be assigned a languagethe bigger the Reprocess Log, the bigger the R3load Log(XML files)the bigger the Reprocess Log, the longer the automatic repair in SUMG after conversion

Solution
First you should try to resolve the problem by increasing the memory. As a rule of thumb you can reckon with about 1500 bytes allocated for a single record in a “Reprocess Log”, .i.e you need about 1 GB memory for holding 650000 records in memory.
If no additional memory is available or case b) is occurring, the only option is to decrease the size of the “Reprocess Log” by decreasing the “Minimum word length” in the SPUMG/SPUM4 options. The default value of 3 might be too large for your specific data situation. Please set the length to 2 or even to 1 in case value 2 still results in too many “Reprocess Log” entries.
The following steps can be iterated until the size of the Reprocess Log is acceptable. Most often only one table is affected, i.e. it is not necessary to repeat the Reprocess scan for all tables but only for the affected ones.

1. Set the “Minimum Word Length” to 1
In general it is not possible to change the value for the “Minimum Word Length” if the scans have been started already. You can use the attached reports Z_UMG_MWL1(basis >4.6C) and ZUM4_MWL1(basis release 4.6C) to set the “Minimum Word Length” to 1.

2. Execute the Scan “Tables without language information”
After repeating the scan you will see many 1 and 2 byte words in the system vocabulary.

3. Try to maintain the vocabulary as good as possible
Sometimes it is very difficult to identify the language of a word having one byte. If you have created vocabulary hints you should execute the hints again because hints depend also on the “Minimum word length”. Therefore one byte words are assigned also a language if covered by the hints.
Also it might be helpful to check the table contents directly in SE16. Search specific for the words in the table field in which the word was found in (change ALV grid layout and add the column “Table Position” to the ALV grid which displays the vocabulary).

4. Start the Reprocess Scana) If the Reprocess Scan finishes without problems you can use report UM4_ANALYZE_REPLOG (4.6C) or UMG_ANALYZE_REPLOG (> 4.6C) to improve further the quality of the system vocabulary. If the report is not included in your current version of SPUM4/SPUMG you have to create the report manually. Use the attached files
ZUM4_ANALYZE_REPLOG(basis release 4.6C)
ZUMG_ANALYZE_REPLOG(basis release > 4.6C)
Documentation is attached to the note (ANALYZE_REPLOG.TXT and ANALYZE_REPLOG.ZIP).b)Please note: the follwing step can only be done in release 4.6C if the correction instructions of this note are applied: (the corrections are included in transport request BIOK017549, see note 867193). For releases > 4.6C the report UMG_STOP_REPROCESS_LOG will be provided in near future.
If the Reprocess Scan fails due to lack of memory, you should limit the maximum size of the Reprocess Log. This can be done with report UM4_STOP_REPROCESS_LOG. If the report is not included in your current version of SPUM4 you have to create the report manually. Use the attached files
ZUM4_STOP_REPROCESS_LOG (basis release 4.6C)
Documentation is attached to the note (STOP_REPROCESS_LOG.TXT).

a) Once you set a limit with report UM4_STOP_REPROCESS_LOG you have to execute the Reprocess Scan for the affected tables again.
b) If the Reprocess scan is stopped at the limit you have to analyze the log with report UM4_ANALYZE_REPLOG and improve the quality of the vocabulary.
c) Afterwards reset the table in the Reprocess scan (use “Reset Scan”, do not use “Repeat Scan” or “Resume Scan”) and execute it again until the Reprocess scan finishes without being stopped by the limit.
The steps a)-c) can be iterated until the size of the Reprocess Log is acceptable.

General recommendation:
If you encounter the problems decribed above during a test, it is strongly recommended that you set the SPUM4/SPUMG option “Minimum Wordlength” to 1 for all further conversion.
If you cannot repeat the scan “Tables without Language Information” due to lack of time, it is possible to add single words to the system vocabulary directly. You have to specify the table name, the word and the language. Use the reports UMG_ADD_VOCABULARY or UM4_ADD_VOCABULARY (4.6C). If the report is not included in your current version of SPUM4/SPUMG you have to create the report manually. Use the attached files ZUM4_ADD_VOC.txt (4.6C) or ZUMG_ADD_VOC.txt.
Note: After adding new short words to the vocabulary and before starting over with the reprocess, you have to set the “Minimum Word Length” to 1 (Z_UMG_MWL1 or ZUM4_MWL1, see above). Otherwise the scan will not take the new words into account.