Why does ISQL attempt to open files containing 0x96 as CP437? A bug?

[Former Member]

Character 0x96 is a dash character outside of the ASCII range. I have this in a script in a line of text that was copied from a Word document.

When I open a file containing this character in ISQL, ISQL attempts to open the file as CP437 instead of the expected windows-1252 code page. This causes a minor problem for my company when staff run the script manually and forget to change the script's code page from CP437 to windows-1252. The CP437 interpretation of the raw data cascades through my systems and causes a minor failure.

Is there a reason why the presence of dash from Word should cause ISQL to attempt to open a file as CP437, or is this a bug?

To reproduce create a text file containing only 0x96. Open in ISQL. Menu gives option to select code page. CP437 is selected by default.

Windows 7. ISQL Version 12.0.1 build 3554.

[VolkerBarth]

Do you use the (Java-based) DBISQL or the older (C-based) dbisqlc utility?

[Former Member · Accepted solution]

The short answer is that DBISQL is behaving as expected.

The problem is that the byte 0x96 is valid in both Cp1252 and Cp437. When you open a file in DBISQL, it scans the file, and tries to detect the file encoding based on a statistical analysis of the bytes it contains. In Cp437, 0x96 is a character; in Cp1252, it's punctuation. The analysis weights letters more heavily than punctuation, which is why DBISQL defaults to selecting Cp437 rather than Cp1252.

To work around this problem, you could introduce a new SQL script file which contains a READ statement with an ENCODING clause to set the encoding to Cp1252. The READ statement would simply read your existing script file.

[Breck_Carter · Accepted solution]

FWIW all my batch ISQL commands include an ENCODING clause as follows...

"%SQLANY11%\\bin32\\dbisql.com" -c "ENG=ddd11;DBN=ddd11;UID=dba;PWD=sql" READ ENCODING Cp1252 "test.sql"

[Former Member · Accepted solution]

Thanks. That explains it. Our solution is to edit the source script and replace the 0x96 with an ASCII dash. Some text strings in the script were copied and pasted from MS Word documents.