What's the difference between calling a simple C function in the servers' context and in the external environments?

[VolkerBarth]

I'm testing the C external environments in SQL Anywhere 11.0.1.2452.

When I have an external C function (with the "new external API") that runs fine in the context of the server, I expect that function behaves (nearly) identical when run in the 32-bit ESQL or ODBC environments. (There are no database calls at all in the external DLL so that aspect shouldn't matter.)

As I understand there are four main differences between calls in the engine's context ("internal") and in the external C environments:

• Internally called DLLs must match w.r.t. bit-ness (32 vs. 64), externally called DLLs can vary.
• Each internally loaded DLL is loaded once per database engine, whereas each connection gets its own external C environment.
• As a consequence, internally loaded DLLs can crash the database engine, corrupt memory and the like, an externally loaded DLL can do such harm only to its isolated process.
• Internal calls are more efficient because there's no interprocess communication and no starting/stopping overhead for the external process.

---

In my test case, only the 2nd point should make a (minor) difference, but the external function behaves differently when run internally vs. externally.

Question: Are there more differences to take care of?

[VolkerBarth]

At a sidenote: How do I debug my DLL when using DBEXTERNC11.EXE? Debugging a DLL inside DBENG11.EXE is easy as I can start that program through the debugger. In contrast, DBEXTERNC11.EXE is started by the database engine, and when I'm using VC++'s "Attach to process", it does not show my sources... - Otherwise, I might have been able to answer my question myself:(

[MarkCulp]

Can you clarify what type of differences you are seeing?

[MarkCulp]

Also, in order to debug your external DLL your DLL must be loaded... and it is not loaded until it is first used.

[VolkerBarth]

@Mark: The difference is as following: The function is basically a wrapper around a Win32 call to wait for an named mutex via WaitForSingeObject(). It returns 1 when the mutex can be owned within the specified time, and 0 otherwise. The function works in different ASA/SA versions when running inside the server context. When declared to run externally, it fails no matter whether the mutex is blocked or not. (And failing to debug that situation so far I really can't tell why.)

[VolkerBarth]

@Mark: To debug, I load the DLL project in MS VC++, start the database, start dbisqlc, call the function (which starts the dbextern11.exe and loads the DLL) and attach the debugger to the dbextern11.exe process. But instead of using the open DLL project, the IDE starts a new project with only the dbextern11.exe as debugee, and I can't set breakpoints in my code or see the debug output. - That's a VC++ problem (aka my lack of experience), obviously, but I'm somewhat stuck here.

[Former Member]

It should be noted that another important consequence of point 2 is that in the internally loaded case, statics within the dll are not secure and are instead shared across all connections whereas in the external C environment case, the statics within the dll are sandboxed.

[VolkerBarth · Accepted solution]

EDITED: According to Karim's answer, the following difference is a bug and is fixed in coming versions. Obviously, it's not a difference by design - in contrast to to ones listed in my question.

---

Well, at the moment I notice a difference when using optional parameters:

Say, I have an external function declared as

create function FKTN_LockMutex(
   strMutexName varchar(255), nMillisecondsTimeout unsigned int default null)
   returns int
   external name 'LockMutex@MyDll';

Inside the C function, I use the following code to get the value of the 2nd parameter (quite unchanged from the ASA 8 "ExternalProcedures" sample):

bReturn = api->get_value(arg_handle, 2, &arg2);

When I call that function and set the 2nd paramter to its default value as in

select FKTN_LockMutex('MtxTest');

or

select FKTN_LockMutex('MtxTest', null);

then the internal call returns a non-zero value as success and shows the arg2.data value correctly as null, whereas the C_ODBC32 environment returns 0 meaning failure.

When the parameter has a not-null value, both environments behave correctly.

Tested with SA 11.0.1.2427, I guess SA 11.0.1.2452 and 12.0.0.2429 behave identically.

[Former Member]

Let's assume you are using the C_ESQL32 environment and your dll is mystuff.dll. Try the following:

1) start external environment c_esql32 // this should get the executable launched 2) attach your debugger to the dbexternc11.exe process 3) create a procedure that loads your dll but maps to a function that does not exist

CREATE PROCEDURE junk() EXTERNAL NAME 'notexist@mystuff.dll' LANGUAGE C_ESQL32

4) call junk() // this will return with an error saying that notexist does not exist, BUT, your dll should now be loaded

5) go back to the debugger and set the appropriate break points

6) make the call that gets you to your breakpoint

See if that does the trick as far as getting the debugger going.

[VolkerBarth]

I just tried that (with C_ODBC32), and I still have the problem that the command "Debug/Attach to process" opens a new project with only the dbexternc.exe, and my original DLL project is closed. As such I don't get any debug output nor am I able to set breakpoints and the like. I have assured per ProcExp that dbexternc.exe has loaded the correct DLL but the debugger doesn't seem to notice that I want to use the attached process to debug my DLL:)

[VolkerBarth]

Well, I had tested with VC++ 6 so far (as this is the IDE on the according test system) and have now converted the project to VS 2005. Now I can attach my DLL project to a running process and look what's different. - Is that a known VC++ 6 limitation?

[Former Member]

Volker,

Thank you for narrowing this problem down and for reporting the bug. The external C environment(s) have been fixed to ensure that a status of 1 is returned when get_value() is called on a NULL argument. The fix will be available in 11.0.1 build 2460 and up. Note that for 12.0.0, the fix will not be available until the first 12.0.0 EBF