Keep an eye on your Native API prototype for interop calls

Few weeks back, we have observed "an unexpected process termination" crash in a WPF app. From the crash dump, provides below callstack and unfortunately it was a a memory corruption

00 006fea30 77172e5c     0x770f7000

01 006fea30 56d0df34     ntdll!NtTerminateProcess+0xc

02 006fea30 56d0b9bc     verifier!VerifierStopMessage+0x344

03 006fea9c 56d0bdda     verifier!AVrfpDphReportCorruptedBlock+0x2fc

04 006feaf8 56d0c2d2     verifier!AVrfpDphCheckNormalHeapBlock+0x11a

05 006feb18 56d0ab23     verifier!AVrfpDphNormalHeapFree+0x22

06 006feb3c 771dfa16     verifier!AVrfDebugPageHeapFree+0xe3

07 006feba4 77143d76     ntdll!RtlDebugFreeHeap+0x3e

08 006fed00 77187add     ntdll!RtlpFreeHeap+0xd6

09 006fed5c 77143c46     ntdll!RtlpFreeHeapInternal+0x783

0a 006fed78 754e3320     ntdll!RtlFreeHeap+0x46

0b (Inline) --------     combase!CRetailMalloc_Free+0x16 [onecore\com\combase\class\memapi.cxx @ 656] 

0c 006fed90 723cf422     combase!CoTaskMemFree+0x30 [onecore\com\combase\class\memapi.cxx @ 445] 

0d 006fedc4 08ef0bc0     mscorlib_ni!System.StubHelpers.CSTRMarshaler.ClearNative+0x2e [f:\dd\ndp\clr\src\BCL\system\stubhelpers.cs @ 125] 

0e 006fee6c 08ef0937     0x8ef0bc0

0f 006fee84 08ef08c3     CppAndCs!CppAndCs.Program.GetNativeBuffer+0x4f

10 006fee98 73df0556     CppAndCs!CppAndCs.Program.Main+0x23

11 006feea4 73df373a     clr!CallDescrWorkerInternal+0x34

12 006feef8 73df9adb     clr!CallDescrWorkerWithHandler+0x6b

13 006fef6c 73f6ff6b     clr!MethodDescCallSite::CallTargetWorker+0x16a

14 006ff090 73f7064a     clr!RunMain+0x1b3

15 006ff2fc 73f70577     clr!Assembly::ExecuteMainMethod+0xf7

16 006ff7e0 73f706f8     clr!SystemDomain::ExecuteMainMethod+0x5ef

17 006ff838 73f7081e     clr!ExecuteEXE+0x4c

18 006ff878 73f6c225     clr!_CorExeMainInternal+0xdc

19 006ff8b4 74dffa84     clr!_CorExeMain+0x4d

1a 006ff8ec 74f0e81e     mscoreei!_CorExeMain+0xd6

1b 006ff8fc 74f14338     MSCOREE!ShellShim__CorExeMain+0x9e

1c 006ff914 76cefcc9     MSCOREE!_CorExeMain_Exported+0x8

1d 006ff914 77167c5e     KERNEL32!BaseThreadInitThunk+0x19

1e 006ff970 77167c2e     ntdll!__RtlUserThreadStart+0x2f

1f 006ff980 00000000     ntdll!_RtlUserThreadStart+0x1b

From the callstack, its clear that, it happens when CLR marshaller tries to free the native memory which is temporarily allocated. The methods are defined as shown below.

C#

[DllImport("CppAndCs.dll", EntryPoint = "GetNativeBuffer",

           CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]

          public static extern string GetNativeBuffer(StringBuilder buffer, IntPtr buuferSize);

C++

void GetNativeBuffer (char *buffer, std::size_t buufer_size);

You may have already figured out the problem. Yes, the return value. CLR’s Interop Marshaler is trying to release memory allocated for return value because C# declaration return type is string. In this case, native method does not have any return value so, CLR try to release unallocated memory and Boom!.  

Fix

===

[DllImport("CppAndCs.dll", EntryPoint = "GetNativeBuffer",

           CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]

          public static extern void GetNativeBuffer(StringBuilder buffer, IntPtr buuferSize);

The correct way to return a value from native is to allocate it in native method implementation which will be freed by CLR’s Interop Marshaler later to avoid memory leak

References

========

https://learn.microsoft.com/en-us/dotnet/standard/native-interop/best-practices

https://limbioliong.wordpress.com/2011/06/16/returning-strings-from-a-c-api/