记一次 .NET 某电商无货源后端服务 死锁分析 (2)

要想追究死锁的原因,只能仔细推敲 线程栈 + 线程栈对象。

0:063> !clrstack OS Thread Id: 0x75e0 (63) Child SP IP Call Site 000000F787774EE8 00007ffb9f230c7a [HelperMethodFrame_1OBJ: 000000f787774ee8] System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef) 000000F787775040 00007ffb1d5e4d5c MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool+ListConnectionHolder.Acquire() ... 000000F78777BD10 00007ffb1e2fc69f System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) [/_/src/System.Private.CoreLib/shared/System/Threading/CancellationTokenSource.cs @ 724] 000000F78777BDA0 00007ffb1e2fc69f System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) [/_/src/System.Private.CoreLib/shared/System/Threading/CancellationTokenSource.cs @ 724] 000000F78777BE30 00007ffb1e2fc69f System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean) [/_/src/System.Private.CoreLib/shared/System/Threading/CancellationTokenSource.cs @ 724] 000000F78777BEC0 00007ffb1e325d55 MongoDB.Driver.Core.Misc.SemaphoreSlimSignalable.Signal() 000000F78777BF10 00007ffb1e35b98e MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool+ListConnectionHolder.Return(PooledConnection) 000000F78777BF70 00007ffb1e35b728 MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.ReleaseConnection(PooledConnection) 000000F78777C0A0 00007ffb1e35b4f9 MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool+AcquiredConnection.Dispose() 000000F78777C0D0 00007ffb1e384164 MongoDB.Driver.Core.Operations.RetryableReadContext.Dispose() 000000F78777C100 00007ffb1e37ee31 MongoDB.Driver.Core.Operations.FindOperation`1+d__120[[System.__Canon, System.Private.CoreLib]].MoveNext() ... 000000F78777E980 00007ffb1dd807bf System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.InvokeContinuation(System.Action`1<System.Object>, System.Object, Boolean, Boolean) [/_/src/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs @ 1019] 000000F78777E9F0 00007ffb1dd80519 System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.OnCompleted(System.Net.Sockets.SocketAsyncEventArgs) [/_/src/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs @ 858] 000000F78777EA50 00007ffb1dd7f751 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*) [/_/src/System.Private.CoreLib/src/System/Threading/Overlapped.cs @ 59] 0:063> !dso OS Thread Id: 0x75e0 (63) RSP/REG Object Name 000000F787774A38 000000f7b82dc750 MongoDB.Bson.BsonBoolean 000000F787774BA0 000000f7b83a9a10 System.RuntimeType 000000F787774CF8 000000f7b853d480 System.Object ... 000000F78777B658 000000f7be3407c8 System.String OperationCanceled 000000F78777B670 000000f7b82c07e0 System.Resources.RuntimeResourceSet 000000F78777B698 000000f7b82bff38 System.Globalization.CultureInfo 000000F78777B730 000000f7a36525c8 System.Char[] 000000F78777B7E0 000000f7be3407c8 System.String OperationCanceled ... 000000F78777BEF8 000000f7b853de10 MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool+ListConnectionHolder 000000F78777BF10 000000f79f79e398 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[MongoDB.Driver.IAsyncCursor`1[[Czkj.AlibabaTools.Models.mongo.CopyRecords, Czkj.AlibabaTools.Models]], MongoDB.Driver.Core],[MongoDB.Driver.Core.Operations.FindOperation`1+<ExecuteAsync>d__120[[Czkj.AlibabaTools.Models.mongo.CopyRecords, Czkj.AlibabaTools.Models]], MongoDB.Driver.Core]] 000000F78777BF40 000000f7b853de48 System.Object ...

由于这代码到处都是 await,async ,所以看这反编译后的线程栈真的头大,经过仔细比对,发现代码流程大概是:

从处理 Mongodb 的异步请求回调开始 (System.Threading.OverlappedData)。

在 MongoDB.Driver.Core.Operations.FindOperation 时不知为啥抛了取消异常 OperationCanceled,然后调用 RetryableReadContext.Dispose()。

记一次 .NET 某电商无货源后端服务 死锁分析

在 ListConnectionHolder.Return() 方法中获取 000000f7b853de48 锁对象。

记一次 .NET 某电商无货源后端服务 死锁分析

在 SignalOrReset() -> SemaphoreSlimSignalable.Signal() 方法中执行一些注册handler逻辑。

记一次 .NET 某电商无货源后端服务 死锁分析

注意:在事件触发中并没有退出 lock 区域。

在几个handler痉挛过程中进入了另外一个线程池的 ListConnectionHolder.Acquire() 方法中,希望能得到该池中的 000000f7b853d480 锁对象。

记一次 .NET 某电商无货源后端服务 死锁分析

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/zgwszf.html