添加链接
link管理
链接快照平台
  • 输入网页链接,自动生成快照
  • 标签化管理网页链接

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More than five years after I first requested it, the Google Play gods have finally answered my prayers and now display assert messages along with stack traces:

This means that we can now pass along information about which game was running when we hit an assert, and similar things!

Unfortunately they also made the stack trace UI incredibly slow. Oh well.

There are a whole bunch of OpenGL initialization crashes as usual, now collected better thanks to some asserts I added like that one, but I still think should probably avoid crashing, show a message and bail, or even automatically restart the activity. The few times I've been able to randomly repro these (almost entirely on very old devices), restarting the app helped.

Anyway, if they fix their server issues, I'll add some more mysterious ones here. I did see one in UninstallExceptionHandler but now I can't get it to display it again.

Hm, seems the UninstallExceptionHandler one is just some somewhat worrying memory corruption:

 #00  pc 0x000000000004e178  /apex/com.android.runtime/lib64/bionic/libc.so (abort+168)
  #01  pc 0x0000000000042818  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::die()+8)
  #02  pc 0x0000000000042e50  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::ScopedErrorReport::~ScopedErrorReport()+32)
  #03  pc 0x0000000000042f18  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::reportHeaderCorruption(void*)+60)
  #04  pc 0x00000000000444a0  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::Allocator<scudo::AndroidConfig, &(scudo_malloc_postinit)>::deallocate(void*, scudo::Chunk::Origin, unsigned long, unsigned long)+332)
  #05  pc 0x00000000007646e8  /data/app/~~_MCmDAcVR2sC4CsnTSQpyg==/org.ppsspp.ppsspp-xYIsGg1ImySc_IODjV9WhQ==/split_config.arm64_v8a.apk!libppsspp_jni.so (UninstallExceptionHandler()+116)
  #06  pc 0x00000000005d968c  /data/app/~~_MCmDAcVR2sC4CsnTSQpyg==/org.ppsspp.ppsspp-xYIsGg1ImySc_IODjV9WhQ==/split_config.arm64_v8a.apk!libppsspp_jni.so (CPU_Shutdown()+16)

This surely must be some kind of shutdown bug? Unless there's some curRenderStep_ confusion (== 0 for some reason)...

backtrace:
  #00  pc 0x000000000049587e  _v7a.apk!libppsspp_jni.so (GLRenderManager::SetNoBlendAndMask(int)+106)
  #01  pc 0x0000000000495311  _v7a.apk!libppsspp_jni.so (DrawEngineGLES::ApplyDrawState(int)+288)
  #02  pc 0x0000000000499071  _v7a.apk!libppsspp_jni.so (DrawEngineGLES::DoFlush()+1664)
  #03  pc 0x00000000005087f9  _v7a.apk!libppsspp_jni.so (GPUCommon::FastRunLoop(DisplayList&)+192)
  #04  pc 0x000000000050838f  _v7a.apk!libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+410)
  #05  pc 0x0000000000507c4d  _v7a.apk!libppsspp_jni.so (GPUCommon::ProcessDLQueue()+104)
  #06  pc 0x0000000000507b13  _v7a.apk!libppsspp_jni.so (GPUCommon::EnqueueList(unsigned int, unsigned int, int, PSPPointer<PspGeListArgs>, bool)+1462)
  #07  pc 0x00000000003a1519  _v7a.apk!libppsspp_jni.so (void WrapU_UUIU<&(sceGeListEnQueue(unsigned int, unsigned int, int, unsigned int))>()+48)
  #08  pc 0x00000000003842a5  _v7a.apk!libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+32)
  #09  pc 0x0000000000002022  [heap]

Locking a destroyed mutex, seems like in a VK driver:

  #00  pc 0x0000000000078b9c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180)
  #01  pc 0x000000000007a9fc  /apex/com.android.runtime/lib64/bionic/libc.so (__fortify_fatal(char const*, ...)+124)
  #02  pc 0x00000000000dd1cc  /apex/com.android.runtime/lib64/bionic/libc.so (HandleUsingDestroyedMutex(pthread_mutex_t*, char const*)+60)
  #03  pc 0x00000000000dd00c  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_mutex_lock+172)
  #04  pc 0x00000000006cf7e8  /vendor/lib64/egl/libGLES_mali.so
  #05  pc 0x00000000007eba54  /vendor/lib64/egl/libGLES_mali.so
  #06  pc 0x00000000006cd898  /vendor/lib64/egl/libGLES_mali.so
  #07  pc 0x00000000006cc328  /vendor/lib64/egl/libGLES_mali.so
  #08  pc 0x00000000006cb320  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (VKRGraphicsPipeline::Create(VulkanContext*)+80)
  #09  pc 0x00000000006cce4c  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (VulkanRenderManager::CompileThreadFunc()+256)
  #10  pc 0x00000000006d0e2c  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (VulkanRenderManager::*)(), VulkanRenderManager*> >(void*)+64)
          

Hm, SetNoBlendAndMask would be the first opportunity for ApplyDrawState() to interact with a missing curRenderStep_ in a lot of cases.

(VKRGraphicsPipeline::Create(VulkanContext*)+80) seems relatively early for the Vk calls in that func, but... if this isn't a driver bug it seems like it would either need to be a shutdown hazard or a pipeline cache lifetime issue, right?

-[Unknown]

Found a hang reported, where one of the threads were sitting here, very odd:

#00  pc 0x000000000055c5e0  !libppsspp_jni.so (JitBlockCache::GetBlockNumbersFromAddress(unsigned int, std::__ndk1::vector<int, std::__ndk1::allocator<int> >*)+128)
#01  pc 0x00000000003d5604 !libppsspp_jni.so (MIPSComp::Arm64Jit::DescribeCodePtr(unsigned char const*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >&)+176)
#02  pc 0x0000000000574f20 !libppsspp_jni.so (Memory::HandleFault(unsigned long, void*)+300)

Wonder if we're just handling a fault over and over, or if this was just random..

  • The very noisy GLES crashes are gone
  • Hangs are relatively getting more common (probably not a real change).
  • One hang I've seen a couple variants of is waiting for the vulkan render loop to exit, which doesn't happen for whatever reason then. Google Play reports this either as stuck in Java_org_ppsspp_ppsspp_NativeActivity_requestExitVulkanRenderLoop (which ends with a simple wait loop) or stuck in the actual thread.
  • Strange TransformUnit::Flush crash (software renderer):

    backtrace:
      #00  pc 0x00000000006c2070  libppsspp_jni.so (TransformUnit::Flush(char const*)+12)
      #01  pc 0x00000000006936d8  libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+688)
      #02  pc 0x0000000000692bd0  libppsspp_jni.so (GPUCommon::ProcessDLQueue()+100)
      #03  pc 0x0000000000693068  libppsspp_jni.so (GPUCommon::UpdateStall(int, unsigned int)+88)
      #04  pc 0x00000000004ce294  libppsspp_jni.so
      #05  pc 0x00000000004a7058  libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52)
      #06  pc 0x0000000000007820 
    

    Wacky bad_alloc in draw buffer (likely just OOM?):

    Thread
    terminating with uncaught exception of type std::bad_alloc: std::bad_alloc
    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    pid: 0, tid: 30603 >>> org.ppsspp.ppsspp <<<
    backtrace:
      #00  pc 0x00000000000895ec  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180)
      #01  pc 0x0000000000e22960  libppsspp_jni.so
      #02  pc 0x0000000000e22ab8  libppsspp_jni.so
      #03  pc 0x0000000000e1f9a8  libppsspp_jni.so
      #04  pc 0x0000000000e1efd4  libppsspp_jni.so
      #05  pc 0x0000000000e1ef30  libppsspp_jni.so (__cxa_throw+112)
      #06  pc 0x0000000000e156c0  libppsspp_jni.so (operator new(unsigned long)+96)
      #07  pc 0x0000000000612a70  libppsspp_jni.so (void std::__ndk1::vector<VKRGraphicsPipeline*, std::__ndk1::allocator<VKRGraphicsPipeline*> >::__push_back_slow_path<VKRGraphicsPipeline* const&>(VKRGraphicsPipeline* const&)+120)
      #08  pc 0x0000000000c597a4  libppsspp_jni.so (Draw::VKContext::DrawUP(void const*, int)+452)
      #09  pc 0x0000000000770588  libppsspp_jni.so (DrawBuffer::Flush(bool)+132)
      #10  pc 0x0000000000765aa4  libppsspp_jni.so (UIContext::Flush()+28)
      #11  pc 0x000000000078c464  libppsspp_jni.so (EmuScreen::renderUI()+3404)
      #12  pc 0x000000000078b3b0  libppsspp_jni.so (EmuScreen::render()+216)
      #13  pc 0x0000000000c61050  libppsspp_jni.so (ScreenManager::render()+212)
      #14  pc 0x0000000000781a74  libppsspp_jni.so (NativeRender(GraphicsContext*)+696)
      #15  pc 0x000000000077a5e8  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+428)
              

    ReadFromZip:

      #00  pc 0x0000000000c83430  libppsspp_jni.so (_zip_error_set_from_source+32)
      #01  pc 0x0000000000c88014  libppsspp_jni.so (zip_source_open+208)
      #02  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
      #03  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
      #04  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
      #05  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
      #06  pc 0x0000000000c8440c  libppsspp_jni.so (zip_fopen_index_encrypted+56)
      #07  pc 0x0000000000c4bca4  libppsspp_jni.so (ReadFromZip(zip*, char const*, unsigned long*)+52)
      #08  pc 0x0000000000c4bfbc  libppsspp_jni.so (ZipAssetReader::ReadAsset(char const*, unsigned long*)+92)
      #09  pc 0x00000000007157c8  libppsspp_jni.so (VFSReadFile(char const*, unsigned long*)+324)
      #10  pc 0x00000000007917fc  libppsspp_jni.so
      #11  pc 0x0000000000790680  libppsspp_jni.so (GameInfoWorkItem::Run()+1792)
      #12  pc 0x0000000000763514  libppsspp_jni.so
      #13  pc 0x0000000000764da8  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct>>, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*>>(void*)+48)
      #14  pc 0x00000000000d4358  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36)
      #15  pc 0x0000000000071aa8  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)
    

    Mic input (!)

    #00 pc 0x000000000004d4b4 /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+276) #01 pc 0x0000000000584490 libppsspp_jni.so (QueueBuf::pop(unsigned char*, unsigned int)+136) #02 pc 0x0000000000584948 libppsspp_jni.so (__MicInput(unsigned int, unsigned int, unsigned int, MICTYPE, bool)+408) #03 pc 0x000000000058217c libppsspp_jni.so #04 pc 0x00000000004a7058 libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52) #05 pc 0x000000000004b444

    JIT finalize block:

    backtrace:
      #00  pc 0x00000000003116b0  libppsspp_jni.so
      #01  pc 0x00000000003114c1  libppsspp_jni.so
      #02  pc 0x00000000003104fb  libppsspp_jni.so (JitBlockCache::FinalizeBlock(int, bool)+194)
      #03  pc 0x0000000000304c09  libppsspp_jni.so (MIPSComp::ArmJit::Compile(unsigned int)+140)
      #04  pc 0x0000000000000106 
    

    Strange TransformUnit::Flush crash (software renderer):

    Hm, unless binner_ is null or broken or something, not sure. Memory corruption? OOM when even allocating binner? It is a bit large with all its buffers.

    ReadFromZip:

    Strange. Not sure if we end up in the "layered" case, but this is a read from assets even? That zip shouldn't be corrupt or anything... it should be a user zip.

    Mic input (!)

    __MicInput doesn't seem to check the pointer validity at all, but this seems to be to a newd temp buffer. Maybe even also OOM...

    Not sure much about the others.

    -[Unknown]

    Been using addr2line on some stack traces that didn't resolve in the UI, there's one in the texture replacer here:

    		ParallelRangeLoop(&g_threadManager, [&](int l, int h) {
    			for (int y = l; y < h; ++y) {
    				memcpy((uint8_t *)out + rowPitch * y, &data[0] + info.w * 4 * y, info.w * 4);  << bad
    		}, 0, info.h, MIN_LINES_PER_THREAD);
    

    Also made a bunch of fixes in #16683

    The most common hang has the following as the only interesting stack trace:

      #00  pc 0x0000000000086b8c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
      #01  pc 0x000000000008a938  /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+144)
      #02  pc 0x00000000000ec784  /apex/com.android.runtime/lib64/bionic/libc.so (NonPI::MutexLockWithTimeout(pthread_mutex_internal_t*, bool, timespec const*)+688)
      #03  pc 0x0000000000e15010  libppsspp_jni.so (std::__ndk1::recursive_mutex::lock()+8)
      #04  pc 0x0000000000c60cf8  libppsspp_jni.so (ScreenManager::axis(AxisInput const&)+60)
      #05  pc 0x0000000000782a60  libppsspp_jni.so (NativeAxis(AxisInput const&)+304)
      #06  pc 0x00000000007790cc  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeApp_accelerometer+84)
    

    This is the only other stack that had PPSSPP code in it:

      #00  pc 0x0000000000086b8c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
      #01  pc 0x000000000008a938  /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+144)
      #02  pc 0x00000000000eab90  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_cond_wait+60)
      #03  pc 0x0000000000dda8e0  libppsspp_jni.so (std::__ndk1::condition_variable::wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&)+20)
      #04  pc 0x0000000000763560  libppsspp_jni.so
      #05  pc 0x0000000000764da8  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*> >(void*)+48)
    

    which makes me think it's during shutdown or startup.

    The second most common hang is a Vulkan shutdown hang, with the following stack traces:

    #02  pc 0x0000000000e1d4dc  libppsspp_jni.so (std::__ndk1::thread::join()+28)
    #03  pc 0x00000000007442e4  libppsspp_jni.so (VulkanRenderManager::StopThread()+432)
    #04  pc 0x00000000007444a8  libppsspp_jni.so (VulkanRenderManager::DestroyBackbuffers()+16)
    #05  pc 0x000000000077c8e4  libppsspp_jni.so (AndroidVulkanContext::ShutdownFromRenderThread()+80)
    #06  pc 0x000000000077a678  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+572)
    
    #00  pc 0x00000000000c67f8  /apex/com.android.runtime/lib64/bionic/libc.so (nanosleep+8)
    #01  pc 0x000000000008b6bc  /apex/com.android.runtime/lib64/bionic/libc.so (usleep+76)
    #02  pc 0x0000000000779870  !libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_requestExitVulkanRenderLoop+96)
    at org.ppsspp.ppsspp.NativeActivity.requestExitVulkanRenderLoop (Native method)
    at org.ppsspp.ppsspp.NativeActivity.joinRenderLoopThread (NativeActivity.java:718)
    at org.ppsspp.ppsspp.NativeActivity.onPause (NativeActivity.java:805)
    at android.app.Activity.performPause (Activity.java:8333)
    at android.app.Instrumentation.callActivityOnPause (Instrumentation.java:1510)
              

    Another, I think, GL one:

    #00  pc 0x000000000009a77c  /apex/com.android.runtime/lib/bionic/libc.so (getuid+12)
    #01  pc 0x000000000000a41d  /system/lib/liblog.so (PmsgWrite(log_id, timespec*, iovec*, unsigned int)+320)
    #02  pc 0x0000000000004ee9  /system/lib/liblog.so (write_to_log(log_id, iovec*, unsigned int)+128)
    #03  pc 0x0000000000004e4f  /system/lib/liblog.so (__android_log_logd_logger+66)
    #04  pc 0x000000000000c923  /system/lib/libbase.so (android::base::LogdLogChunk(android::base::LogId, android::base::LogSeverity, char const*, char const*)+98)
    #05  pc 0x000000000000c87f  /system/lib/libbase.so (void android::base::SplitByLogdChunks<void (android::base::LogId, android::base::LogSeverity, char const*, char const*)>(android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*, void  const(&)(android::base::LogId, android::base::LogSeverity, char const*, char const*))+834)
    #06  pc 0x000000000000c535  /system/lib/libbase.so (android::base::LogdLogger::operator()(android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)+30)
    #07  pc 0x000000000001a24d  /system/lib/libartbase.so (std::__1::__function::__func<art::InitLogging(char**, void (&)(char const*))::LogdLoggerLocked, std::__1::allocator<art::InitLogging(char**, void (&)(char const*))::LogdLoggerLocked>, void (android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)>::operator()(android::base::LogId&&, android::base::LogSeverity&&, char const*&&, char const*&&, unsigned int&&, char const*&&)+76)
    #08  pc 0x000000000000d8a5  /system/lib/libbase.so (android::base::SetLogger(std::__1::function<void (android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)>&&)::$_2::__invoke(__android_log_message const*)+120)
    #09  pc 0x00000000000050c9  /system/lib/liblog.so (__android_log_write_log_message+88)
    #10  pc 0x0000000000005281  /system/lib/liblog.so (__android_log_print+112)
    #11  pc 0x00000000005b6c73  !libppsspp_jni.so (AndroidLogger::Log(LogMessage const&)+82)
    #12  pc 0x00000000005ad797  !libppsspp_jni.so (LogManager::Log(LogTypes::LOG_LEVELS, LogTypes::LOG_TYPE, char const*, int, char const*, std::__va_list)+370)
    #13  pc 0x00000000005ad5d9  !libppsspp_jni.so (GenericLog(LogTypes::LOG_LEVELS, LogTypes::LOG_TYPE, char const*, int, char const*, ...)+68)
    #14  pc 0x0000000000ade36d  !libppsspp_jni.so (ScreenManager::render()+64)
    #15  pc 0x00000000005c1373  !libppsspp_jni.so (NativeRender(GraphicsContext*)+558)
    #16  pc 0x00000000005ba157  !libppsspp_jni.so (UpdateRunLoopAndroid(_JNIEnv*)+26)
    #17  pc 0x00000000005bbcd1  !libppsspp_jni.so
    #18  pc 0x000000000032b88d  !libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)()> >(void*)+24)
    #19  pc 0x00000000000aad13  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
    #20  pc 0x0000000000064193  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
    #00  pc 0x000000000005e3e8  /apex/com.android.runtime/lib/bionic/libc.so (syscall+28)
    #01  pc 0x0000000000063559  /apex/com.android.runtime/lib/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+92)
    #02  pc 0x00000000000aa393  /apex/com.android.runtime/lib/bionic/libc.so (pthread_cond_wait+32)
    #03  pc 0x0000000000bfbe49  !libppsspp_jni.so (std::__ndk1::condition_variable::wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&)+12)
    #04  pc 0x000000000057def7  !libppsspp_jni.so (GLRenderManager::ThreadFrame()+142)
    #05  pc 0x00000000005ba235  !libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeRenderer_displayRender+64)
    #06  pc 0x0000000000009053  /oat/arm/base.odex (Java_org_ppsspp_ppsspp_NativeRenderer_displayRender__+74)
    at org.ppsspp.ppsspp.NativeRenderer.displayRender (Native method)
    at org.ppsspp.ppsspp.NativeRenderer.onDrawFrame (NativeRenderer.java:33)
    at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1591)
    at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1286)
    
    at java.lang.Object.wait (Native method)
      at java.lang.Object.wait (Object.java:442)
      at java.lang.Object.wait (Object.java:568)
      at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1515)
      at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1286)
              

    Also unrelated, another TransformUnit one, but surely also OOM:

      #00  pc 0x000000000052c05c  !libppsspp_jni.so (TransformUnit::SetDirty(SoftDirty))
      #01  pc 0x00000000005272ed  !libppsspp_jni.so (SoftGPU::Execute_Prim(unsigned int, unsigned int)+296)
      #02  pc 0x0000000000529549  !libppsspp_jni.so (SoftGPU::FastRunLoop(DisplayList&)+72)
    

    Another, I think, GL one:

    That's weird, ScreenManager::render() only logs in bad situations - transparent over nothing or no screen. That probably would just hang...

    -[Unknown]

    Here's an oldie but goodie that have never fully gone away:

      #00  pc 0x00000000006951b4  !libppsspp_jni.so (GPUCommon::DoExecuteCall(unsigned int)+52)
      #01  pc 0x0000000000693b4c  !libppsspp_jni.so (GPUCommon::FastRunLoop(DisplayList&)+92)
      #02  pc 0x000000000069367c  !libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+596)
      #03  pc 0x0000000000692bd0  !libppsspp_jni.so (GPUCommon::ProcessDLQueue()+100)
      #04  pc 0x0000000000693068  !libppsspp_jni.so (GPUCommon::UpdateStall(int, unsigned int)+88)
      #05  pc 0x00000000004ce294  !libppsspp_jni.so
      #06  pc 0x00000000004a7058  !libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52)
      #07  pc 0x0000000000010eac 
    

    I'm considering throwing in an assert_msg so we can figure out which game it is happening in.. But surely it's from executing a bogus display list somehow.

    argh, looks like I caused a new kind of java exception:

    Exception java.lang.RuntimeException: Can't toast on a thread that has not called Looper.prepare()
      at android.widget.Toast$TN.<init> (Toast.java:411)
      at android.widget.Toast.<init> (Toast.java:121)
      at android.widget.Toast.makeText (Toast.java:290)
      at android.widget.Toast.makeText (Toast.java:280)
      at org.ppsspp.ppsspp.NativeRenderer.onSurfaceCreated (NativeRenderer.java:60)
      at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1541)
      at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1272)
    

    trying to display an error message:

    ppsspp/android/src/org/ppsspp/ppsspp/NativeRenderer.java Line 60 bd2a0c9

    Also hit this one:

    (GLRenderManager.h:SetBlendAndMask:801) Critical: [curRenderStep_ && curRenderStep_->stepType == GLRStepType::RENDER] *** Assertion ***

    Seems the thing I added to include the game ID isn't working? darn. don't see how that's possible though? Maybe we hit a length limit?

    Got a few variants of these:
    art/runtime/thread.cc:1238] Native thread exited without calling DetachCurrentThread: Thread[14,tid=30521,Native,Thread*=0x9ba87900,peer=0x12dc60a0,"Thread-1467"]

    I don't think Android normally asserts in this situation, might be a weird OS build. But, we are not handling thread detachment properly in all cases - getEnv() just attaches, and there's no mechanism to detach.

    This one we can't do much about, though we can handle it more gracefully:

    (MemArenaAndroid.cpp:Find4GBBase:160) Critical: [base != MAP_FAILED] Failed to map 256 MB of memory space: Out of memory (ULES00502 Grand Theft Auto: Vice City Stories)

    A bit weird (32-bit):

      #00  pc 0x0000000000691a74  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (XXH3_hashLong_64b_default)
      #01  pc 0x00000000006916c9  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (XXH3_64bits+596)
      #02  pc 0x00000000003107db  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (JitBlockCache::FinalizeBlock(int, bool)+98)
      #03  pc 0x0000000000304f49  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (MIPSComp::ArmJit::Compile(unsigned int)+140)
              

    That's expected, you need to uninstall to switch between APK and playstore due to mismatching signing keys.

    I currently don't know of a way to fix that.

    Here's one I haven't noticed before (as we stomp these out, I discover rarer crashes in the pile):

    Thread
    FORTIFY: pthread_mutex_destroy called on a destroyed mutex (0xd23f5370)
    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    pid: 0, tid: 23399 >>> org.ppsspp.ppsspp <<<
    backtrace:
      #00  pc 0x0000000000062d00  /apex/com.android.runtime/lib/bionic/libc.so (abort+172)
      #01  pc 0x00000000000ac23f  /apex/com.android.runtime/lib/bionic/libc.so (__fortify_fatal(char const*, ...)+26)
      #02  pc 0x00000000000ab979  /apex/com.android.runtime/lib/bionic/libc.so (HandleUsingDestroyedMutex(pthread_mutex_t*, char const*)+20)
      #03  pc 0x00000000000ac11f  /apex/com.android.runtime/lib/bionic/libc.so (pthread_mutex_destroy+126)
      #04  pc 0x0000000000c18971  libppsspp_jni.so (std::__ndk1::mutex::~mutex()+8)
      #05  pc 0x0000000000446433  libppsspp_jni.so (LocalFileLoader::~LocalFileLoader()+30)
      #06  pc 0x0000000000446469  libppsspp_jni.so (LocalFileLoader::~LocalFileLoader()+4)
      #07  pc 0x00000000005ce427  libppsspp_jni.so (GameInfoWorkItem::Run()+402)
      #08  pc 0x00000000005aaf11  libppsspp_jni.so
      #09  pc 0x00000000005ac0d9  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*> >(void*)+28)
      #10  pc 0x00000000000aaf93  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
      #11  pc 0x0000000000064203  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
              

    This should just be an OOM, right?

    Thread
    /buildbot/src/android/ndk-release-r21/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:72: abort_message: assertion "terminating with uncaught exception of type std::bad_alloc: std::bad_alloc" failed
    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    pid: 0, tid: 13497 >>> org.ppsspp.ppsspp <<<
    backtrace:
      #00  pc 0x000000000001ccb6  /system/lib/libc.so (abort+58)
      #01  pc 0x000000000001cf2f  /system/lib/libc.so (__assert2+22)
      #02  pc 0x0000000000c1fe3d  armeabi_v7a.apk
      #03  pc 0x0000000000c1ff3d  armeabi_v7a.apk
      #04  pc 0x0000000000c1e4fd  armeabi_v7a.apk
      #05  pc 0x0000000000c1ddab  armeabi_v7a.apk
      #06  pc 0x0000000000c1dd73  armeabi_v7a.apk (__cxa_throw+74)
      #07  pc 0x0000000000c189dd  armeabi_v7a.apk (operator new(unsigned int)+56)
      #08  pc 0x000000000034482f  armeabi_v7a.apk (std::__ndk1::vector<unsigned char, std::__ndk1::allocator<unsigned char>>::__append(unsigned int)+100)
      #09  pc 0x00000000004665a3  armeabi_v7a.apk (SaveState::SaveToRam(std::__ndk1::vector<unsigned char, std::__ndk1::allocator<unsigned char>>&)+118)
      #10  pc 0x000000000046ba11  armeabi_v7a.apk (SaveState::StateRingbuffer::Save()+132)
      #11  pc 0x000000000046af03  armeabi_v7a.apk (SaveState::Process()+3314)
      #12  pc 0x000000000046ee77  armeabi_v7a.apk (PSP_RunLoopWhileState()+94)
      #13  pc 0x00000000005ca65d  armeabi_v7a.apk (EmuScreen::render()+220)
      #14  pc 0x0000000000adf29b  armeabi_v7a.apk (ScreenManager::render()+126)
      #15  pc 0x00000000005c20b7  armeabi_v7a.apk (NativeRender(GraphicsContext*)+558)
      #16  pc 0x00000000005bc2c7  armeabi_v7a.apk (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+334)
      #17  pc 0x0000000000009105  /data/app/org.ppsspp.ppsspp-ARrsgkSFQAgRLhxXT4yFDQ==/oat/arm/base.odex (org.ppsspp.ppsspp.NativeActivity.runVulkanRenderLoop+92)
              
    backtrace:
      #00  pc 0x00000000003105c2  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (JitBlockCache::ProxyBlock(unsigned int, unsigned int, unsigned int, unsigned char const*)+278)
      #01  pc 0x000000000030599f  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::ReplaceJalTo(unsigned int)+510)
      #02  pc 0x00000000002f8d79  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::Comp_Jump(Memory::Opcode)+204)
      #03  pc 0x0000000000456dcb  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSCompileOp(Memory::Opcode, MIPSComp::MIPSFrontendInterface*)+166)
      #04  pc 0x0000000000305229  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::DoJit(unsigned int, JitBlock*)+424)
      #05  pc 0x0000000000304f3d  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::Compile(unsigned int)+128)
      #06  pc 0x0000000000000106 
    

    This one is actually interesting. I think we might be missing an IsFull and wipe check before we link/proxy blocks...

    Thread
    /buildbot/src/android/ndk-release-r21/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:72: abort_message: assertion "terminating" failed
    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    pid: 0, tid: 26034 >>> org.ppsspp.ppsspp <<<
    backtrace:
      #00  pc 0x0000000000065668  /apex/com.android.runtime/lib/bionic/libc.so (abort+172)
      #01  pc 0x00000000000658d7  /apex/com.android.runtime/lib/bionic/libc.so (__assert2+22)
      #02  pc 0x0000000000c1fe3d  libppsspp_jni.so
      #03  pc 0x0000000000c1fec3  libppsspp_jni.so
      #04  pc 0x0000000000c1e4fd  libppsspp_jni.so
      #05  pc 0x0000000000c1e495  libppsspp_jni.so (std::terminate()+28)
      #06  pc 0x0000000000c1cc1d  libppsspp_jni.so (std::__ndk1::thread::~thread()+12)
      #07  pc 0x000000000047b459  libppsspp_jni.so (std::__ndk1::__shared_ptr_pointer<std::__ndk1::thread*, std::__ndk1::default_delete<std::__ndk1::thread>, std::__ndk1::allocator<std::__ndk1::thread> >::__on_zero_shared()+12)
      #08  pc 0x0000000000477f87  libppsspp_jni.so (GameManager::InstallGameOnThread(Path const&, Path const&, bool)+154)
      #09  pc 0x0000000000477e65  libppsspp_jni.so (GameManager::Update()+624)
      #10  pc 0x00000000005c1ead  libppsspp_jni.so (NativeRender(GraphicsContext*)+36)
      #11  pc 0x00000000005bae57  libppsspp_jni.so (UpdateRunLoopAndroid(_JNIEnv*)+26)
      #12  pc 0x00000000005bc9e1  libppsspp_jni.so
      #13  pc 0x000000000032bbcd  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)()> >(void*)+24)
      #14  pc 0x00000000000b0567  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
      #15  pc 0x0000000000066b37  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
              

    here's one curious assert that we probably should try to avoid crashing on (omitting the callstack because uninteresting):

    (VulkanFrameData.cpp:AcquireNextImage:80): [false] (ULES00502 Grand Theft Auto: Vice City Stories) vkAcquireNextImageKHR failed! result=VK_TIMEOUT
              

    Are all the most common crashes captured here / potentially resolved? Should we close this for next time around?

    -[Unknown]