usage in windowing environment (crashes)

Design / architecture / roadmap discussions related to future of Ogre3D (version 2.0 and above)
Post Reply
bayoubengal
Gnoblar
Posts: 10
Joined: Wed Sep 05, 2018 3:18 pm

usage in windowing environment (crashes)

Post by bayoubengal » Wed Sep 05, 2018 3:43 pm

hi.

I'm attempting to stage Ogre 2.1 in a cocoa app to evaluate its use for a feature of desktop mac application for data visualization. I need to configure the system so that Windows that draw 3D content can be open and closed on demand. I had little trouble getting the metal renderer to function without crashing when closing and reopening a window, but the opengl backend crashes. I've been attempting to trace through your code and figure out appropriate workarounds so I can proceed to prototyping the drawing code.

The first problem I faced was GL3PlusRenderSystem using a CocoaContext pointer after it had been deleted by "CocoaWindow::destroy(void)". To work around that problem, I added

GL3PlusRenderSystem *rs = static_cast<GL3PlusRenderSystem*>(Root::getSingleton().getRenderSystem());

if(this->mContext != 0)
{
rs->_unregisterContext(this->mContext);
}

before "OGRE_DELETE mContext;"


Then, it crashed in GL3PlusVaoManager::waitFor() because stale data was left in GL3PlusVaoManager::mFrameSyncVec. I added the following function to work around the problem:

void GL3PlusVaoManager::clearFrameSync()
{
GLSyncVec::iterator itor = this->mFrameSyncVec.begin();
GLSyncVec::iterator end = this->mFrameSyncVec.end();

while( itor != end )
{
OCGE( glDeleteSync( *itor ) );
*itor = 0;
++itor;
}

}

and called it from GL3PlusRenderSystem::_unregisterContext(GL3PlusContext *context)

after "mCurrentContext->endCurrent();"


Now, I am finding that HlmsPbs is crashing because mPassBuffers has a stale pointer left in it and it crashes at "*passBufferPtr++ = (float)viewProjMatrix[0];"


However, I'm not sure what the right thing to do here is. How am I supposed to know which pointers in the pass buffer list belong to the RenderWindow/SceneMgr/CompositorWorkspace that is being destroyed by the destruction of the window controller. I want to close out the resources that belong to the current window controller object while leaving intact the resources to other existing window controllers.

I can put up some code snippets to show what I'm doing, but before I do that, could I get some explanation of what the conceptually correct process is for my use-case?
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: usage in windowing environment (crashes)

Post by dark_sylinc » Wed Sep 05, 2018 7:23 pm

Could you post a callstack?

In OpenGL, the GL context is tied to the window. Usually when dealing with your kind of programs (non-games, uses multiple windows that close/open on demand) the recommendation is to create a hidden 1x1 window that is always alive so the GL context doesn't die; as recreating the context without making mistakes is painful.
Ogre usually assigns the first window you create as the owner of the context. Probably your bugs are happening because you're destroying the first window, without destroying the rest of the windows.

Once you've destroyed the last render window, it's no longer valid to perform more rendering calls. Crashing inside GL3PlusVaoManager::waitFor indicates that either you're attempting to run more 3D commands after the window has been destroyed, or Ogre has a bug in which the window is deleted too early.
0 x

bayoubengal
Gnoblar
Posts: 10
Joined: Wed Sep 05, 2018 3:18 pm

Re: usage in windowing environment (crashes)

Post by bayoubengal » Thu Sep 06, 2018 3:31 pm

thanks!

So, the first window I create should be a static, hidden window that should outlast all normal UI windows? Is this context the one referenced by the variable "GL3PlusRenderSystem::mMainContext"?

Do I need to use the same technique for the metal backend? My design requires the window to function on both metal-capable systems and legacy opengl-only systems. I'm statically linking the two rendering plugins and dynamically deciding which one to initialize the first time a 3D window is needed.

-James
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: usage in windowing environment (crashes)

Post by dark_sylinc » Thu Sep 06, 2018 4:24 pm

bayoubengal wrote:
Thu Sep 06, 2018 3:31 pm
So, the first window I create should be a static, hidden window that should outlast all normal UI windows? Is this context the one referenced by the variable "GL3PlusRenderSystem::mMainContext"?
Yes to both.
Create a 1x1 hidden window first, and it should outlast all normal UI windows.
bayoubengal wrote:
Thu Sep 06, 2018 3:31 pm
Do I need to use the same technique for the metal backend? My design requires the window to function on both metal-capable systems and legacy opengl-only systems. I'm statically linking the two rendering plugins and dynamically deciding which one to initialize the first time a 3D window is needed.
To be honest I don't know. Metal in theory does not tie windows with devices, so it should not need this, but there may be old lingering Ogre code that may destroy the Metal device pointer if it sees it ran out of windows.
Our D3D11 backend for example requires the 1x1 window due to old code carried over from D3D9. I wrote the Metal RenderSystem from scratch, and from what I remember the Metal device should not be getting destroyed with windows, but I may be wrong (specially if the base/shared code does something I'm not considering).

Cheers
0 x

bayoubengal
Gnoblar
Posts: 10
Joined: Wed Sep 05, 2018 3:18 pm

Re: usage in windowing environment (crashes)

Post by bayoubengal » Thu Sep 06, 2018 5:06 pm

thanks.

I'll report back if I see a crash. So far, I have seen none.


on the subject of the metal backend, in several headers, you are making objc pointer declarations in header files and your system is configured for ARC.

examples:

id<CAMetalDrawable> mCurrentDrawable;
id<MTLTexture> mMsaaTex;
NSWindow *mWindow;
id mResizeObserver;

in MetalRenderWindow. This design causes complications if the host app is not using ARC. class declarations containing objc pointers cannot be used across an ARC/non-ARC boundary. The clang docs declare such a pattern to be a violation of the one-definition rule. Doing so causes a retain/release crash that a naive client programmer will have no way to understand. I knew what it was doing because I have a history of having to mingle ARC and non-ARC code and could immediately recognize the pattern.

The problem occurs when one of these headers such as OgreMetalRenderWindow.h gets included in a non-ARC module in the host app. There is a workaround, but you have to know why it is failing to understand that you need the workaround. if you wish to avoid the problem, the best course of action would be to store your objc pointers in your headers as CFTypeRef's then write private accessor functions that access the variables and map them back to objc types internally using bridge casts and such. That way, client code would never see them as objc pointers causing clang to inject unnecessary retains, releases and autoreleases. It is possible to write a smart pointer template that preserves the objc type while encapsulating a CFTypeRef for the actual storage but works correctly regardless of ARC state.

Sadly, Apple (in its usual half-baked fashion) didn't fully think out how to correctly interface arc and non-arc modules when c++ class/struct definitions are involved. they just cited the one-definition rule and punted...leaving their developers to suffer hidden pitfalls.


-James
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: usage in windowing environment (crashes)

Post by dark_sylinc » Thu Sep 06, 2018 5:19 pm

I see, I was not aware of that problem. Though I recall assuming most apps would chose to use ARC in the future.

A simple solution I can see is to ifdef them or wrap them around a macro to define them as void ptrs. e.g.

Code: Select all

#if __has_feature(objc_arc)
    #define OGRE_ARC_SAFE( decl, variable ) decl variable
#else
    #define OGRE_ARC_SAFE( decl, variable ) void* variable
#endif

OGRE_ARC_SAFE( id<CAMetalDrawable>, mCurrentDrawable );
Since I'm far from an expert in dealing with ARC - non-ARC enabled interactions, if you see this is a viable solution you're welcome to test it and submit a PR :)

To reduce verbosity it looks like this type of thing would only be needed on classes that normally face the user, such as MetalRenderWindow and MetalDevice.
0 x

bayoubengal
Gnoblar
Posts: 10
Joined: Wed Sep 05, 2018 3:18 pm

Re: usage in windowing environment (crashes)

Post by bayoubengal » Thu Sep 06, 2018 5:52 pm

a CFTypeRef is a const void*

...another poor design choice by Cupertino's C-zealots who work as API designers...but it is what it is. trying to bridge objc pointers to non-CFType pointers is very painful due to compiler type errors.

yes. you only need to worry about it on headers exposed to public code so long as all your internal code is all ARC.
if you see this is a viable solution you're welcome to test it and submit a PR :)
sure. I have stuff pre-baked for all of this, but my existing code requires at least c++14 which you aren't using yet. But I can whip up some templating that solves the problem conveniently for c++03.
most apps would chose to use ARC in the future.
most NEW code uses ARC. but OSX devs have as much as 30 years of legacy code going back to classic MacOS. taking legacy code to ARC is often not-viable. Personally, i don't like ARC even for new code because it brings its own set of problems because the compiler is implicitly injecting actions that aren't necessarily visible to the human eye. you can get subtle bugs because you created retain-cycles without knowing it. Then you have complexities with bridge-casts whenever you have to interface with CFTypes or play unusual games with pointers....like passing them to legacy C APIs that require context pointers. I wrote my own system of smart pointers using c++14 features that work wonderfully at the expense of a little extra typing without all the implicit behaviors and casting complexities of ARC...or the fragility problems of manual retain/release management.

-James
0 x

bayoubengal
Gnoblar
Posts: 10
Joined: Wed Sep 05, 2018 3:18 pm

Re: usage in windowing environment (crashes)

Post by bayoubengal » Thu Sep 06, 2018 7:11 pm

ok. I set up the following opengl init code once the renderer was loaded:

Code: Select all

   Ogre::NameValuePairList tmpRenderWindowParameters;
   auto tmpMainRenderWindowPtr = theRoot.createRenderWindow("main window", 1.0f, 1.0f, false, &tmpRenderWindowParameters);

   static auto statMainRenderWindowSPtr = make_custom_guard(tmpMainRenderWindowPtr, [](Ogre::RenderWindow* thePtr) mutable -> void
   {
      auto tmpOgreRootPtr = Ogre::Root::getSingletonPtr();
      
      if(tmpOgreRootPtr == nullptr)
      {
         return;
      }
      
       tmpOgreRootPtr->destroyRenderTarget(thePtr);
      
   });
then removed all the patches I made to avoid crashes. It still crashed at

Code: Select all

        // It's ready for switching
        if (mCurrentContext)
            mCurrentContext->endCurrent();
when opening a new window instance.

calling "_unregisterContext()" in "CocoaWindow::destroy(void)" seems to alleviate the crash.

Code: Select all

            // Unregister and destroy OGRE GL3PlusContext
            GL3PlusRenderSystem *rs = static_cast<GL3PlusRenderSystem*>(Root::getSingleton().getRenderSystem());
           
            if(this->mContext != 0)
            {
               rs->_unregisterContext(this->mContext);
            }
            OGRE_DELETE mContext;


Is this the correct answer?





crash backtrace:

Code: Select all

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x00000001015aab1c Vx76-[Debug]`Ogre::GL3PlusRenderSystem::_switchContext(this=0x0000000106844200, context=0x0000600007240540) at OgreGL3PlusRenderSystem.cpp:3281 [opt]
    frame #1: 0x00000001015aad54 Vx76-[Debug]`Ogre::GL3PlusRenderSystem::_setRenderTarget(this=0x0000000106844200, target=0x0000000103b563d0, viewportRenderTargetFlags='\x01') at OgreGL3PlusRenderSystem.cpp:3458 [opt]
    frame #2: 0x00000001014ef28f Vx76-[Debug]`Ogre::CompositorWorkspace::CompositorWorkspace(this=0x0000000103961be0, id=1, definition=0x0000000103961090, externalRenderTargets=<unavailable>, sceneManager=<unavailable>, defaultCam=<unavailable>, renderSys=0x0000000106844200, bEnabled=<unavailable>, executionMask='\xff', viewportModifierMask='\0', vpOffsetScale=0x0000000101b73eb8, uavBuffers=0x0000000000000000, initialLayouts=0x0000000000000000, initialUavAccess=0x0000000000000000)0> > > const&, Ogre::SceneManager*, Ogre::Camera*, Ogre::RenderSystem*, bool, unsigned char, unsigned char, Ogre::Vector4 const&, std::__1::vector<Ogre::UavBufferPacked*, Ogre::STLAllocator<Ogre::UavBufferPacked*, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceLayout::Layout, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceLayout::Layout>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceAccess::ResourceAccess, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceAccess::ResourceAccess>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*) at OgreCompositorWorkspace.cpp:92 [opt]
    frame #3: 0x00000001014dd146 Vx76-[Debug]`Ogre::CompositorManager2::addWorkspace(this=0x0000000103f400b0, sceneManager=0x000000012f880a10, externalRenderTargets=size=1, defaultCam=0x0000000132280c00, definitionName=<unavailable>, bEnabled=true, position=-1, uavBuffers=0x0000000000000000, initialLayouts=0x0000000000000000, initialUavAccess=0x0000000000000000, vpOffsetScale=0x0000000101b73eb8, vpModifierMask='\0', executionMask='\xff')0> > > const&, Ogre::Camera*, Ogre::IdString, bool, int, std::__1::vector<Ogre::UavBufferPacked*, Ogre::STLAllocator<Ogre::UavBufferPacked*, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceLayout::Layout, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceLayout::Layout>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceAccess::ResourceAccess, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceAccess::ResourceAccess>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, Ogre::Vector4 const&, unsigned char, unsigned char) at OgreCompositorManager2.cpp:502 [opt]
    frame #4: 0x00000001014dceef Vx76-[Debug]`Ogre::CompositorManager2::addWorkspace(this=0x0000000103f400b0, sceneManager=0x000000012f880a10, finalRenderTarget=0x0000000103b563d0, defaultCam=0x0000000132280c00, definitionName=<unavailable>, bEnabled=true, position=-1, uavBuffers=0x0000000000000000, initialLayouts=0x0000000000000000, initialUavAccess=0x0000000000000000, vpOffsetScale=0x0000000101b73eb8, vpModifierMask='\0', executionMask='\xff')0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceLayout::Layout, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceLayout::Layout>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, std::__1::map<Ogre::GpuResource*, Ogre::ResourceAccess::ResourceAccess, std::__1::less<Ogre::GpuResource*>, Ogre::STLAllocator<std::__1::pair<Ogre::GpuResource* const, Ogre::ResourceAccess::ResourceAccess>, Ogre::CategorisedAllocPolicy<(Ogre::MemoryCategory)0> > > const*, Ogre::Vector4 const&, unsigned char, unsigned char) at OgreCompositorManager2.cpp:474 [opt]
    frame #5: 0x0000000100e95222 Vx76-[Debug]`::-[C3DWindowController initSPtr](self=0x0000000103b297f0, _cmd="initSPtr") at C3DWindowController.mm:800
    frame #6: 0x000000010014ab20 Vx76-[Debug]`::-[com_prg_console_ui_window_WindowManager createWindow:usingIdentity:withSnapshotDictionary:andFrame:andLogicalDisplayId:andToolbarInfo:andShow:](self=0x0000000103a01cc0, _cmd="createWindow:usingIdentity:withSnapshotDictionary:andFrame:andLogicalDisplayId:andToolbarInfo:andShow:", theControllerClassPtr=C3DWindowController, theIdentitySPtr=0x00006140001468d0, theSnapshotDictionaryPtr=0x000060400a674c00, theFrame=(origin = (x = 166, y = 282), size = (width = 611, height = 693)), theLogicalDisplayId=1, theToolbarInfo=size=3, doShow=true) at manager.mm:1201
    frame #7: 0x0000000100149f3b Vx76-[Debug]`::-[com_prg_console_ui_window_WindowManager createWindow:usingIdentity:withSnapshotDictionary:andFrame:andLogicalDisplayId:andToolbarInfo:andShow:usingCompletionHandler:](self=0x0000000103a01cc0, _cmd="createWindow:usingIdentity:withSnapshotDictionary:andFrame:andLogicalDisplayId:andToolbarInfo:andShow:usingCompletionHandler:", theControllerClassPtr=C3DWindowController, theIdentitySPtr=0x00006140001468d0, theSnapshotDictionaryPtr=0x000060400a674c00, theFrame=(origin = (x = 166, y = 282), size = (width = 611, height = 693)), theLogicalDisplayId=1, theToolbarInfo=size=3, doShow=true, theCompetionHandler=0x00006140001468f0) at manager.mm:1144
    frame #8: 0x0000000101062d63 Vx76-[Debug]`auto com::prg::system::objcSendSuper<Vx76WindowManager, objc_class*, std::__1::unique_ptr<NSString, com::prg::system::retain_counted_obj_desc>&&, NSDictionary*, CGRect, unsigned long, std::__1::tuple<bool, NSToolbarDisplayMode, NSToolbarSizeMode>, bool, std::__1::packaged_task<void (std::__1::shared_ptr<VStandardWindowController>)>&&>(theObjPtr=0x0000000103a01cc0, theSelector="createWindow:usingIdentity:withSnapshotDictionary:andFrame:andLogicalDisplayId:andToolbarInfo:andShow:usingCompletionHandler:", theArgs=0x00006140001468d8, theArgs=0x00006140001468d0, theArgs=0x00007ffeefbfd710, theArgs=(origin = (x = 166, y = 282), size = (width = 611, height = 693)), theArgs=0x00006140001468c8, theArgs=size=3, theArgs=0x00006140001468e0, theArgs=0x00006140001468f0)>&&&&) at objcSupport.h:1490
    frame #9: 0x0000000101062b16 Vx76-[Debug]`-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27::operator(this=0x0000614000146890)() at Vx76WindowManager.mm:3615
    frame #10: 0x0000000101062659 Vx76-[Debug]`std::__1::__packaged_task_func<-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27, std::__1::allocator<-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27>, void ()>::operator()() [inlined] decltype(__f=0x0000614000146890)(std::__1::forward<>(fp0))) std::__1::__invoke<-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27&>(-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27&&&) at type_traits:4428
    frame #11: 0x0000000101062648 Vx76-[Debug]`std::__1::__packaged_task_func<-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27, std::__1::allocator<-[Vx76WindowManager processWindowCreation:fromStoredState:andShow:usingCompletionHandler:]::$_27>, void ()>::operator(this=0x0000614000146880)() at future:1826
    frame #12: 0x00000001015ca273 Vx76-[Debug]`std::__1::packaged_task<void ()>::operator()() [inlined] std::__1::__packaged_task_function<void ()>::operator(this=0x00006140000a5c60)() const at future:2003
    frame #13: 0x00000001015ca263 Vx76-[Debug]`std::__1::packaged_task<void ()>::operator(this=0x00006140000a5c60)() at future:2223
    frame #14: 0x00000001015c9cfa Vx76-[Debug]`com::prg::system::handleDispatchingQueuePackage(theContextPtr=0x00006140000a5c40) at system.cpp:398
    frame #15: 0x00000001015dfd5e Vx76-[Debug]`com::prg::system::handleDispatchingQueuePackage_objc(theContextPtr=0x00006140000a5c40) at system.mm:686
    frame #16: 0x00007fff61aa1db8 libdispatch.dylib`_dispatch_client_callout + 8
    frame #17: 0x00007fff61aad395 libdispatch.dylib`_dispatch_main_queue_callback_4CF + 1148
    frame #18: 0x00007fff39c2fe59 CoreFoundation`__CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
    frame #19: 0x00007fff39bf266a CoreFoundation`__CFRunLoopRun + 2586
    frame #20: 0x00007fff39bf19b7 CoreFoundation`CFRunLoopRunSpecific + 487
    frame #21: 0x00007fff38ed1d96 HIToolbox`RunCurrentEventLoopInMode + 286
    frame #22: 0x00007fff38ed1b06 HIToolbox`ReceiveNextEventCommon + 613
    frame #23: 0x00007fff38ed1884 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 64
    frame #24: 0x00007fff37182a73 AppKit`_DPSNextEvent + 2085
    frame #25: 0x00007fff37918e34 AppKit`-[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044
    frame #26: 0x00007fff37177885 AppKit`-[NSApplication run] + 764
    frame #27: 0x00007fff37146a72 AppKit`NSApplicationMain + 804
    frame #28: 0x0000000100460622 Vx76-[Debug]`main(argc=3, argv=0x00007ffeefbff3a0) at main.mm:29
    frame #29: 0x00007fff61adb015 libdyld.dylib`start + 1
0 x

Post Reply