[GSoC 2011 - Accepted] Unit Testing Framework

Threads related to Google Summer of Code
Post Reply
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by jacmoe »

You can freely create pages - best would be to let them be child pages of your main page.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 135
Contact:

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by spacegaier »

At least for now...after the project is done, we should probably then move it in a more fitting wiki area ("Using Ogre"?).
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Okay, I suppose for now it makes the most sense to keep it all in one place (as a child (or children) of the main project page), but eventually it'll be moved somewhere more general.

As I said before I'm bogged down studying right now, but I've begun outlining stuff a bit, and I'll be able to really get into it after tomorrow morning.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

wiki page titled "Visual Unit Testing Framework"

Post by Assaf Raman »

Create a new wiki page titled "Visual Unit Testing Framework" and reference it from your wiki page.
That page plus the fork changes are the result of the your summer project.
The current wiki page you have for the project - should focus on the process of your project, the page titled "Visual Unit Testing Framework" is for the ages and will be part of the permanent OGRE documentation - long after how this feature started will be forgotten (you can give yourself a small credit there but the focus of the page is the feature and not you or the summer project), it intended audience are readers that have basic OGRE knowledge, but know nothing about this feature, meaning you have to explain in full - the why, how and what for a novice reader.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Alright, that's what I thought, thanks for clarifying. :)

Here's the initial skeleton of the page (most of the content is off the top of my head, and could probably be explained better, but it's a start): http://www.ogre3d.org/tikiwiki/Visual+U ... +Framework

It's a child of the GSoC project page at the moment, but it can eventually be migrated to wherever it fits best as general documentation.

I'll be adding to and refining it a over the next while (and as the project progresses, of course).
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

@Praetorian: Good work, the wiki page you created is what I meant.

BTW: Do we have any good solution for the case where random numbers are used (ex: particle systems) - or is it an open issue (it is ok for it to be) ?. (the problem I see is that the images will differ between screenshots...)
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Yeah, particles (and other random stuff) are indeed the main area I'm a little shaky on.

In theory I suppose just always srand-ing with something predictable might do it (I've dug about a bit and everything seems to just use plain old rand (Math::UnitRandom calls asm_rand, which just calls rand)), although that feels a bit iffy, I'll play around with it a bit and get back to you.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Well, the delta time between frames is also random - so rand init is not the only issue.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Assaf Raman wrote:Well, the delta time between frames is also random - so rand init is not the only issue.
For animations and such, just using a fixed time step, regardless of the actual delta was sufficient (in my earlier testing I just always passed 1/100 of a second to the update function for the tests; this meant it would run faster/slower depending on framerate, but that you could guarantee the state would be identical after a set number of frames). This seems to work fine for anything being timed in the test itself (animations being advanced, stuff being moved around, etc).

For timing that's done automatically within OGRE, it appears that particles and most anything of that nature (texture animation, ribbon trails, etc) can be controlled with ControllerManager. So I think using ControllerManager::setFrameDelay might just do it.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Sounds right.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Good news! I got some time to test particles, and after srand-ing with a constant value and using ControllerManager to set a fixed timestep, it seems to be nice and deterministic. :D

I'm getting identical images (checked by hashing), with a lot of particles, emitted emitters, etc.

Here's the test shot, I just dropped a few particle systems into one of the playpen tests I was using earlier: (it's 1920x1080, so I won't embed and break the forum width...)
http://img807.imageshack.us/img807/2062 ... letest.png
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Can't see the image, use google docs or photobucket for image upload.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Alright, I uploaded to photobucket: http://s1233.photobucket.com/albums/ff3 ... leTest.png

EDIT: Hmmm, looks like that re-sized it a good deal, so for good measure, here is the raw file on my personal site.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Thanks.
Regarding what you wrote - nice work, sounds good, you are on the right way.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Alright, so I haven't posted here in a while, but I didn't really feel like I had anything else worth mentioning...

Over this weekend I'll be finalizing the initial wiki documentation and sorting out planning and preparation for the start of coding, I'll post as I make progress and/or have questions. Looking forward to getting started! :)
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

What is the plan for this week?
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

I'll be creating a "VisualTest" base class for the tests (derived from Sample, with the addition of screenshot timing, a fixed timestep and other measures to ensure determinism), and converting the existing 10 or so playpen tests that are already setup as samples to use this.

The end result will be an initial set of playpen tests that run in the existing sample browser, with deterministic behavior and timed screenshots. This is pretty much what I've already done for the sake of testing the concept so it should be pretty straightforward, but it will be a nice way to ease into it and get the workflow figured out.

At the moment I'm getting more familiar with cmake.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Good week.
Are you going to code today?
If so - what are today's goals?
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Well, it's 1am here, so it'll be a quite a bit later today, but I think I will start a bit of coding.

Rough Goals:
  • 1) - Setup directory that most of the project will live in (Tests/VisualTests seems appropriate?), with a basic CMakeLists.txt, add it as a cmake subdirectory, and sort out any other build considerations (there won't be anything to build yet, but at least get the basics in place).
  • 2) - Make the basic skeleton/interface of the VisualTest class
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Sounds good.
I added numbering to your goals - for easy reference.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

Good morning.
How did it go yesterday?
What are today's goals?
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

I did the basic setup and added a basic skeleton of the VisualTest class as planned. Just pushed to the fork.

It looks like the existing Sample Browser only works with SdkSample's (as opposed to plain Sample; it includes some camera/sdktray stuff which is unnecessary for tests), in order to make the tests work with the browser I'll be deriving from SdkSample for the time being.

However, once the TestContext is created (next week, or perhaps this weekend), I'll be able to derive from the plain Sample class, and make everything a little more cohesive (the screenshot taking/timing callbacks can be called from the Context at frame start/end, and the sranding and ControllerManager timing stuff can be moved over to the TestContext as well).

So overall I should be able to get the basic functionality in place for this week, but it will be changing up a bit once the TestContext is created.

Tomorrow I think I'm going to mainly be doing some schoolwork to clear up time later in the week, but if I have time I may try and get one of the playpen tests converted and running.
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

I am not sure what you mean...
Can you create a simple sequence diagram (possibly using this site) - to demonstrate the use of the "TestContext" class.
Watch out for my OGRE related tweets here.
User avatar
Praetorian
Google Summer of Code Student
Google Summer of Code Student
Posts: 171
Joined: Fri Aug 10, 2007 10:37 pm
Location: WA - USA
x 5

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Praetorian »

Oh, sorry for the confusion! It's getting late here and I don't think I could make a very clear diagram in the time I have right now, so for now I'll just try and explain:

The TextContext is to tests what the SampleBrowser is to samples, it will be derived from SampleContext. So it's responsible for setting up the common environment for the tests to run in, starting/stopping and updating the tests (so as I was saying, it would be cleaner to move the timing stuff to the context; but until then I'll have to work around the SampleBrowser a little bit to get the initial tests working).
My Google summer of code 2011 topic: Unit Testing Framework
My Google summer of code thread
My Google summer of code wiki page
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2011 - Accepted] Unit Testing Framework

Post by Assaf Raman »

I see, ok.
Watch out for my OGRE related tweets here.
Post Reply