Motion Capture Workflow

Today I have been trying to produce a work flow to get my motion capture data into maya, remapping the data onto a custom rig.

My first task i wanted to do was to use the norman rig driven by my mocap data. For this i recorded a take in motion builder, I left it how it was, as i wanted to produce a workflow of gtting my data onto a rig, and then exported the motion file as a .FBX file.

I imported this into maya, which worked and moved fine. I then imported norman and moved him into position.

I then labeled the joints on both skeletons the same. I then used the Human IK tab to remap the motion from the kinect data onto my norman rig. To do this i created new character nodes and mapped the joints to the corresponding slots and characterised both of the skeletons.  Up to this point everything has worked fine.

I then tried to create a control rig for the animation to be mapped onto which would then control the rig, however when i tried to do this maya threw up constant error messages telling me that bones had incoming connections ect.. I tried removing all the connections, but nothing seemed to work.

My next idea was to try use the kinect skeleton to drive the character via constraints, so i set up locators over all the joints and parent constrained them to them so they would move with the joints, however when i tried to orient/point constraint them it wouldnt let me as the bones had incomming connections from the animation that i could not break.

After lots of research all saying do exactly what i just tried, i decided to take a new approach at it. I recorded a new take, with my motion capture starting in the T pose. I then exported it from MB into maya and instead of trying to map it onto a custom rig i thought i would try to use the skeleton generator within the HIK toolset.

I mapped out the bones and labled them, I then characterized the skeletons, added a control rig to the character and then retargeted the animation, this worked nicely, i just need to find a way to use this method on custom rigs.

I also need to work out a way of cleaning up my motion capture data, I created a control rig in MB that would feed data back into my kinect skeleton, I added Foot contact to the ground and some stabilization, however apart from the foot contact it doesn’t seem to be having much effect on cleaning up the data.


Motion Capture

I have been testing the kinect a bit more this week, I found that although the tracking struggles with extreme movement or turning around, it does work nicely for more subtle animation.

Here i had Luke just explaining something to me as i captured the take, this way i could achieve more natural movement.

(the acting in this isn’t very good so it feels stiffer than it should.)

I then fed this motion capture data into a pre made characterised rig within Motion Builder, so i could work on cleaning the motion capture data up a bit.

I then started cleaning up the data to remove the jitters ect. I started this by adding human limits and foot stabilization. However the character i am using has not been properly mapped so i decided to try using an actor from Motionbuilder and setting up the marker points my self.

I did this by dragging the actor from the asset browser into my viewport, i then moved the actor into position over my kinect skeleton. I created a new marker set for the actor and used the kinect as reference. I draged the apropriate bones from the kinect over the marker set and oriented the joints.

This is the initial outcome.

As you can see there is much better motion using marker sets.

I now need to work out how to characterize the actor so i can clean up the data, and work out how to use that data in maya, which is my task for today.

Motion Capture: The initial stages.

Although i havent had much time this week with trying to get the third year dynamics completed and my dissertation proposal, i have managed to make a start on my motion capture project.

I first installed OPENNI software , NITE and the kinect primesense drivers along with the Brekel plugins for Motion Builder, this allowed me to begin my tracking.

Here are two videos to demonstrate the initial setup, however i had this running on my less powerful pc, which in order for me to achieve the highest frame rate had to turn off the point clouds and other visuals.

Even with these turned off the tracking was dodgy at best, running at a highest of 30 frames per second, whilst munching up all of my cpu.

However i have just switched over to my beasty computer, where it uses at most 2% of my cpu and runs at a much higher frame rate.

I am going to continue testing on my new system and i will upload my progress tonight.

Pulleys, Trains and snow.

As the deadline looms for the third year project i have manage to complete the dynamic shots that were required.

The main shot i have been working on was a pulley system, which would pull a key up from under the floor boards as a train wraps the web around the display unit. This however proved much harder than i initially thought.

My first task was to make the web wrap around the display unit as the train went around it.

This was my first test.

As you can see i need to turn the base plate into a passive colider.

Here are some tests to get the web to wrap around the display.

Once i had it wrapping around ok, i had to find away to get the key to pull up. My first thought when i was planning the simulation was that i could just have the web pull through the ring like a pulley as the train pulls one end. However this did not work, no matter what i did the web would not pull through the ring so i had to find a different way around this.

First i tried adding a constraint from the end of the web to the key and then animated the key, however this just made the web hang.

To do this i added a torus that had a passive collider and animated it to pull the web up, however i found that when moving at a high speed the passive collider would drop the web, which i could have compensated for by increasing the solver even higher (which would increase sim time).

I decided to change this passive collider to a nconstraint.

I know it goes a bit weird at the end but i have changed that, i don’t have a playblast of that atm though.

My next task with in this scene was to make the web infront of the train pull out, being influenced by the rocking horse, allowing the train to go around. This was also pretty hard as the rocking horse was in the wrong place to pull the web out as if it was real so i had to over come this. To do this i animated two point to surface constraints to make it look as if the horse pulls the web.

I had one more task on this project and that was to animate the snow blowing through the door when it is forced open. I already had the snow for this so i just added in some air fields and performed a make collide on the front sections of the house so the snow would only go through the door when open.

Its abit hard to see in this as the snow is grey in the viewport as is the background.

Im looking forward to see what my dynamics look like rendered as i have only been able to see them in the viewport.

Change of plans

I have been thinking alot about my negotiated brief and have decided to change my idea.

My original idea of creating an interactive game using the kinect, although it would be awesome to make, i think that it would not be possible within the timelimit for this brief, as i would have to learn alot of coding to make this plausable.

So i have changed my idea, i would like to create a series of motion capture tests using the kinect, I can also link this in to my animation technology brief, where i can do alot of research on motion capture, ways to use it and where its heading in the future, which in turn should help me develop my tests throughout my negotiated brief.

I have been looking at to get ideas on what i could create and also what uses the kinect has other than gaming.

Here is some stuff i found which could prove usefull.

This is a short using the kinect for all the animation, however it is a bit glithcy.

Markerless 3d motion tracking.

Facial Tracking.

Negotiated Brief / Animation Technology

Ok so our new brief is pretty much open to make anything we want to.

So here is my idea.

I would like to create a small game or series of tests using the kinect. I want to use it as a virtual reality tool where everything with in the enviroment is interactive so you could pick any thing up and move things around.

The way i want it to work is from a first person perspective, so when you move your hands infront of you in real time, the game would mimic your actions, i would also like to encorporate this with walking as well (so you move around by walking on the spot or running to go faster)

Looking around would be controlled by a tracker on the head making it completly interactive.

However i do not know how much of this will be possible and i will need to speak to Georg about how i can approach my ideas and how i could make it work.

As i said this may just need to be done in tests rather then trying to create a whole game.

We also have an animation technology brief, in which we have to write an essay and give a presentation on a piece of technology giving a brief history and then applying your research to estimate where it could be heading in the future. I am going to link this in with my negotiated brief, where i will be researching interaction within games and what the future holds for this field.

Dynamics Progress

Well i fel lbehind abit with blogging my work over the easter holidays, so i will compile it all into one post.

As you know i have been working on the dynamics within the film The Little Helper.

I will post my progress in the form of playblasts from the beginning of each shot to the completion of each shot.

My first task was to make an object bounce between two webs that are situated on a ship, then for the object to bounce into some toy soilders. I set the scene up using ncloth for the webs and constraining the webs to the ship, using a point to surface constraint. I made the falling object that is to interact with the webs a passive collider.

I started off trying to get the web to hang correctly from the initial pose.

The web stretches to much so i increased the webs stretch resistance.

Ok so now i wanted to stop the bounce, to do this i found a frame where the web sits at the bottom of its hang, and set the initial state of the ncloth to there.

This is sitting ok however the object seems to stick to the web, deforming the middle as it leaves its bounce. I added the same settings for the second web so they both sit nicely to begin with.

Working on the first web, i next tried to remove the deformity caused by the falling object.

I have managed to reduce the deformity on the first web a little bit. I then changed some of the ncloths atributes to try to further remove the deformity, in which i was sucessful, however the web now seems to move way to much.

After going out and looking at some spider webs i realised that their webs dont hang down as much as the ones i have created, mine almost being like a hammock rather than a web. I noticed that the webs were alot flatter in real life. To try achieve this i increased the ridgidty of the webs.

This is to ridgid and way to bouncy, i changes some more atributes however this hasnt made it much better.

I decided i would try a new direction with this as im not getting the results i wanted. I striped off the ncloth and all the dynamics and started from fresh. I decided i would constrain the web all the way around the outside, this way i can get a better bounce from the web.

I added a passive collider to the object to see how it would interact with the web.

Already this is looking much better than my first attempts.I then used the same settings for the second web and playblasted to see how it looked.

Next i need to move on to makeing the soilders fall over. I am going to set up a ridgid body simulation to do so. I had a few problems with this to start off with, as all the props within the scene were from reference files rather that actually in the scene i had to combine the meshs of the soilder and the gun, for the ridgid bodys to become active.

This is the steps i took to make the toy soilders fall, i wont explain each one as i was just changing settings and thier positions so that

I added in some animation with the catapult being used when the soilders fall into it.

However after i had done this jenny informed me that they had a new referencing system so i would have to add these dynamics to seperate shots.

The Title Scene.

My next scene to add dynamic to was the title scene. My objectives of this scene were to make the spider web move when sally walks over it, i also had to make the letters making up the title look as if they were stuck in the web.

My first part to tackle was adding dynamics to the web to make it look real, to do this i added an ncloth to the web and constrained all the sides with a transform constraint.

As you can see i need to turn the stretch resistance up as the web just sags towards the floor. I turned this down abit but obviously not enough.

I got the web to sit correctly, so then i added passive colliders on the feet of the spider and increased the stickyness so that you could see the web move under his feet.

I increased the collision strength to have move impact on the web when the spider walks over it.

My next task was to make the letters look as if they were stuck to the web, i achieved this by turning all the letters into ncloth, with the ridgidity turned up to 10, i then selected all the vertecies on the ncloth and point to surface constrained them to the webs ncloth.

I had to do a bit of shuffling around with the letters so that they would behave the way i wanted them to, this was because the web wasnt detailed enough so they swang around a little bit.

I didnt like the bounce the web had when the simulation started so i removed the gravity from the ncloth.

I then removed the little bits of twitching on the web.

Close up shots.

My next task was a series of close up shots of the spider.

I started with shot 2. First i added an ncloth to the web then constrained all the edges as i did before. I then added passive colliders onto the feet of the spider.

At first the web was to bouncy so i decided to remove the gravity as i did with the previous webs, this allows me to have better control over the web.

I then moved on to shot 5. I added the same settings to the shot however it did need tweeking.

I need more movement than this on the web, so i increased the collision strength on the feet, however it was slightly to high.

This seems to be a little better but still needs some adjustment to it.

This seems to be working much nicer now.

My next scene that i had to complete was shot 32.

Again i started off using the same settings then went through and tweeked it accordingly.

With Initial Settings.

I felt there was to much movement on the web, which gave it an underwater feel. I will try tune this out.

Initially i made this worse, i tryed to move the web around (up and down) to find the best place for it to sit, however i think i bought it up to high as the foot sweeps through the web causing it to deform.

I managed to get rid of the deformity, however the impact is still to much.

I turned down the stickyness of the front foot, i think its starting to look much better.

Jenny also wanted some shaking on the web when the spider starts shaking its legs, as i could not achieve this just using the legs as deformers, i decided to make three spheres which were passive colliders, and animate them on the left legs. This way i can have some more control over the web and increase the shaking how i want.

First i just put basic movements in to see how it would look.

I tried to tweak them but it caused them to deform the web.

I managed to get rid of the deformity.

A few little tweaks later…

My next tasks on this film are to make ncloth clothing for Frankie and make a pully system made from webs.

Stay tuned for updates…