Phonics with Phonzy

Android/iOS interactive mobile app that helps children practice correct pronunciation of phonics sounds and words aloud with an entertaining 3D character I was the  lead 3D artist on this project where I was involved in all stages of 3D asset creation from modelling through to animation. I also assisted in integrating the animated 3D character into the game development software.  

  • Model
  • Blends
  • Rig
  • Anim

Keeping the poly counts low was of utmost importance because a heavy, higher resolution 3D model would greeatly affect performance, particularly as the target device for this app was android and apple mobile devices. At the same time, certain levels of detail had to be reached according to the designers and directors requirements.

2D designs of the 3D character.

2D designs of multiple concepts for the main 3D character. Designed by Declan Moore

Therefore when modelling this character, care was taken to ensure that any unnecessary polys and edge loops were removed or reduced. Where possible textures were used in replace of modelling certain details such as the inner ears and pupils. During the 3D modelling process the model was frequently imported into the game development software Unity3D with simple bone based animations and tested on mobile devices to make sure that the poly count was not too heavy and there were no performance issues.

3D character statistics wireframe textures technical statistics

3D wireframe renders of the main character model.

In the following video and image we see early development stages of modelling where overall size was changed and proportions and thicknesses were adjusted to match the designers 2D designs. UV unwrapping and texture mapping was carried out in parallel once the model reached a certain level of completion.

Progression of the 3D Phonzy character, showing the stages of modeling from early development

3D Phonzy character, showing stages of modeling from early development

This project was a great exercise with working withing a low poly budget while trying to achieve the best visual 3D result. It has solidified my understanding on the pipeline of building an animated 3D game character and bringing it into Unity 3D. Take a look at Phonzy here in 3D…

The following are some key components learned in game dev with Unity 3d:

  1. Work with one 3D object rather than multiple separate ones. Phonzy is made up of multiple body parts such as eyeballs, eyelids, teeth, tongue and hair. These were combined into one object which means that you only have to skin the rig to one object as well as work from a single texture. This makes it easier to manage in the 3D program as well as having a knock on good effect with performance as the application does not need to render multiple objects with multiple textures.
  2. Separate animations into individual FBX’s. Phonzy has 107 different animations to bring him to life! These are made up of the different phones and words, actions like his dance and idle animations. Alternatively all of these animations could be created in one combined FBX that is then seperated in Unity 3D, however this requires exact animation start and stop times and can be quite difficult to manage if chages are made to animation
  3. Use textures for details wherever possible. Poly counts were reduced on Phonzy by creating details such as the inner ears and pupils. This meant that instead of having to model additional areas of detail that requires more polys, 2D textures would be used instead. This is certainly the case for more realistic 3D models that make use of high res Normal Maps on low poly characters that immensely increases detail
  4. Model eyeballs perfectly spherical. This means that when you add the rig to control eye movement a rotating skin bone central to the eyeball will create the action of eye movement. Similarly the eyelids can be modeled based on this perfect spherical shape and can rotate on this similar center point that will create blinking action

This purpose of this app is to assist in the education of children on correct word pronunciation and related phonic sounds. Therefore the lip syncing of this 3D character was of huge priority and this is where the majority of time was spent. The director of the app has great expertise in speech technologies and lipreading science and we worked closely together in the facial animation of Phonzy to ensure correct mouth shapes, tongue movements and placements and head actions. From the outline of the project it was agreed that seven sets of six words would be animated for the first version of Phonics with Phonzy. Each word also had a phonetic sound associated with the word to assist with pronunciation so as an example the activity for the work “book” has an accompanying “buh” phonic . In total a count of 7x6x2 which amounts to 84 individual phonic/word animations were needed.

 sets of phonics and words used in the 3D app Phonics with Phozy

Sets of phonics and words. UI by Declan Moore

It was then evident that efficient control was needed for facial movement and lip syncing animation. The version of Unity 3D at the time of the development of this app allowed for animated blendshapes and therefore it was decided to use blendshapes for facial animation as opposed to a facial rig system.  Based on the phones and words of the app we put together a list of multiple mouth shapes that could be used for a range of different scenarios for example the same pressed lips of an “mmm” sound can be used for “buh” or “puh” sounds.

modeled 3d charcter blendshapes

Total of blendshapes modeled for the 3D Phonzy character

A facial rig UI was built within the 3D program that connect to these corresponding blendshapes so animation could be achieved much faster and is also a cleaner way of controlling the character. Controls were also made for the tongue to allow for rolling sounds and certain phonetics. Teeth controls were similarly put in place if the placement of teeth needed to adjusted.

Facial rig UI within the 3D program showing different sliders that connect to corresponding blendshapes

A custom biped rig was built for the main body that allowed for standard character controls such as IK/FK arms and legs, FK spine and finger/foot controller attributes. On top of this some additional non standard rig parts were built such as the tail and eyeball/eyelid parts.  When importing 3D characters into Unity 3D  you do not need any rigging elements or controllers from the 3D software. All that’s required is the character mesh (in this case is only one object), the animated blendshapes node associated with it and the bones that the mesh is skinned to.

From initiation of the rigging process it was decided that there would be two systems of bones that would work together in order to output to Unity 3D. One bone system was connected to the control rig and meant that all the bones did not form a hierarchical system, instead it was broken up in order for certain rig parts to work. The second system had no control rig and was constrained directly to the corresponding bones of the control rig mentioned previously. This second system followed a hierarchical pattern which stemmed from the hip bone. These constrained bones are then connected to the character mesh through skinning and they will inevitably be output for Unity 3D integration.

unity 3D bone hierarchy 3d character rig

The hierarchy of bones of Phonzy in Unity 3D

Because there were a multitude of many different animations and it was a lengthy process to isolate the required components to output to Unity 3D an automated process to do this work was scripted which meant that exporting an animation FBX only took 5 seconds as opposed to 5 minutes. This was a huge time saver in the Unity 3D integration process and it also removed any human error in doing this procedure manually. Some additional features of this character’s rig are the eye movements and blinks. Keeping character eyes spherical  is very important, particularly for 3D game characters. It makes rigging eyes much easier and the rotational movement of the eye is much smoother if a single skin bone is placed at the center of the spherical eye and skinned accordingly. Similarly if the eyeballs are spherical so too are the eyelids and they can also benefit of smother rotations and makes rigging easier. Here is a look at how the geometry around the eyes move and rotate.

As mentioned previously the creation of blendshapes to ensure that the character could meet all word and phonic pronunciation was the most lengthy process in this project. There was the benefit however that much of the ground work for animation was laid out because of this which made the animation stages fluid and quick. A singular file was made in the 3D program which contained  all 84 phones and words that were spaced out incrementally that linked with a spreadsheet that mapped out frame numbers with phones and words. This meant animation could be cut and pasted into individual animation files and animation curves could be adjusted to meet the directors requirements.

This huge workload was approached in group of animation whereby the first set of 6 phones and 6 words were animated to full completion. When all 12 animations of the set were complete the FBX for each of these animations were zipped and sent to the director who then viewed the animations using an FBX viewer where the fully animated and textured character could be reviewed for any corrections or additional requests. This was all documented using shared Google documents which was a fantastic resource to record changes and get instant feedback.

The following clip shows Phonzy pronunciation the word Umbrella

The next few clips show some reaction animations where Phonzy interacts with the user after a level has been complete or if the user has made a funny noise.

Phonics with Phonzy will be release for Android and iOS device shortly. Please checkout MeowTek for further information and announcements of the launch

Phonzy_Full_ThumbsUp_Recommendation

Darragh is a talented 3D artist who delivers excellent results on all aspects such as animating, modeling and rigging. Darragh was very professional and worked really well with the developers and designers which really helped deliver a quality product. Most importantly I really enjoyed working with Darragh and hope to again soon. 

Patricia Scanlon

CEO, MeowTek

This is a unique website which will require a more modern browser to work!

Please upgrade today!