InWorldz Blog

Where your Dreams are our Vision!News about the InWorldz Virtual World

Tech Blog


The InWorldz future initiative

Announcing the InWorldz virtual world future initiative

InWorldz is a tech company, and we are a company run by people with big hearts. We understand the empowerment that comes to many people when they can enter a virtual world and dance again when this has been stolen from them in their real lives. We want to help spread this technology in any way that we can so that more people can get their hands on it.

InWorldz is also a business. We are responsible for our employees, and we are responsible for maintaining a product that works to our customer’s satisfaction. We do our best to control the direction of the product and listen to customer feedback to provide a focused virtual world solution. We are the gatekeepers to what goes into and doesn’t go into InWorldz. We have to be agile.

As of March 2014, we currently handle more than twice as many 30 day active users than any other single opensim based grid, and greater than 10x more active users than the majority of opensim grids. Along with the monthly active users, InWorldz routinely sees peaks between 300 and 400 simultaneous users in-world.

These higher than average numbers have brought us knowledge and experience into the vast array of technical challenges that come with virtual worlds. We have utilized a combination of tried and true technologies along with new cutting edge software and distributed systems to make sure that InWorldz can handle the multi-pronged load that is inherent to virtual worlds, as well as make sure InWorldz is ready to scale out when we need it to.

For various reasons in 2009/2010, InWorldz forked OpenSim and brought development in-house. Therefore much of our work has had to stay private as many of our changes were massive and would not be easily ported back. We have on a few occasions hired security consultants to try to find holes in our protocols and in each instance where a bug was found, we asked these white hats to report the problems back to someone privately. We have also privately committed patches and bug reports to related projects like OMV, which provide the backbone data structures to SL based simulators.

While this kind of backchannel reporting and submission can help virtual worlds as a whole, we think we can do more. We want to see virtual worlds succeed and we want to be part of a massive push to get them there. With that in mind, this year we will begin to release and develop some standalone components under permissive open source licenses. The first of these components in development is something that many interactive businesses outside of virtual worlds can probably use, but that is being designed with virtual world use cases in mind. Given our experience combined with recent advances in distributed systems engineering, we believe we have a good perspective on the solutions to some problems that are begging to be solved.

Working together we can show the world how awesome virtual reality is, and in time, I hope to be able to share it with more friends and family than ever before.  


2014 progress report

Hello everyone Tranquillity here!

This year has seen some massive work completed on the InWorldz grid with a lot more to go. Since we are now into the second quarter of 2014 I’d like to list some of the things we’ve accomplished, some of the things we’re still working on, and some targets we’ve missed or that I am predicting we will miss. Towards the end of this post, I’ll also make announcements about other significant events in the pipe for 2014. [edited: I moved that announcement to its own post, you can read it here] You can see our original schedule here: InWorldz Roadmap for 2014 which will explain some of these projects in more detail.


The good stuff

Bot Support

First incase you missed it, we released a highly requested feature for role players. “Bot” or NPC support hit the grid with huge popularity, as well as a few problems along the way. The bot support is modeled after the AuroraSim bot API with significant enhancements to functionality for InWorldz. Many creators had fun testing and finding uses for the bots within their stores, RP environments, and just about everywhere else. Unfortunately the design was a bit too open and we had to pull the support to add some limits. But the good news is that InWorldz 0.8.14 is scheduled to hit the grid between Friday, April 18th and Saturday, April 19th and it reenables the support.

Release cycle change

We successfully switched to a 3 week grid release cycle. There have been a few bumps in the road, but it has significantly improved our releases and allowed us to better concentrate on just the features and changes that we know are ready for a roll out. Over time as the release process becomes more of second nature for the developers, we can already see the gains in productivity and quality will be apparent.

Project stratus hybrid asset system

InWorldz project Stratus hybrid asset system is active on the beta grid and is ready for roll out. We are waiting for an openstack feature to be deployed before we can proceed. The awesome folks at Rackspace were able to get a feature added for us to make lower latencies possible for our use case on their cloud. Once this feature is deployed, we will roll out Stratus which will make it easier for us to scale out while maintaining the performance you’ve become accustomed to on InWorldz.

Scenic region partner rezzing

The ability for someone to permanently rez objects on their partner’s scenic region has been deployed to the main grid

Region crossing and teleport enhancements

I have assigned myself to the region crossing changes and am currently in the middle of a complete rewrite of the legacy code that was handling connection management and neighbor region logic. The first set of these enhancements is due to hit the main grid on the next release on the 18th. This should lessen cases of “child agent update failed” during teleport as well as timeouts during region crossing handoffs. The next stage of the changes should hit the beta grid within 4 weeks and will help alleviate the vast majority of remaining teleport and crossing issues.


Delayed projects

Unfortunately, we’ve also missed our mark on a few projects. This is mainly due to changes in priorities and problems that came up while we were trying to complete our tasks.

InWorldz InShape open beta and release

This project is delayed until I finish up the teleport and crossing enhancements. I expect to resume work on this again in about 4 weeks.

Project Maestro deployment

We’re going to try to squeeze this in within the next few weeks.


Moving to a 3 week release cycle

The past months have seen some massive work being completed on the InWorldz grid. During this time, InWorldz developers have tried their best to keep up with the high level of feature requests and bug reports. Unfortunately our last major release had many regressions that didn’t get exposed in testing, and took InWorldz developers a couple of weeks to sort out. Due to these problems and the schedule slips they will undoubtedly cause, InWorldz will be moving to a new hard set release schedule.

From now on we will perform a development freeze every two weeks. This means that when we are considering features to add to InWorldz, no feature completed outside of this timeframe will be included in the upcoming release. We will also not fix long standing bugs during the development freeze. We will only concentrate on new regressions caused by previous fixes or features. New features and bug fixes from the two week development cycle will be tested on the beta grid in the beginning of the third week of the release cycle. After a few days of beta grid only testing, we will deploy this release to the sandboxes for final regression tests. This version will then go to the main grid during a roll out at the end of the third week.

During the development freeze, development may continue on other features or longstanding bug fixes, but these changes will not appear in the release branch until the testing phase of the next cycle. This will prevent us from trying to slip in just “one more requested feature/fix” before a rollout which can cause feature creep as well as unintended regressions.

Release Chart


Along with this new schedule we want to emphasize that it is extremely important for content developers to help us test during the third week of the schedule. We need your help on the beta grid as well as the sandboxes. Please do not wait for us to roll changes to the grid only to discover that they have caused problems with your content. We do our best to avoid content breakage whenever possible, but we’re all human and mistakes will be made.

You can sign up for the beta grid on your account page on the website. You can follow the instructions here to learn everything you need to know about participating in beta testing on the beta grid. When a release is deployed to the beta grid for testing, you will see release notes posted in the grid notices section of the InWorldz forum found here. You should keep an eye on this forum to know when a release is ready for testing.

We hope these changes will lead to less regressions making their way to the main grid as well as helping us stay on schedule. We thank you for your continued help in testing, for your patience, and look forward to many more grid improvements in the future.



InWorldz Roadmap for 2014

(This blog is soon to be converted from the InWorldz tech blog to the general InWorldz blog. As such, we’ve chosen to post this general grid news here)

2014 will bring some exciting changes to the InWorldz platform that will improve upon the performance, reliability and features of the virtual world that many of you call home. I’d like to take a bit of time to go over our current development work list and then dig a bit deeper into upcoming features for this year.

We’ll start with large projects currently being worked on or tested. They mostly deal with performance, management, and reliability.

Large projects with completion in the short term

Maestro – Grid Management

(code is 95% complete, beta testing starts within a week)
Mike Chase, Tranquillity Dexler

Maestro is the name of our new region and platform management backend. Our old backend named Zookeeper has done a very good job at allowing us to perform easy updates, configuration changes, and execute parallel commands upon the entire grid whenever we needed to. However, this backend was developed over time based on constantly changing requirements and really needed a complete overhaul.

Enter Maestro, a rewritten grid management system. Maestro will allow us to go well beyond the capabilities of Zookeeper and provide you with faster deployments, better region monitoring, and high availability. Maestro will allow us to collect and keep more detailed information on the performance of regions, and provide more powerful statistical and fault analysis. Developers will be able to easily and remotely look into the inner workings of a running region all the way down to threads and call stacks for powerful debugging. This means bugs that pop up only once in a blue moon, or only when there are 60+ real avatars on a loaded region will be easier to identify and fix.

Maestro has better task management, and access to a more intelligent deployment database. Region deployments will be more intelligent and fully automated to provide a quick response and “best choice” for placement on hardware when someone would like to make a new purchase. Maestro deployment software is aware of the average resource consumption for the given product and will use this data to assess options and provide top performance.

Better task management means that when someone schedules a region restart it will not conflict or override system events like automated rollout restarts or automated region backups. This will ensure better consistency and less problems that arise due to conflicting instructions given to a region at critical points.

With all these features, we have decided that region takedowns will still be a manual process as there are many reasons why someone may be late with a payment here or there. The InWorldz team doesn’t believe in automating and removing the human side from what can be a sensitive and upsetting event. However, temporary takedowns will be easier for support, so if there has been a payment mistake your region will be started back up more quickly.


InWorldz Viewer 2.x – Pushing to be the default download
McCabe Maxsted (Adric)

There are a few bugs remaining in the InWorldz v2 viewer (mainly crashers) that are preventing it from becoming the default download when someone creates a new account on InWorldz. We expect within the first quarter of 2014, those crashers will be resolved and the v2 viewer will be the default starting viewer for new avatars. This will better enable all new users to see and utilize mesh functionality.


Scenic region partner rezzing
(100% complete, in testing)
Jim Tarber

InWorldz scenic regions are an affordable and attractive choice for land owners that would like to add surrounding land to their existing regions. These regions are meant for light use land and waterways. As part of that goal, InWorldz restricted scenic regions so that they could not be split up and rented out which would cause the servers hosting them to be overloaded and cause poor performance to many other customers on the server. We also prevented anyone but the region owner from creating (rezzing) permanent objects on scenic land.

This restriction has been loosened a bit in our development branch to allow region owners and their partners to rez objects on the region. We hope this change can make them more enjoyable while preserving their original intent and high performance levels.


InWorldz “Project Stratus” hybrid asset system
(80% complete, in development)
Tranquillity Dexler

We have created a high performance addition to our asset storage systems that is designed around a hybrid solution utilizing Rackspace Cloud Files in conjunction with our current WHIP asset system to provide better fault tolerance, better recovery in case of failure and cheaper per-gigabyte storage for the company. This will allow InWorldz to continue to scale out in a cost effective manner and provide residents with better fault tolerance now and into the future.


Smaller, but no less significant changes coming in the short term:

  • Physics accelerated llCastRay is now available on the beta grid (Matt)
  • llSetKeyframedMotion in development (Matt)
  • Fixes and improvements for many physics functions including llMoveToTarget/region crossing and physical object grab and drag. (Balpien Hammerer)
  • Many fixes for vehicle functions including boat hulls seemingly scraping/bottoming out even in deep channels. (Balpien Hammerer)
  • Vehicle fixes in progress to make more SL vehicle scripts behave closer to how they work in SL. (Balpien Hammerer)

On that last bullet point, I’d like to make a comment. Please, if you notice a problem with vehicle support or anything else on InWorldz that isn’t behaving properly, take the time to check for and/or file a mantis report ( ). There were a few SL airplane scripts that weren’t behaving properly, but because everyone assumed we were working on vehicle fixes, no one reported specific bugs, and we had no idea there were any remaining problems with common scripts. After the issues were reported, they’ve been investigated and a few of them fixed already in under 24 hours.


Up and coming projects for 2014

Large planned projects – 

InWorldz InShape – open beta testing and launch
(Q1 2014)
Tranquillity Dexler

The holiday season, problems with google groups, and grid work that needed to be completed held up InShape ( ) over the holidays. I am pleased to say that I have my backlog just about caught up and will be resuming InShape testing in an open beta format sometime in Q1, 2014. We had a lot of positive feedback from people that really love this project and I even used it to run the ACS Relay for Life track live in 2013! Once the open beta has completed, and we have the rest of the bugs worked out, we will begin our weekly workout sessions and will invite residents to use the InShape technology to invent their own exercise machines and routines.


Project Thoosa Phase 2 & 3
(estimate Q2-3 2014)
Tranquillity Dexler, others as needed

Project Thoosa phase 2 will change the storage and loading mechanism for all region data. Utilizing high throughput services, InWorldz enhancements, and efficient data formats, we expect region restart times to drop by an order of magnitude. Phase 2 will also remove single points of failure from the InWorldz backend stack and utilize highly available storage systems to place and retrieve region data.

Project Thoosa phase 3 remains a hush hush project, but combined with phase 1, and 2, and a few other InWorldz projects, it will push the boundaries and offer services that have never before been seen in virtual worlds.


Full analysis and debugging for region crossing and teleport issues due to client side interaction
(estimate Q1 2014)
Jim Tarber

We won’t quit until we crack this one all the way open. With the vast majority of server to server issues fixed for teleports, avatar, and vehicle crossing, we have come to the point where we’re looking closely and debugging the interaction between the viewer and the servers during crossing failures. Jim has stepped up to go the full gamut on this one and we will not quit until we fully understand all the failure modes (including internet issues) and do our best to mitigate, mask, or fix crossing problems. We are pissed, and we are not going to take it anymore. Crossing issues that arise from viewer/server interactions will be beat to a pulp.


Website makeover and support work
(estimate Q3/4 2014)
Kingpin (Web team)
The InWorldz website will be revamped to provide more functionality, easier registration, and will come with a brand new look. The backend code will be reworked to provide more efficient development and updates.


Noteworthy projects in 2014 with deadlines not yet set:

  1. Avatar-as-a-prim
  2. Export permission implementation
  3. Inter-region communications enhancements and fixes
  4. Object name registry


Business Development projects

Just because you build it, doesn’t mean they’ll come
(All of 2014, but especially starting Q3)
The entire InWorldz team and YOU!

InWorldz will be devoting a large amount of resources to promotion this year. We feel that the world needs to know more about virtual worlds and virtual environments and how they can really add to the fun and quality of life. We want to attract new residents who may not have any idea about virtual worlds, and that may need some extra help before they finally “get it”. We want them to explore InWorldz and get a taste for compelling virtual world environments.

In conjunction with promotion, InWorldz will be focusing on retention through several formats. Starting with an overhaul of the registration process, getting information on what the user is interested in and guiding them directly to that point, whether it’s creating, socializing, or RP’ing. We’ll be working with active communities to provide a compelling area that targets the user’s end experience and gets them started in whatever they want to explore.

Along with InWorldz backed advertising campaigns and retention plans, we want people to read all about what types of things you’re working on in InWorldz, and what you enjoy doing here. Through your blogs and social media, you offer people an important window into the workings of a virtual world. You will help us to answer the whys and the hows for people that might be curious but don’t fully understand the concept.

InWorldz NEEDS you to write, to blog, to shoot video, and to share with the world what it means to be a resident here. We need you to show them the possibilities of the platform, and how it helps you to make some of your dreams come true. As a resident, your words and demonstration will be compelling. They will believe what you have to say more easily than they will believe us.


Thank you everyone. Have a great 2014 and see you InWorldz!


Project Thoosa Phase 1 Enters Beta

Around the beginning of Q2 2013, we began working on an internal project we named “Thoosa”. The etymology of the word touches on the goals of the project:

In Greek mythology, Thoosa or Thoösa (Θόωσα, Thoōsa) was a sea nymph associated with swiftness

The project has one primary goal and that is to make scene objects and their associated operations faster, more efficient, and more reliable.  Getting Thoosa to Phase 1 has entailed massive and sweeping changes to the core simulator architecture to accommodate more efficient storage, representation, and transfer of simulator objects and linksets. As our primary mission is to continue to improve the core of our product, Thoosa is a necessary step towards achieving a more seamless, massively scalable, and persistent virtual world.

This project will be deployed in Phases, and we have just completed Phase 1 and consider the code mature enough to enter beta testing. Phase 1 includes enhancements and optimizations in the following:

  • Taking objects to inventory will be more efficient on the server side.
  • Rezzing large objects should be noticeably snappier.
  • Teleporting to a new region especially when wearing many scripted attachments will be noticeably quicker and attachment scripts will start up almost instantly once you reach the destination.
  • Many avatars teleporting into a region will generate much less server side lag and will cause less of an impact on scripts currently executing.
  • Vehicle crossings, especially physical vehicle crossings for complex objects have demonstrated greater than an order of magnitude improvement in the time it takes for the object to move between regions.

The last point was one of our main goals to achieve on Project Thoosa. We recognized a need for physical vehicles to have very good region crossing behavior for our upcoming release of LSL vehicle physics functions. Below you can view a video that demonstrates a 27 times increase in the speed of a physical vehicle crossing between two separate regions in different processes:

As you can see from the video despite using many different torus shapes and including a ton of different scripts, Thoosa Phase 1 has made tremendous progress in creating a more seamless region crossing experience. This will enable InWorldz users to develop best in class physical planes, cars, boats, and more without having to worry about region edges being perilous due to the complexity of their vehicles.

We hope you find that these changes help you with your personal and business projects. We look forward to providing you with continued innovation in the virtual worlds space.


Vehicles and Physical Objects in PhysX

PhysX is here, presently in the beta grid and by the end of November it will be in the main grid, starting in the sandbox regions, oceans, straits, and a few selected private regions. The first phase roll-out makes it possible to make most objects physical. Physical objects fall; they collide with other objects, roll and bounce. Scripts will have access to the basic forces functions too (listed at the end of this article), and with those functions it is possible to make vehicle scripts.

But first let us focus primarily on the construction of physical objects, get a sense of the physics limits and then learn how to work through those limits.

Counting Physics Cost

We are used to making some extremely complex static objects in InWorldz. It is not unusual to see several hundred prim detailed constructs sitting on land or floating in the air. It is tempting to want to see these beautiful complex structures move but let us remember that there have always been limits to how complex physical objects can get.

Ye Olde Historie of Physiks

A long time ago in a grid far, far away, the limit for physical objects used to be 32 prims minus the number of seated avatars. This was a severe limit and yet physical vehicles managed to get built anyway. What often happened, though, is that complex vehicles ended up being powered using non-physics based movement scripts. The physical object prim limit was raised to 32 prims flat, and then later with the introduction of mesh some rather complex formulae were developed to try to determine the true physics costs of having physical objects (it is a bit mind numbing) but the number remained fixed at 32 prim equivalents.

In the New World

In InWorldz physics (using the PhysX engine), an object’s physics complexity is determined solely by the number of perfect spheres, boxes and convex hulls required to make up its physical representation — that number is used as the physics cost.

The current limit is 256. How this number relates to prims is a little complex but not unduly so.

All objects are represented by a mesh. Prims are just precompiled meshes which made it easy for people to build complex objects without having to deal with the intricacies of 3D modeling. A convex hull is a special mesh that has no concavities (no depressions or holes), and the primary reason convex hulls are important to physics is that they have mathematical properties that make it far easier to figure out if an object is colliding with another object (or land). The following diagrams help to explain what is going on.

The left diagram shows how an object with lots of concavities has to be represented with several convex hulls, six in this example. The diagram on the right shows how another object can be represented by a smaller set of convex hulls (three in this example) because the object has fewer concavities. This is one of several ways an object could be wrapped by convex hulls, and the point is to show that an object’s overall shape strongly influences how many convex hulls must be used to create a physical shape that comes close to mapping how the object looks to everyone.

Mapping an arbitrary object can be an arduous task. Here is an example of a bowl in mesh form. The mesh brings out the details for viewer rendering. To create a physics shape this mesh is remapped into convex hulls algorithmically. The diagram on the right shows how the physical representation of the bowl is approximated. Physics shapes need not be perfect fits. They tend to be imperfect fits because it can take a lot of time to compute the convex hulls. So, the fidelity is a trade off between time and resources needed to map the object.

Prims to Convex Hulls

When the basic prims are made physical, they are converted to convex hulls depending on their prim type and whether any transformations have been performed on them.

Boxes, prisms, cylinders and spheres are usually represented with 1 convex hull. However, when modified to have a hole, at least 4 convex hulls are needed to represent the prim (3 for prisms). A physical object made up of basic prims could well be made from at most 256 prims. However, add holes or path cuts to those prims and the maximum number of physical prims in an object drops to 64.


The basic torus requires 25 convex hulls to represent its physics shape. It has to be sliced up into many convex hulls to create a reasonable physical approximation. It gets worse when the torus is twisted, where the cost can rise to 79 convex hulls. Because a torus requires so many convex hulls, the number of torus prims becomes a major limiting factor for physical objects. A vehicle made from toruses is limited to 3 to 10 torus prims depending on whether they are twisted or not. Tubes and rings also have higher physics costs though not quite as high as torus prims. A basic tube requires 15 convex hulls and a ring requires 19 of them.


Sculpts To Convex Hulls

Physical sculpts are treated quite differently for now. All physical sculpts are wrapped with one convex hull no matter the number of holes or concavities. This poses some problems because a sculpt prim will behave quite differently when it is physical as compared to when it is static (non-physical). The following two photos illustrate the differences.


In the non-physics hull, the avatar can stand inside of it. This is the expected behavior of sculpts in InWorldz – they are surface accurate auto-generated meshes. Once made physical, the physics shape becomes a single convex hull and so it is not possible to stand inside of the sculpted cavity because the concavity is covered over by the convex hull. At some point in the development of physics, physical sculpts will have auto-generated convex hulls.

This can all get daunting trying to determine the maximums for a physical object, but fortunately, there is a way to find out the physics cost of prims by using the LSL function, llGetObjectDetails(), which fetches the physics cost of an object. I have provided a free physics explorer HUD that shows the physics cost of any object in front of the avatar’s camera. It is available in the main grid here: Ardhon en Ceredir – InMotion

Sometimes it is Too Much

Whenever an object is made physical (whether manually in the viewer or by a script), the simulator checks to see how many convex hulls are required to represent the object. If the number exceeds 256 convex hulls, an error pop-up message is sent to the viewer and then the object is made non-physical. In the example here, the object requires 406 convex hulls and so it reverts to nonphysical after the pop-up error is sent to the viewer

The builder faces several choices. He or she could simplify the object by eliminating some prims. Groups of prims could be replaced using sculpts, (eventually groups of prims can be replaced by mesh assemblies, although that has its own complexities), or selected prims could be made to have no physics shape at all.

Physics Shapes

When an object is made physical each of its prims is given a physics shape. A standard box and sphere (one with no transforms – no path cuts, slices, holes, twists, dimples) is represented by its pure physics shape. A transformed box or sphere and all other prims are represented by an approximation made up by one or more convex hulls.

You can, however, directly set the physics shape of each prim to be one of three shape types: a prim, a convex hull, or no shape at all. You do this in new viewers by editing the object and selecting the physics shape type (not available in current IW viewers). Scripts can also change the prim physics shape via llSetPrimitiveParams([PRIM_PHYSICS_SHAPE_TYPE, …]). I have written a script to help you set the physics shape of an object. It is describe at the end of this article.

The physics shape type of prim (PHYSICS_SHAPE_PRIM) means to use whichever set of internal shapes best fit the prim. For PhysX this can be a pure shape (box or sphere) or a set of convex hulls that wrap the prim to make an accurate physics shape.

The physics shape type of convex (PHYSICS_SHAPE_CONVEX) means to create one convex hull that wraps the prim. This does mean that any holes or concavities in the prim are filled in. But, this also reduces the physical complexity of the prim.

The physics shape type of none (PHYSICS_SHAPE_NONE) means that the prim is phantom. There is no physics shape backing the prim.

Planning Explicit Shapes

Deciding to adjust the overall physics shape of your build permits greater flexibility and allows for stunning detailed vehicles, but requires some planning on your part. A tall sail boat will have much detailing on its rigging but each of those lines and stays cost dozens of physic shapes, many unneeded. Even with simpler designs, use of fanciful shapes can well increase the physics cost. For example, this fantasy sailboat is made up of 19 prims using a hybrid of sculpts and prims. Its physics cost is 186 convex hulls. By eliminating the sail and sail beams physics shapes, then switching the twisted toroid forms to simple convex hulls, the same boat now has a physics cost of 30. This photo shows how it was done:

In your planning consider where passengers will be walking, where the major walls will be located. Make these surfaces prims. They can be visible ones if that works out in the design. They can be invisible if the viewable surfaces are made from sculpts. Simplify any fanciful shapes into convex hulls to reduce the physics cost. Or, if some of the detailing can be left phantom, set those prims to have no physics shape at all. Soon you can have your castle or mega-cruiser sailing through the skies or waters physically.

One final note, there is no such thing as a free lunch. Although the physics costs can be reduced by simplification, a 2000 prim object still has to be rendered by the viewers, and that process takes up a lot of rendering time. Too many moving prims can lag out viewers on slower computers. So though the regions will be OK, some people may experience problems nonetheless.

Setting Physics Shapes via Script

Eventually the new InWorldz viewers will be able to edit objects and set the physics shapes directly. That is a separate project in progress, but in the interim it is fairly straightforward to modify objects to set those properties.

The following script will do that (click on the script icon for a copy). The way it works is that you should edit each prim that you wish to modify its physics shape, and then set the description field of a prim to one of these values: physics_shape_prim, physics_shape_convex, or physics_shape_none. The values can be upper case or lower case.

The default is physics_shape_prim, so only the prims that you want to be different need the description changed. The script will use the description field shape set in the root prim as the default it applies to all the child prims. If you have an object in which you wish to set the physics shape type of all its prims to one type, just set the description field of the root prim to that shape type.

Once the description fields are filled in, drop in the following script and it will do the rest of the work. Once completed, you may remove the script as the physics shape types are properties that become part of the object.

Supported Physics Related LSL Functions

The following list of LSL physics related functions are presently supported in PhysX. Additional functions will be added in ongoing development.

Lvl Function
--- ------------------------------
1 llApplyImpulse
1 llApplyRotationalImpulse
1 llGetAccel
2 llGetCenterOfMass
1 llGetForce
1 llGetMass
1 llGetMassMKS
1 llGetOmega
2 llGetPhysicsMaterial
1 llGetStatus
1 llGetTorque
1 llGetVel
2 llGroundRepel
2 llLookAt
1 llMoveToTarget
1 llPushObject
2 llRotLookAt
1 llSetAngularVelocity
1 llSetBuoyancy
1 llSetForce
1 llSetForceAndTorque
2 llSetHoverHeight
2 llSetPhysicsMaterial
1 llSetStatus
1 llSetTorque
1 llSetVelocity
2 llStopHover
2 llStopLookAt
1 llStopMoveToTarget
1 llTargetOmega
1 llVolumeDetect


InWorldz PhysX physics goes beta

In April 2012, InWorldz began full time work on integrating the PhysX physics SDK into our virtual world platform. Over the course of the 4 – 5 months that followed, we have made tremendous progress and are finally ready to begin beta testing.

We are scheduling the first limited beta for August 25th, 2012 at 12:00 pm PDT. We ask that you only sign up for this first beta if you are a content developer and plan on utilizing PhysX for your projects (personal or commercial). If you would like to be part of this beta test, please send an email to with the full name of your InWorldz avatar and a sentence or two of how you are planning to use physics for your projects.

This first beta test will be limited in number of participants so that we can properly address issues that arise during testing. Don’t be disappointed if we cant fit you into the first test, there will be more.

This beta test is for Phase1 of the physics project which includes:

  • Much attention given to stability and avoiding physics related crashes. This is our number one reason for region downtime on ODE even without prim physics enabled.
  • A complete rewrite of avatar motion and avatar physics. Problems fixed include: not being able to walk up reasonably sized steps, randomly bent legs that require you to sit down and stand up to fix, and control delays between pressing controls and your avatar reacting.
  • Implementation of physical primitive objects and link sets with realistic looking collisions that respect object inertia and center of mass.
  • Support for LSL collision notifications for both physical and non-physical prims
  • Support for trigger shapes (llVolumeDetect)
  • Support for the application of forces and basic physics parameters. Build flying objects, rockets, as well as force based land, sea or air vehicles that can ride on the water, terrain or prim roads.
  • Support for complex physics machines. Use physics shapes to link parts together to build chains, rotating machines, gears, and other connected parts.
  • Change the force of Z axis gravity on your region for space, underwater, or other simulations

Phase1 of the project does not include:

  • Mesh physics implementation
  • Vehicle specific LSL parameters and functions (llSetVehicleType, llSetVehicleFloatParam, etc).

PhysX phase2 development will begin immediately following the deployment of phase1 to the main grid and will implement the vehicle functions with incremental releases along the way for the various predefined vehicle types.

This release puts InWorldz in a position to provide advanced and unique physics functionality to all of our residents. In the future we will implement functions to support more complex machines, physical attractors (think gravity guns), accurate mesh physics, springs, and other physics joints.

Thank you for your enthusiasm and support through this period of InWorldz development. We can’t wait to see what you’ll create.


Internet scale architecture and InWorldz

Whether it is dedicated servers, virtual servers/Virtual Machines, or a cloud computing host, there is no escaping that as a business grows on the internet they will need to utilize more and more resources toward supporting the core of their operations. These core services need to stay up and running at all costs, and need to be able to scale out depending on the demand. Any failures in these core services will cause total downtime.

Without proper support of core business processes a business on the internet is doomed to fail under high growth, or temporary high load scenarios. To application engineers this doesn’t become apparent until they scale out past the single server level, or have a write scaling issue that a traditional database cluster can’t help with.

With that said, I present a rough diagram of the InWorldz core architecture for Q4 2011 and into the future.

Important services to our customers include:

  • Asset Cluster:  (4 servers) Access to assets which represent all the textures, sounds, scripts and other low level data in world.
  • Inventory Cluster: (4 servers) Access to the structure of your inventory, the way your folders and items are laid out. The name of, and permissions on your items in inventory.
  • Region Data Cluster: (3 servers) When your region restarts, where all the “stuff” comes from to bring it back and where all the changes are stored.
  • Simulation: (lots of servers with Virtual Machine partitions) The pool of servers that powers the actual simulation on your region. These are the machines you actually connect with when you log in.

You’ll note there are 13 servers that power just the InWorldz, LLC core. The first cluster in the diagram holds in world assets. Our asset systems run custom written software that help to load balance and make sure assets are available at high speeds to our customers. These assets are stored in a format that is fast to write and easy to back up.

The inventory cluster will be running a NoSQL solution to support heavy reads and writes as well as a three server replication level. This means we can lose up to two servers out of the cluster simultaneously and not have data loss that would result in having to restore a backup. As inventory requirements grow, so does this cluster simply by adding servers and maintaining a data balance. This is the same type of technology used by larger internet sites such as FaceBook and Twitter and it will be employed at InWorldz to provide you with fast and redundant inventory access.

The region data cluster will utilize a high performance SAN and a sharded MySQL installation on a high availability virtual machine software platform.

All this is meaningless without good hardware and a good host.

All of these servers run on our own gigabit network with the core of the datacenter running at 10 gb/s. All of our servers have dual power supplies connected to fully redundant batteries and backup generators and we have a scalable connection to the internet.

Recently San Diego had a major power outage that affected 1.4 million of SDG&E’s customers. Our host Cari.Net is one of those customers. But we didn’t notice at all, and didnt have any hiccups in service. This is because the power infrastructure at Cari.Net is second to none with redundancy all the way to the supply, two huge generators that keep everything running, and a fuel delivery service to keep those generators pumping until the power comes back on. (for more details click here)

Big PowerWe are proud to have a partner working with us that feels the same way we do about quality.

Why does InWorldz need all this “stuff”?  Because we’re thinking big, not small. We want to power your dreams, not your catnaps. We’re looking into the future where we see a need for supporting vast concurrency on an interconnected grid with thousands of concurrent visitors. We’re looking into a future where virtual world technology is so important to some that it can not bear frequent or long lived downtime and outages.

Could we run the grid much cheaper? Yes we could. Would it sacrifice quality? Absolutely, and unacceptably so. As a company that is in this for the long haul we have no intention of trying to be the cheapest flavor of the month. We’re putting serious thought and investment into our network, our partners, and our people to ensure we can grow. We believe we run at a price point that is sustainable for growth and allows us to heavily invest in R&D and quality of service for the future.

There will be bumps in the road while we work towards this ideal, and it is no doubt a long road. There will be continuous learning and changing but in the end a persistent vision of a reliable, dependable world will be realized. At InWorldz, we don’t treat virtual worlds as games. We know they represent far more than that to the people that use them.


Non-Physical Land Racer on Phlox



The Phlox Virtual Machine

My intention for this blog is not only to display some of the work that InWorldz has done to our grid, but also to share some of what we’ve learned and hopefully get aspiring geeks, junior geeks, and certified geeks excited about virtual worlds and all the possibilities they bring. These articles will be long but I hope they can be of some value to you.

Along with the technical challenges we have to meet comes a lot of hard work, but also a lot of fun, and a lot of learning. This article is the beginning of a multi part series on the construction of the Phlox script engine.  I’m hoping that by presenting this material it can spark an interest in software technology and how it can be applied to something fun like a virtual world, to something serious like a domain specific language for bank transactions.

Normally with a series on script engines I would start from the bottom at the parser and lexer.  However, I think that starting at the end in this case will give some perspective on what the lower components are trying to accomplish. Therefore I’d like to introduce you to the component called the virtual machine. Let’s break down those two words:

Virtual: Existing in essence or effect though not in actual fact
Machine: A device having parts that perform or assist in performing any type of work

So by definition a virtual machine in the context of computing is a machine that performs the work of a physical computer but doesn’t actually exist. Much like a virtual world, a virtual machine has a lot in common with the real thing. It is expected that if you feed a virtual machine instructions it will perform operations in a similar manner to a real, silicon based, honest to goodness computer.

You may not know it, but you probably use a virtual machine somewhere in your everyday life. If you utilize InWorldz as your virtual world host, you have actually been using two different virtual machines to work and play, even before phlox was ever released.

At the first level, our software and OpenSim proper runs on top of one of the flavors of the CLR or Common Language Runtime. Originally developed by Microsoft and then also implemented as an open source product called Mono, this platform powers a great deal of software available today. The virtual machine behind both of these runtimes understands a language called CIL or Common Intermediate Language (more information here). The CIL bytecodes are the instructions that tell the machine how to run the OpenSim software.

Running below the InWorldz server software is actually another virtual machine that pretends to be real hardware to the operating system. This allows InWorldz to easily change server configurations, add and remove RAM and even processor cores. These virtual machines are out of the scope of this article but are related in many ways to the types of VMs we’ll talk about.

Outside the realm of InWorldz, virtual machines are also used in portable devices. If you happen to have an android powered phone in your pocket, you are also using  a virtual machine called Dalvik (really geeky stuff here) which efficiently runs all the apps you’ve decided to download this week.

Dalvik, Phlox, and the CLR all mirror their physical counterparts not only in function, but also in design. In your desktop computer at the lowest levels your CPU has support for storing data in registers and on something called the stack. Likewise, there are two types of virtual machine designs. The first is a stack machine, and the second is a register machine. Each of the two designs comes with trade offs in runtime performance, and implementation complexity. Registers.. Stacks.. what are these things and what do we need them for?

In a virtual machine, (as well as a real machine) registers and the stack are used to store temporary results that are needed while we complete a problem. Though both stack and register machines can do the same types of work, we’re going to concentrate on stack based machine examples, since Phlox is a stack machine.

Each problem given to a machine is reduced to it’s simplest form and then the machine solves and combines the results of many small problems into a solution for the whole thing.  A very simple example follows:

You need to design a machine and can add an unlimited amount of numbers together. Work with the sample 1 + 2 + 3 + 4 to come up with a solution.

The first thing we’ll notice is that there is no limit to the number of terms we might need to deal with. We cant just design a machine that can only add exactly four numbers. We need to deal with any amount. If we were to design this adding machine, we would have to add two of the numbers together at a time, and then put the result somewhere. Then we would need to add that result with the next term and store that result somewhere and so on until the addition problem was completely solved and we had a final answer.

[1 + 2] + 3 + 4
1 + 2 = (3)

[(3) + 3] + 4
3 + 3 = (6)

[(6) + 4]
6 + 4 = (10)


Using this method and a stack to complete the example problem above would look something like this:

Push the number 1 onto the stack    (stack: [1])
Push the number 2 onto the stack   (stack: [1, 2])
Pop the first two numbers off of the  stack and add them [1 + 2]. Push the result of the addition (3) onto the stack (stack: [3])
Push the next number 3 onto the stack (stack: [3, 3])
Pop the first two numbers off of the stack and add them  [(3) + 3]. Push the result of the addition (6) onto the stack (stack: [6])
Push the next number 4 onto the stack (stack: [6, 4])
Pop the first two numbers off of the stack and add them [(6) + 4]. Push the result of the addition (10) onto the stack (stack: [10])
The result of the entire operation remains at the top of the stack. (10)

As we can see, a simple adding stack machine would need instructions for pushing variables and constants onto the stack, and adding stack members together. Each instruction we added to the machine would be given a unique number. This number would allow the machine to read through the stream of instructions and perform the simple operations. When defining these numbers, which we call bytecodes we may also give them a name that makes sense to us as people. Instead of referring to the add instruction as instruction #1, we might simply call it “ADD”. Likewise for pushing an operand onto the stack, we wouldn’t want to say “use instruction #2” we would simply want to say “use PUSH”. This leads us to a representation of our source code where instructions can be listed out in the lowest level human readable form before actually being assembled into the raw numbers the virtual machine will use.

The bytecode for our simple adding machine example above could be:

push 1
push 2
push 3
push 4

The Phlox virtual machine has an instruction set very similar to this, but with a lot more operations to be able to deal with all the features of LSL and general purpose programming languages. When you input LSL source code into the text editor on InWorldz and press save, a lot of really neat things happen. First the lexer and parser makes sure that the source code is formed correctly and is grammatically correct. Then Phlox performs semantic checks to make sure you’ve not mixed up types, that you haven’t passed the wrong number of arguments to a function, and other similar mistakes. Once these tests pass, the compiler will compile your script into Phlox bytecode some of which we can see below:

Everything above the ^Z in the image is the LSL source code. Below it is the byte compiler output for the simple program that adds 4, 5 and 6 together. The form of the phlox bytecode is very similar to our simple adding machine in this case, but we can also see that Phlox has additional responsibilities over and above the adding machine.

  • .globals Tells the Phlox VM how much space we need for global variables. In this case there is one global variable, i, hence the line .globals 1
  • .statedef Defines all the states that are in this LSL script. If there were more states, there would be more .statedef lines
  • iconst Pushes an integer constant onto the top of the stack
  • iadd Adds the top two integers on the stack together
  • gstore Removes the top operand on the stack and stores it in globals memory (in this case, the variable i)
  • halt Tells the machine that this script is suspended until it receives another event
  • .evt Tells the Phlox VM about an event present in this script. In this case we only have one, which is state_entry and it is part of the default state. It takes no arguments (args=0) and there are no local variables defined inside of the event handler (locals=0)
  • ret Returns to the caller. When called from an event, the script is suspended until another event is triggered.

Every time you press save on a script, this intermediate form is generated before finally compiling down to the machine level bytecode.

InWorldz chose to design a virtual machine for many reasons, but the biggest one comes down to control. The first problem when you compile a script to native code as was being done before Phlox, you lose all scheduling control to the operating system. You can not choose how many instructions will be executed for each script and scripts can run forever in a loop consuming an entire operating system thread. This is bad because each of those threads also comes with a memory penalty that is equivalent to it’s stack size (usually around 1 MB) and too many threads trying to run at the same time will cause the processor to make tons of context switches and eat up valuable CPU time that you would really rather be using for the rest of the simulation than your avatar’s animation override.

If compiling to C# (as was done on InWorldz pre-phlox)  your best bet for ensuring a script does not run too long is to generate a yield return trampoline that emulates coroutines ( Essentially you’re adding in the C# yield return keyword at specified points in the program, say after every few lines of code. This solution would mostly work for allowing you to better control the scheduling of your scripts and would prevent a script from eating up an operating system thread, but it makes the design of the script engine very cumbersome. Every function now needs to return an enumerator type instead of its normal return type, all the compiled code needs to be able to deal with these and other changes, and you still haven’t solved all the requirements of an LSL environment.

A big part of LSL is that your scripts need to be able to have their state retrieved and saved no matter what code they’re currently running. Even if your AO is in a tight loop selecting various animations and recursively calling functions, this must not interfere with the ability to retrieve the script’s state for a save. At any time we need to be able to pause a script, grab the values of all the global as well as local variables in the script, and preserve the current call stack no matter how deep into a recursive call you are. When you compile down to C# and run the code through the CLR, your access to this information has been revoked. Your script could be JIT compiled and be running native code. Even if you could retrieve and fully save it, the native callstack from one machine’s scripts will not translate to another machine. You’ve lost control of very important information and an incomplete state save here will break the script. The only options here are to wait for the script to exit any events before taking a state save which could take an arbitrary amount of time, or insert some voodoo magic into the runtime you’re using, and hope that it’s Mono where you have the source and can add these changes. This also puts you in a position to have to continuously update your voodoo whenever Mono makes changes leading to extra burden on development.

The design of Phlox separates the runtime state and the compiled script code into separate independent pieces. Using a custom built virtual machine, we always have everything we need to do a complete state save available to us. Even if your script is in an infinite loop calling functions we can suspend it at any time and know that the current state of the script will be entirely preserved. That makes region crossings with active scripts smooth and straightforward. This design also enables bytecode sharing, whereby loading the same script 40 times in 40 separate objects only results in one copy of the script in memory.

The virtual machine also allows us to easily track the amount of memory being used by a script, and kill it if it goes over it’s quota (currently 32kb). Memory usage for each script is minimized by small bytecode since we don’t add features we don’t need. State saves are compact and quick.

Phlox and virtual machines have allowed us the freedom we needed to create an LSL environment that is fully compliant with your expectations. We can easily add features in the future and quickly fix bugs due to the architecture and can expand the runtime environment to make sure our residents can make their dreams come to life. All of this thanks to a machine that thinks it is, but isn’t actually a machine.

Older posts «

» Newer posts

Site Navigation

search engine optimization