Whether it is dedicated servers, virtual servers/Virtual Machines, or a cloud computing host, there is no escaping that as a business grows on the internet they will need to utilize more and more resources toward supporting the core of their operations. These core services need to stay up and running at all costs, and need to be able to scale out depending on the demand. Any failures in these core services will cause total downtime.
Without proper support of core business processes a business on the internet is doomed to fail under high growth, or temporary high load scenarios. To application engineers this doesn’t become apparent until they scale out past the single server level, or have a write scaling issue that a traditional database cluster can’t help with.
With that said, I present a rough diagram of the InWorldz core architecture for Q4 2011 and into the future.
Important services to our customers include:
- Asset Cluster: (4 servers) Access to assets which represent all the textures, sounds, scripts and other low level data in world.
- Inventory Cluster: (4 servers) Access to the structure of your inventory, the way your folders and items are laid out. The name of, and permissions on your items in inventory.
- Region Data Cluster: (3 servers) When your region restarts, where all the “stuff” comes from to bring it back and where all the changes are stored.
- Simulation: (lots of servers with Virtual Machine partitions) The pool of servers that powers the actual simulation on your region. These are the machines you actually connect with when you log in.
You’ll note there are 13 servers that power just the InWorldz, LLC core. The first cluster in the diagram holds in world assets. Our asset systems run custom written software that help to load balance and make sure assets are available at high speeds to our customers. These assets are stored in a format that is fast to write and easy to back up.
The inventory cluster will be running a NoSQL solution to support heavy reads and writes as well as a three server replication level. This means we can lose up to two servers out of the cluster simultaneously and not have data loss that would result in having to restore a backup. As inventory requirements grow, so does this cluster simply by adding servers and maintaining a data balance. This is the same type of technology used by larger internet sites such as FaceBook and Twitter and it will be employed at InWorldz to provide you with fast and redundant inventory access.
The region data cluster will utilize a high performance SAN and a sharded MySQL installation on a high availability virtual machine software platform.
All this is meaningless without good hardware and a good host.
All of these servers run on our own gigabit network with the core of the datacenter running at 10 gb/s. All of our servers have dual power supplies connected to fully redundant batteries and backup generators and we have a scalable connection to the internet.
Recently San Diego had a major power outage that affected 1.4 million of SDG&E’s customers. Our host Cari.Net is one of those customers. But we didn’t notice at all, and didnt have any hiccups in service. This is because the power infrastructure at Cari.Net is second to none with redundancy all the way to the supply, two huge generators that keep everything running, and a fuel delivery service to keep those generators pumping until the power comes back on. (for more details click here)
Why does InWorldz need all this “stuff”? Because we’re thinking big, not small. We want to power your dreams, not your catnaps. We’re looking into the future where we see a need for supporting vast concurrency on an interconnected grid with thousands of concurrent visitors. We’re looking into a future where virtual world technology is so important to some that it can not bear frequent or long lived downtime and outages.
Could we run the grid much cheaper? Yes we could. Would it sacrifice quality? Absolutely, and unacceptably so. As a company that is in this for the long haul we have no intention of trying to be the cheapest flavor of the month. We’re putting serious thought and investment into our network, our partners, and our people to ensure we can grow. We believe we run at a price point that is sustainable for growth and allows us to heavily invest in R&D and quality of service for the future.
There will be bumps in the road while we work towards this ideal, and it is no doubt a long road. There will be continuous learning and changing but in the end a persistent vision of a reliable, dependable world will be realized. At InWorldz, we don’t treat virtual worlds as games. We know they represent far more than that to the people that use them.