Headquartered in Melbourne, Australia, Zero Latency VR is the global leader in free-roam, arena-scale, multiplayer, virtual reality gaming. Through partnerships with licensees, Zero Latency VR has over 50 active venues in 23 countries around the world. NinjaOne has been heavily integrated into Zero Latency VR’s development, IT operations, and customer support team since 2018.
Zero Latency VR offers experiences in which up to eight participants freely roam a venue while immersed in virtual reality. A game master readies players for the game, oversees gameplay, and provides front-line assistance while in-game. Each participant wears a virtual reality headset attached to a wearable backpack computer.
All-in Zero Latency VR supports an infrastructure of over 3,400 backpack computers, servers, and virtual infrastructure to deliver over 1.7 million mind-blowing VR experiences worldwide.
Creating a stable infrastructure
Zero Latency VR’s technology is licensed to clients who are experts at drawing in a crowd, but who often have little to no technical background. “It’s very important that our solution is stable and easy for our partners to run – it needs to be as much of a turnkey solution as possible so our partner can focus on growth and ticket sales,” says Scott Human, Head of IT Operations at Zero Latency VR. “Our investors expect 35% growth year over year. Platform stability and customer experience are critical to our ability to deliver that growth.”
Prior to Ninja, the greatest driver of gameplay interruption was instability caused by Windows updates. Zero Latency VR’s IT team would spend 40+ hours per week testing patches, rolling back patches, installing patches, and making sure Windows updates didn’t break their software.
Zero Latency VR now uses Ninja to actively block all Windows patches from deploying to the backpack computers on which the VR headsets run. Ninja automatically rejects all Windows patches for VR-attached devices so that endpoints are all configured exactly the same. “Keeping our backpack computers in a constant state allows us to deliver a more reliable and stable solution to our partners and end users,” says Scott.
Ninja allows Zero Latency VR’s IT team to focus on projects that contribute to growth — like strategic improvements to their solution and delivering new products to the market — rather than on support and maintenance.
“The biggest benefit of Ninja for us is peace of mind – we just don’t have to worry about our fleet,” Scott says.
Innovating through agile deployment
When Zero Latency VR purchased Ninja, they had approximately twenty-five locations around the world and around 1,500 devices under management. “Prior to Ninja, whenever we wanted to update our software, support had to remote into each server and deploy the software update manually. It took us at least forty hours per deployment,” explains Zbigniew Nastalski, senior software engineer at Zero Latency VR.
“When it takes a full week of technician time to deploy software, it impacts your development and release cycle,” explains Zbigniew. Zero Latency VR’s release process led to long waits for bug fixes and new features, which didn’t align with their customer satisfaction or growth goals.
Switching to Ninja has allowed Zero Latency VR to fully automate this process. “Deploying updates of our software across all 3,400 devices now takes just a few clicks and is completed in just a few minutes,” explains Zbigniew. “With the release process being so streamlined now, we can schedule releases every week or two, which opens a ton of possibilities on the software development side,” he says. Whether it’s quickly testing updates in small batches or running diagnostic tests, the development team can update, test, and iterate very quickly and with minimal impact on end-users by using Ninja.
Delivering urgent updates
“I remember at one of our first on-site run-throughs with our technology, we had a patch go through that changed a registry flag on the backpack computers and ended up completely disabling the VR headsets,” says VR technology strategist Hunter Mayne. “While on site, I was able to write and deploy a PowerShell script through Ninja that fixed the registry flag so we could perform the demonstration.”
Any urgent fixes – from system configuration changes to software patches, to diagnostic tool deployment – are deployed via Ninja. “We’ve actually built our release and bugfix process around having Ninja,” explains Hunter. Scott continued, “We have a lean team across the board here at Zero Latency VR. We’ve been able to rely on Ninja as the engine for software and script deployment so our developers can focus on our core competencies and strategic business priorities.”
When a problem arises that the Zero Latency VR team cannot easily track down or replicate, they can quickly release a diagnostic update, wait for the issue to repeat itself, check the error logs, and remediate, often with a new version of their software going out within a few weeks of the bug being reported.
Ninja also acts as an automated monitoring and auditing tool for Zero Latency VR – checking device performance, monitoring for configuration changes, and verifying files. These background processes are never seen by customers or licensees, but are critical for the efficient and effective operation of their solution.
In-session remote support
Players can get really intense and heated, especially when playing something like an extreme zombie survival game. “When something does go wrong with a player’s gear or the game itself, we don’t want that intensity to dissipate,” says Zbigniew. When a game master calls support, the Zero Latency VR support team can take control of the game master’s endpoint to quickly identify and resolve the issue, so players can get back to their game. “Being able to see what players or the game master is seeing at the click of a button is critical,” Zbigniew explains. “Especially because we’re providing support to partners across many locations and languages.”
“We have an incredibly lean operation here,” says Scott Human. “All 3,400 devices we manage are served by an eight-person support team.” By automatic patch rejection and enforcing device configuration compliance, Ninja makes solving problems easier when they do arise. “Our customer support team doesn’t need to be deeply technical,” says Scott. “We can standardize remediation because we know the exact configuration and status of every machine. That makes our support team incredibly efficient.”
Ninja also makes it easy to quickly find a device and launch a remote assistance session when support is required. “Ninja’s TeamViewer integration works really well for us – TeamViewer’s excellent compression algorithm allows us to see what our users are seeing and provide support without crashing either the game master’s endpoint or the backpack VR sets,” says Zbigniew.
“Ultimately,“ says Scott, “we’re both a high-growth technology company and an entertainment company. We need to deliver an amazing customer experience in order to continue growing rapidly. With Ninja, we’ve built a process that allows us to innovate rapidly, respond to issues proactively, and deliver support to customers in a way that sets us apart in the industry.”
Customer Story
