loader image

Analysis of the effectiveness of a virtual reality simulator

by Dec 7, 20210 comments

Analysis of the effectiveness of a virtual reality simulator

by Dec 7, 20210 comments


The first experience of virtual reality makes it clear that this is an impressive technology, but after the second it becomes clear that everything isn’t so simple.
Our team has come a long way from simple mobile VR for marketing to training several thousand employees with the development of a unique methodological program and testing.

Virtual reality can not only consolidate knowledge but also provide people with completely new information. Our numerous tests and methods have proven to us that VR is effective many times. But unfortunately, everything isn’t so clear. We had the opportunity not only to observe, but also to implement several software products in virtual reality that were completely useless, fortunately, due to this, we know a little more about the simulators and their effectiveness now


 Before solving the problem using virtual reality you need to ask and answer the questions:

  • Is there a problem at all?
  • Are you and your employees willing to take the time to solve this problem?
  • What problems will this simulator solve?

Also problem must have at least one of these points:

  • Carrying out work poses a danger to life
  • High cost of error
  • Impossibility of simulating a situation in real life

Regardless of the field of activity, all VR simulators can be divided into 2 types: training and testing

If, after answering these questions, you have nevertheless come to the conclusion that you want to try to implement virtual reality in your enterprise, do not rush to write a technical assignment and enter into an agreement, it will be much more profitable for all parties to cooperate if you do a little retraining and finally become convinced of the need for implementation.

Will it be cheaper and more effective than other teaching methods?

The last question is usually the most difficult to answer, and this is what my article will be devoted to, in which I will try to answer the question of how to conduct an analysis and understand whether you need an expensive simulator in virtual reality.

Where to begin?

For a primary analysis of the complexity, cost and effectiveness of creating a VR simulator, it’ll be enough to simulate a small part of the upcoming work and compare it with other training methods.
Let’s imagine that we need to teach people how to assemble a chair.

What is a chair made of?

  • 4 legs
  • Seat
  • Back
  • 4 fastening mechanisms

First, we need to understand whether such difficult and costly actions as creating a simulator will be more effective than more accessible training methods, discussing here only the point that we can’t train on the chair itself, as for example it would be a non-existent prototype or work with this chair would pose a great danger, otherwise the creation of such a simulator is initially ineffective in comparison with practice on a regular chair.

What methods are available for assembling chair? We know that we need to put all the pieces together by fastening them together, but we don’t know yet which method will be most effective, so we’ll start by determining how we can teach it:

a) Text

You need to take piece a and attach it to piece b e.t.c

b) Video 

We create animated 3D visualization

c) Augmented reality

We create an application that, when you point at an object, shows its name and animation of attaching

d) Virtual reality

We create an application in which the subject will be given the opportunity to go through the stage of attaching independently following the prompts.


Create a simple example using only a small part of all operations, this will help us save money and time by determining the effectiveness of a particular method, for example, screwing one leg to a seat using one fastening element.

We create 4 simple training systems for options a, b, c, d.

Next, we assemble four groups of 3–4 people for each simulator and let them go through training once.

Subjects should take part in the groups without theoretical and practical knowledge of performing this operation. If this isn’t possible, we use the “U-criterion” to equalize groups of subjects using a knowledge analysis test system.

The next step is to create the same training materials from all teaching methods. At this stage, we have to decide on the subject’s information to be analyzed:

  • chair assembly time
  • number of possible acceptable errors
  • correctness/incorrectness of actions convert to points, where correct = 1, incorrect or not done = 0

The learning process divide into the following stages:

  • Analyze the total time spent on training and repetition of the stage
  • Analyze the cost and time of creating the training material stage
  • Analyze the time and errors of the subjects and the points they scored

Calculate efficiency based on the received data

Create a simple test in which we must obtain the following information from the subject:

1. Describe the names of the necessary parts (1 point)
2. Describe the relationship and method of fastening (1 point)
3. Describe the order of fastening (1 point)

At these stages, we don’t have to follow the correct assembly of the chair; we are only interested in the teaching system and methods of the most effective order of assimilation. Simply put, the main thing is to understand whether the material being taught in educational materials a, b, c, d, e is clear to the subject.

For example, we receive the following information:

Group a:

Subject 1
Points: 2

Subject 2
Points: 1

Subject 3
Points: 1

Total: 4

Group b:

Subject 1
Points: 1

Subject 2
Points: 1

Subject 3
Points: 1

Total: 3

Group c:

Subject 1
Points: 2

Subject 2
Points: 3

Subject 3
Points: 1

Total: 6

Group d:

Subject 1
Points: 3

Subject 2
Points: 3

Subject 3
Points: 2

Total: 8

This is a simple example, for greater reliability we should use a large number of groups, and it’s advisable to pre-test people from the groups to analyze age, understanding of the material, etc. thereby ensuring the best homogeneity of subjects, further using more complex systems for analyzing the results.

Having understood which teaching method was the most effective, It’s needed to take into account the time and cost of creating this method and choosing the best option for your problem.


Virtual reality has come a long way, and we have seen from our own examples that its use not only effectively complements existing training systems, but also creates new ones. At the moment, there is no technology that provides immersion in content like virtual reality, and the development of body movement tracking and hand controllers allows the user to interact with the world around him, bringing it as close as possible to real conditions in recent years.

However, VR remains quite a complex and expensive technology despite all the existing libraries for developers and ready-made script designers, so before starting the complex and lengthy process of introducing this technology into your enterprise, you should make sure that it’ll be justified and more effective than traditional training models.