Augmented reality and virtual reality in our daily life

ABSTRACT


INTRODUCTION
The fundamental value added by Augmented Reality is that, it brings components of the digital world into a person's perception of the real world. Using AR, we can augment any object into the real world. There are many existing applications that use AR and many more that are still trying to implement their application using AR in their respective fields. The first application that was developed using AR in 1990's was used for pilot training [1] and since then its applicability has been complemented with a variety of fields. The significance of AR is observed when they are used for those training purposes that are generally difficult to do in real life, such as pilot training, medical diagnosis, medical experiments, army training, automobile industries etc. Augmented Reality in conjunction with technologies like IoT, virtual reality has evolved to an extent that makes artificial intelligence sound like a task that can be easily accomplished. This paper explains a model that saves the time taken to perform preliminary examinations in medical diagnosis.
The preliminary process of examination is eliminated by digitally transporting the patient's details to the doctor and hence, the doctor can evaluate them when he is available. The doctor can further suggest the required tests and diagnosis for the respective patient. AR could be used for medical education through which the student interaction in classes could be increased and there could be more of practical discussions in the class. This will trigger the interest of the students. Virtual reality takes the person to that place and makes us to learn new things in the environment that is needed for the learning process. In Virtual Reality, a whole new environment is created according to the demand. On the other hand, in Augmented Reality, images are super-imposed into the real world than creating a completely new environment. Augmented Reality supplements reality, rather than completely replacing it. The virtual objects display information that the user cannot directly detect with his own senses. AR not only increases our learning curve but also improves our brain's productivity. It changes the way one thinks, works and how one learns. Instead of having just one angle of any concept, we get multiple angles of the same concept.
Sergey et al, in [2] which discusses about interactive environment for education using AR and 3D visualization. Bouras in [3] reviews the importance of virtual environment in e-learning. Charvi Agarwal et al in [4], confers about recent practices of AR in daily life. [5][6][7] examines the e-learning and e-learning based improvement of skill set of the user. [8] Explains about the applications of AR.
The following research papers [2] and [8] were referred to for AR technology which discuses about interactive environment for education using AR and 3D visualization. [9] Discusses about kinetic fusion technology and the implementation of kinetic camera. [10] Discusses about advantages and limitations of AR in educational environment. [4] Discusses about the recent practices of AR in daily life. [3] Discusses about the importance of virtual environment in e-learning. [11] Discusses about structural equation and modeling approach for explaining the enhancement in learning outcomes through desktop VR. [12] Discusses about quantitative analysis of education through social learning outcomes.

PROPOSED ALGORITHM 2.1. Augmented reality and applications
In order to evaluate where virtual objects are to be blended in real environment, the following approaches are considered. Marker based approach in a marker-based approach [13], the images and the corresponding image descriptors to be recognized are provided beforehand. The markers that are used can be QR Codes or black and white square mirrors. Marker-less Approach, in a marker-less approach, an AR application recognizes things there were not directly provided to the application beforehand. This scenario is much more difficult to implement because the recognition algorithm running in your AR application has to identify patterns, colors or some other features that may exist in camera frames.
The advancement of technology in the field of medicine has grown to an extent that makes diagnosis very simple and efficient. Doctors can use augmented reality as a visualization technique and training aid for surgery [14]. The input data of the patient can be either images, or metadata about the heart beat and so on which will be transported to the doctor, along with the patient's 3D hologram. Thus, by using AR in this field, it gives the doctor a complete vision like an X-Ray of the patient. The goal is to make sure the diagnosis process is accelerated.

Virtual reailty and its applications
Virtual reality based two e-learning systems are discussed in the following section. The first is the elearning based virtual car driving system (VIRECAR) and the second is the e-learning based virtual cricket training system. The car driving system provides road safety measures which could help the learners to avoid accidents. The other system will help the players to learn more by not losing their energy and train themselves before appearing for a real match.

RESEARCH METHOD
A system that aims to simplify the doctor/patient preliminary examinations shown in Figure 1 is the system that is newly proposed in this paper. The idea is to digitally transport the patient to doctor's room as a simple simplified 3D object, with all necessary details to facilitate the findings. Also, without just transporting a digital object the system will impart interactivity to these objects where in, at the doctor's side, it gets rendered as augmented reality models superimposed in the environment. Thus, simulating the patient's appointment.
Data Collection Module, The first module of the system comprises of collection of patient data. This is done at the clinical premise. Patient data includes preliminary medical examination data like, temperature, breathing level, blood pressure. The collection and gathering shall be conducted by an assistant who uses the respective devices to capture these data. The image of the patient is then captured using Kinect Cameras which develops a 3D image of the patient using appropriate depths and coloring.

3D Model Building
The videos are to be recorded, featuring the patient with the help of Kinect Cameras. These are special cameras with sensors to detect the depth of the patient dynamically. Images and videos are captured 207 via Depth cameras and they appear to be as 2D images. These images when fed into Kinect Fusion, will churn out 3D modelling of the scene captured along with objects.
The conversion of 2D images can be performed by Kinect API's. Although the modelling of the 2D image into 3D models can be done in real time using Kinect Fusion [14], such an approach is traded for more accurate and detailed models by processing it separately. Though the process takes a lot of time, the needed quality of the 3D images that are rendered is achieved. The images captured as photographs are then overlayed onto the 3D objects to improve the originality of the objects. The same process is applied to any particular part of body, under examination.

− Exclusion
The subject of interest at the hand is only the patient and his vitals, any other objects present in the surrounding can be ignored. Thus, removal of the other objects is done in this component of the system. Selective capturing of images is not possible using Kinect Cameras, hence the approach used is to remove other unnecessary objects from the scene. This additional processing can be achieved through any image/graphic processing libraries like OpenCV, OpenGL and so on. The sequence of the steps would be: a) Detect unnecessary surfaces, b) Select them iteratively, c) Remove. The approach may vary for each patient.

− Order of Processing
It is ideal to process the video in a particular order. 3D models are constructed first and then the background objects are removed from the video. However, this reversal may not work due to the presence of the objects in the surrounding area vital for depth sensing.

− Medical data correlator
The patient's data includes his/her appearance, in most cases patient come with complaints and no particulars. In all of the medical check-up, the complaint of the particular patient forms the core of the data collector. The complaint as narrated by the patient and recorded by the medical assistant is the vital information to any decision; general notes/ observations about mental state of the patient are also included. The data collected includes vital signs and basic checkups/ examination.
Technologies and its definitions the intention of the proposed system is to use some of the existing software technologies, to make the preliminary diagnosis process effortless and uncomplicated. Some of the technologies that are used in the system are noticeably defined here. This section is in reference to the readers who do not have a clear idea about the definitions of certain technologies. Kinect Fusion [14] is a new and widely-available commodity sensor platform that incorporates a structured light based depth sensor. Real-time infrastructure-free tracking of a handheld camera whilst simultaneously mapping the physical scene in high-detail promises new possibilities for augmented and mixed reality applications. In [14], a detailed method with analysis of what may be the first system that permits real-time, dense volumetric reconstruction of objects/things using a handheld Kinect depth sensor. Users can simply pick up and move a Kinect device to generate a continuously updating, smooth, fully fused 3D surface reconstruction.

− Unity 3D Kit
Unity can be used to primarily create three dimensional and two dimensional games for computers. It can be defined as a cross-platform game engine, to create simulations. Unity can be used together with AR to create gaming applications that superimpose objects into the real world. One of the examples of a game using AR, is the Pokemon Go. In order to build the application, we utilise the SDK (Software Development Kit) of Unity, called Vuforia. It uses Computer Vision technology to identify objects, registers their position. And orient them. Vuforia is a free technology and is an ideal solution that serves the purpose.

− HoloLens
As one of the first AR headsets available today on the market, the Microsoft HoloLens is bound to have a profound impact on the development of AR applications [15]. The HoloLens is a head-mounted display unit connected to an adjustable, cushioned inner headband, which can tilt HoloLens up and down, as well as forward and backward.

Smart education System for deaf-mute children
A new era of technology (EduAR) comprises of augmented reality with Holograms that can be deployed in the field of education. AR adds to the reality that is ordinarily seen rather than replacing the reality. The ultimate goal of EduAR is to create a convenient and natural immersion such that, it becomes extremely easy to understand and interpret what we learn. Compared to the previous years, education today has improved and expanded to an extent that it becomes hard to follow and keep track of everything that is been taught. This is where EduAR comes of help. One of the unique features of EduAR is the ability to ease the learning of deaf-mute students. Using their vision and EduAR, these students can understand concepts as easily as any other student.
Consider a scenario of a biology class. Using EduAR, holographic images of the brain are used to explain the minute and intricate parts, which in reality is difficult for the teachers to describe just based on textbooks. As a result, EduAR helps the student understand how a brain works visually, thereby helping them to perform better during operations. It also enables medical students to acquire knowledge and understanding about the human body by means of interaction within a virtual environment in which no patients are at risk. This in further, can be extended to different streams in a way that is usually difficult for a student to interpret. Teaching and learning has to be much more than a simple process of knowledge transfer by memorizing. EduAR improves the student's learning experience thereby, developing the interests of an individual. Many labs that have 3D models of brains or hearts, do not really give much information on the deeper layers of the brain which will be one the major advantages of using EduAR. Also, with EduAR dynamic labeling will be an asset. Teachers can modify or add labels of parts as and when required rather than buying a new 3D model every time. Thus, EduAR focuses on transforming the overall education experience.

E-learning based virtual car driving system
Simulations of real-world scenarios can be achieved with a high value of realism using the concept of virtual reality. This characteristic of virtual reality environments gives an advantage towards e-learning where the solution to real-world problems can be learned virtually. Gesture recognition is another technology that has seen a steady increase in popularity as a method of human-computer interaction. Specific gestures can be translated to perform specific tasks. With hand gesture detection devices like Leap Motion, gesture detection to communicate with an environment has become simplistic, and usable. Combining such technologies with the learning process provides lucrative methodologies, which can be leveraged to simplify the process of learning. After implementing the VIRECAR virtual simulator system, it was analyzed with the feedback from a sample of 20 licensed drivers and 20 new learners in driving. E-learning process in VIRECAR as shown in Figure 2. The learning experience obtained from VIRECAR is explained in three stages in this paper: First Stage is considered as the Pre-Class Content in e-learning and Second Stage is considered as the In-Class Content and the Third Stage is considered as the Post Class Content, where these three stages of e-learning are already explained in [5], [6]. In the first stage -Basics of Driving, the user is required to understand the road and safety regulations prescribed by the Road Transport Office and are also required to understand the basic functionalities of the car. Towards the completion of this phase, the learners are required to undertake a test evaluating their competency in the basics of driving a car.
In the second stage -Learning through VIRECAR, VIRECAR is used to understand the working of a car and getting used to the driving environment. At this stage, the Concept of an instructor is introduced. The instructor at this stage is present at regular intervals evaluating the learning process undertaken by the learner and feedbacks are provided as and when necessary.
The third stage -Actual Driving, which can also be described as a post-class exercise is to drive an actual car based on the learning's obtained from VIRECAR. In this phase, the instructor is present full-time which allows him to give a feedback. The learner at this stage is expected to drive a car with the experience and feedback obtained from the second stage and the knowledge of the rules and regulations obtained from the first stage.
The above three steps occurring for the first time constitute the first iteration of the learning process. In the forthcoming iterations, the learner is not required to go through the first stage again since, the requirement to proceed to the second stage is to master the concepts in the first stage. Thus, here, a concept of back-propagation is used.
Based on the feedback obtained from the third stage of the VIRECAR e-learning process, the learner might have to gain more experience from the virtual environment before gaining that experience in the realworld. Essentially, the learner is expected to experience each aspect of driving a car including changing gears, using the pedals for acceleration, brake, and clutch, to steer the wheel and so on. This allows for a safe environment for the learner to effectively experience the learning process described in this paper aimed at learning, driving a car properly.

RESULTS AND DISCUSSION
To analyze the VIRECAR simulator system that is described in this paper, the system was tested with a sample of 12 licensed drivers and thus the efficiency of the VIRECAR was evaluated by their feedback. They were introduced to the whole system as described in Figure 2. Furthermore, they were asked to try the simulator to answer certain questions that were deemed vital, towards evaluating VIRECAR.Some of the following questions that were put forward: − How close to reality was it? − How helpful was it in understanding the driving scenario? − How useful was the system in learning rules and regulations? − If you had not driven an actual car, would you prefer to learn using VIRECAR? − Do you think you will be able to rectify any mistakes through VIRECAR?  Table 1 shows a consolidated data that was obtained from the testers of VIRECAR. The drivers gave an average score of 7.833 and 8.167 to the questions regarding driving scenario and understanding the rules and regulations as shown in Table 2. This gives us the view that the phase one of VIRECAR must be made more stringent and some of the elements from phase one must also be implemented in phase two to provide a sense of continuity. This paper discusses about the use of virtual reality in the e-learning based car driving system. Similar such mobile based learning system could be developed with the learning platform and frameworks that were discussed in [16]. This paper also discusses the use of augmented reality in the field of medicine. This will help the patients to get a second opinion from the doctors who will be capable of detecting the abnormalities in the patient, from their experience, even when the doctors are not in the same geographical location. Similar such video-based abnormal detection is discussed in [17]. These two systems will be very much useful for the end-users as they are replacing the original working models and giving the same experience to the end users. In the case of the e-learning based car driving system, there is no fuel wastage, no added traffic due to the novice drivers, and no need to worry about pollution too. Now days, there are few research happening for replacing the fuel and taking a chargeable vehicle instead of that [18]. Such problems are all solved as the driving system developed by the authors is fully based on virtual reality.