Experimental Tasks
Two different tasks were used, a simple piloting task of level flight over the ocean and a more complex piloting task through the Grand Canyon. We used the first task to develop a method for decoding of neural states associated with response to a perturbation and the second task to investigate the generalizability of the method to a related but more complex situation. In both tasks the participant was given a first-person unobstructed view from the airplane (the view was as if from a camera in the front of the aircraft, see Figures 2A–D). The aircraft model simulated was an F22—Raptor using the X-plane flight simulator (Version 9.75, Laminar Research). The data for various flight parameters (elevator, aileron, rudder deflections, pitch, roll, yaw, heading, speed, dive rate, structural g-forces, latitude, longitude, altitude, etc.) and the control stick (NATA Technologies MRI and MEG compatible) deflections were collected at a mean sampling rate of 400 Hz using a UDP Matlab interface. The experimental conditions could be controlled via Matlab by using the UDP interface to give commands to the flight simulator.
Functional MRI
Our goal to develop a classifier of operator intention to undertake a rapid action to avoid a perturbation was to use a neuroimaging method with high temporal resolution, such as EEG or MEG. We used MEG in the current study, but in order to bolster our ability to localize MEG activity to intracortical sources, we also conducted an fMRI study of the same piloting tasks in order to establish seeds for conducting source localization analyses of MEG data. In the fMRI experiment participants underwent two sessions of the simple piloting task. Visual presentation of the flight simulation was projected by mirrors to a screen behind the head coil that could be viewed by the participant by a mirror mounted on the head coil. An fMRI compatible control stick (NATA technologies) was used by the right hand of the participant to control the elevator (back = pitch up; forward = pitch down) and aileron (roll left and right) deflections. Trigger timing of the fMRI scanning was directly read into one of the flight parameters of the flight simulator by means of a National Instruments Hi Speed USB NI USB-9162 BNC analog to digital converter.
MEG
In the MEG experiment participants underwent three sessions of the simple piloting task and one session of the complex piloting task. The first two sessions of the simple piloting task were used for training the decoding algorithm. The third session of the simple piloting task was used to evaluate the effectiveness of the trained algorithm in decoding neural states when participants perform the same task. As discussed previously, however, an effective classifier should be able to decode not only neural states on the same task that it has been trained on, but on more complex versions of the task that the classifier has not been trained on—that is, whether the classifier can achieve transfer. Accordingly, we also assessed the effectiveness of the classifier in decoding neural activity preceding detection and response to a perturbation in the complex piloting task. Visual presentation of the flight simulation was projected to a mirror to a screen above the participant’s head. An fMRI compatible control stick (NATA technologies) was used by the right hand of the participant to control the elevator (back = pitch up; forward = pitch down) and aileron (roll left and right) deflections. Trigger timing for the start of each trial and the start of the perturbation was registered by a photodiode placed on the screen. A small white square was constantly presented on the lower center part of the screen (out of the view of the participant) at the start of each trial and at the onset of the perturbation the small square turned black for 20 ms. The light intensity change was detected by the photo diode and written directly to one of the extra channels on the MEG.
https://www.frontiersin.org/articles/10.3389/fnhum.2016.00187/full