ABSTRACT

Virtual reality (VR) is a technology that can teach surgeons new procedures and can determine their level of competence before they operate on patients. An overview of the current state of development for the "Karlsruhe Endoscopic Surgery Trainer", a VR technology based training system for laparoscopic surgery, is presented. Special attention is addressed to geometric modelling techniques for graphical realtime performance and elastodynamically deformable tissue models.

Keywords: simulation, tissue modelling, deformable tissue models, VR-based surgery training

INTRODUCTION

Minimally invasive surgery (MIS) has been established among surgeons as an elective technique in a number of general surgery interventions. Endoscopic surgery of the abdomen, like gall-bladder and appendix removal, have now become commonly performed surgical operations. However, beneficial to the patients, these techniques require intensive training of the surgeons, practicing skills like 3D-orientation, hand-eye coordination and instrument handling. Surgical education has traditionally depended on the apprentice-mentor relationship and is the basis of residency training programs. Physical patient models like the pelvi-trainer lack realistic anatomical features. The concept of virtual reality (VR) surgical simulators is attracting considerable attention. Progress in computing performance allows the development of training systems for endoscopic surgery based on VR technology [3].

As a basis for our R&D activities, we use the 3D-graphical simulation program KISMET [1] (Kinematic Simulation, Monitoring and Off-Line Programming Environment for Telerobotics), which has been under development at Forschungszentrum Karlsruhe (FZK) for a number of applications. Because of its high quality realtime graphics capabilities and additional features like geometrical and kinematical modelling, multibody-dynamics, and its database concept allowing for multiple detail-levels, it was found to be an ideal platform for computer aided surgical simulation. The software has been improved significantly during the past years to meet the specific requirements of physical model based minimally invasive surgery simulation. Using the advanced capabilities of high-performance graphical workstations combined with state-of-the-art simulation techniques, it is possible to generate virtual endoscopic views of surgical scenarios with high realism.

The "Karlsruhe Endoscopic Trainer"

The MIS trainer produces a synthetic mono or stereo image of the view, which is in reality provided by the endoscopic camera [Fig. 4]. As a surgeon-computer interface, a "Mechano/Electrical Box" was developed as an artificial cavity together with the correct instrument set so that the interface normally presented to the surgeon is maintained using a physical simulation. This set-up [Figs. 1, 2, 3] enables following practical exercises:

• coordination of different instruments
• handling of an endoscope mock-up with its corresponding synthetic camera view
• teamwork of surgeon, assistant and cameraman
• simulation of new instrument designs and manipulator control devices
• measurement of handling speeds in a simulated environment

The design concept of the Karlsruhe Virtual Endoscopic Surgery Trainer takes into account the kinematics of conventional endoscopic handling with four degrees of freedom. Furthermore it allows for future extension of the training interface with dexterous instruments with 6 or more joints, i.e. kinematically redundant mechanisms. We have developed simulation techniques which allow the modelling of "virtual tissue" based on a data-model which reflects the physical characteristics of like mass, stiffness and damping of real living tissue. A collision test algorithm detects contact between surgical instruments and the manipulated virtual objects. As a by-product, contact forces between the tissue and the instrument end-effector are calculated which can be used in future surgeon interfaces to drive force-reflecting input devices. An advanced interaction module allows grabbing, cutting, clipping [Fig.4]. and coagulation of virtual tissue and "organs”.

The trainee surgeon manipulates the instruments in the normal way, and in our case the movements are transferred to the graphics workstation by means of a PC-based joint angle measurement system. The PC provides up to 48 analog 12-Bit input channels and 32-Bits digital input for foot switches. The sensor data are transferred with 30 Hz to the graphics station by means of asynchronous RS-232 communications. We use potentiometers hinged to all internal instrument degrees of freedom together with custom designed and manufactured mechanics inside the "Box" to aquire the relative joint positions. On request of the graphical workstation, the signals are submitted via serial Interface with 38400 Bits/sec. The maximum response time delay for aquisition and transmission of one data block is less than 30 ms. In addition to the instrument box, several foot switches are used to control surgical interactions (coagulation) and system control functions (model reset, instrument change).

SIMULATION TECHNIQUES AND MATHEMATICAL FRAMEWORK

KISMET is implemented under the UNIX operating system on SILICON GRAPHICS 4D (SGI) high-performance graphics workstations. This configuration allows for synthetic generation of any view in a surgical simulation with interactive rates, depending on the complexity of the models. In our demonstrator, 20 images per second are calculated, using an SGI-Onyx RE2 workstation with two 200 MHz Mips-R4400 CPU's. Multiple window display and stereoscopic viewing using shutter glasses are supported by KISMET as display options. The objects can be rendered in high quality shaded modes with surface texture, as wireframes, or as transparent models. The display mode can be set interactively as an object attribute for any part and/or for groups of parts.

Our research and software development is currently directed to the simulation of realistic interactions between surgical tools and the organs, which are modelled as deformable bodies [Fig.5]. We have developed simulation techniques which allow the modelling of "virtual tissue" based on a data-model which reflects the physical characteristics like mass, stiffness and damping of real tissue [4]. Virtual organs are modelled in KISMET as elastic NURBS objects. The control points of the NURBS surface form together with additional nodes an elastic mesh of virtual mass points, which are interconnected by springs. The equation of motion for the dynamic spring/mass-node system is solved by discrete integration as a coupled system of second order ordinary differential equations

(1)

A collision test algorithm detects contact between surgical instruments and the virtual organs. The algorithm allows for realtime simulation of tissue elasticity. The algorithm used in KISMET allows for realistic interaction (deformations) between surgical instruments and tissue surfaces in the virtual surgery scenario. As a by-product, contact forces between the tissue surface and the instrument end-effector are calculated, which can be used to drive a force-reflecting surgeon interface. This feature will be used for future tactile feedback in our MIS trainer.

MODELING AND SIMULATION OF SURGICAL SCENARIOS

The kinematics of a mechanical manipulator can be modelled in KISMET as an open or closed-loop articulated chain with several rigid bodies (links) connected in series by either revolute or prismatic joints, driven by actuators. High-level joints (planar, spheric etc.) are modelled with combinations of the two basic types. We use formalised methods to model the kinematical structure of mechanisms to provide a systematic and generalised approach to define and calculate mechanism motion with respect to a fixed reference frame. Thus, KISMET was used at FZK to support the design of the instruments and medical manipulators used in the ARTEMIS telesurgery system [2]. The kinematical simulation of these multi-link mechanisms allows detailed studies during the design stage. The behaviour of these mechanisms can easily be modified by interactive modification of the kinematical design parameters. The influence of machining errors in the range of microns has been studied with KISMET by a tolerance analysis. Geometrical optimisation has been carried out in order to avoid internal collisions in the mechanisms.

Geometrical models of the surgical environment, i.e. the organs, cannot be easily modelled with regular shape primitives. We have used commercially available free-form surface modelers for tissue modeling. In close cooperation with medical staff from University Tübingen (Prof. Buess), we have developed surgical scenarios of the human digestive system and another one of the upper abdominal organs which is mainly used in the laparascopic surgery trainer project showing the organs and conventional instruments relevant for the colecystectomy operation (removal of the gall-bladder).

For tissue modelling we use in KISMET hybrid polyhedron and NURBS-geometry representations. Additionally we have developed a program (CREATOR) to generate simple geometric datasets (a cushion like geometry and tubes) together with the physical properties for elastodynamical simulation. These simple shapes are further interactively edited with KISMET to create the final shape of the "organs".

Another research topic is an interface to KISMET from volume (voxel) based systems as used for display of CT- and MRI-datasets. Such an interface is mandatory for modeling the unique anatomy of a specific patient.

CONCLUSION

It was demonstrated that KISMET can be used effectively in medical virtual reality scenarios with high graphical realism during planning, teaching and surveillance of minimally invasive surgery procedures :

• Development of a Virtual Reality abdominal surgery training system.
• 3D-geometric, kinematic and multibody-dynamics simulation, animation and analysis of complex mechanisms during the design phase of new surgical instruments and medical manipulators
• Operation room scenario simulation
• Realtime visualisation of voxel-volume datasets using 3D-texturing techniques
• High-quality 3D-models of the human abdominal anatomy have been created

ACKNOWLEDGEMENT

We greatfully acknowledge the medical advice given by Prof. G. Buess and his team from Minimal Invasive Surgery at Universitätsklinikum Tübingen/FRG.

REFERENCES

[1] Kühnapfel, U.: "Grafische Realzeitunterstützung für Fernhandhabungsvorgänge in komplexen Arbeitsumgebungen im Rahmen eines Systems zur Steuerung, Simulation und Off-Line-Programmierung"; Doctoral Dissertation, University Karlsruhe (1991).

[2] Voges, U. et.al.: "Experimenteller Telemanipulator für die minimal invasive Chirurgie"; in: Report FZKA-5670, 106-111 (Nov 1995); ISBN 0947-8620, ISSN 0949-7404

[3] Kühnapfel, U., Neisius, B.: "CAD-Based Graphical Computer Simulation in Endoscopic Surgery"; Endoscopic Surgery and Allied Technologies, Georg Thieme Verlag, Stuttgart New York, Vol. 1 No. 2, 181-184 (1993).

[4] Terzopoulus, D., et.al.: "Deformable Models"; The Visual Computer, 4, 306-331 (1988).

List of Figures

Fig. 1: Karlsruhe Endoscopic Virtual Surgery Trainer

Fig. 2: Component overview

Fig. 3: Instrument box

Fig. 4: Simulated endoscopic view inside the virtual abdomen

Fig. 5: Deformable tissue modelled with virtual massknots and connecting springs

CONTACT ADDRESS:

Mail:
Forschungszentrum Karlsruhe GmbH
Institut für Angewandte Informatik
Dr. Uwe G. Kühnapfel
Postfach 3640
D-76021 Karlsruhe, FRG

Voice: +49-(0)7247 82 2567
Fax: +49-(0)7247 82 5786
E-Mail: jet @ iai.fzk.de
www: http://iregt1.iai.fzk.de/