From c29c3fc0ccbae648b0c5d60e1b452f6e4379d3cb Mon Sep 17 00:00:00 2001 From: Tobias Klauser Date: Thu, 2 May 2013 08:58:28 +0200 Subject: Final version of report --- doc/abstract.tex | 2 +- doc/conclusion.tex | 29 ++-- doc/discussion.tex | 55 +++++++- .../locomotor_primitives_reported_forces.png | Bin 0 -> 56218 bytes doc/images/locomotor_primitives_sequence.png | Bin 0 -> 47081 bytes doc/images/locomotor_primitives_sequence.tif | Bin 0 -> 754226 bytes doc/images/simple_leg_sequence.png | Bin 0 -> 251357 bytes doc/images/simple_leg_sequence.tif | Bin 0 -> 8026082 bytes doc/images/simpleleg_sim_0.png | Bin 52694 -> 0 bytes doc/images/simpleleg_sim_1.png | Bin 39199 -> 0 bytes doc/images/simpleleg_sim_2.png | Bin 65248 -> 0 bytes doc/images/simpleleg_sim_3.png | Bin 58261 -> 0 bytes doc/introduction.tex | 6 +- doc/masterproject_klauser.pdf | Bin 0 -> 1660782 bytes doc/methods.tex | 154 +++++++++++++++------ doc/results.tex | 65 +++++++-- doc/title.tex | 15 +- 17 files changed, 244 insertions(+), 82 deletions(-) create mode 100644 doc/images/locomotor_primitives_reported_forces.png create mode 100644 doc/images/locomotor_primitives_sequence.png create mode 100644 doc/images/locomotor_primitives_sequence.tif create mode 100644 doc/images/simple_leg_sequence.png create mode 100644 doc/images/simple_leg_sequence.tif delete mode 100644 doc/images/simpleleg_sim_0.png delete mode 100644 doc/images/simpleleg_sim_1.png delete mode 100644 doc/images/simpleleg_sim_2.png delete mode 100644 doc/images/simpleleg_sim_3.png create mode 100644 doc/masterproject_klauser.pdf (limited to 'doc') diff --git a/doc/abstract.tex b/doc/abstract.tex index 83b3932..c97014f 100644 --- a/doc/abstract.tex +++ b/doc/abstract.tex @@ -2,4 +2,4 @@ Coordinating and learning to coordinate the movement of the human body is a non-trivial problem. One theory of how this could possibly be achieved suggests that complex movements are a result of the combination of simple, rudimentary building blocks called motor primitives. -In the context of this project thesis data from a study on the development of motor primitives for biped locomotion was applied to a musculoskeletal model of the human legs in the OpenSim simulation software. The model could not be made to reproduce walking behavior, however valuable insights were gained on the capabilities of the OpenSim environment. OpenSim is targeted at -- and primarily used in -- studies in biomechanics were detailed clinical data are available. Thus the application of OpenSim to the analysis of more abstract data is not easily possible and would call for the availability of additional high resolution data and the precise modeling of ground contact forces. \ No newline at end of file +In the context of this project thesis data from a study on the development of motor primitives for biped locomotion was applied to a musculoskeletal model of the human legs in the OpenSim simulation software. The model could not be made to reproduce walking behavior, however valuable insights were gained on the capabilities of the OpenSim environment. OpenSim is targeted at -- and primarily used in -- studies in biomechanics were detailed experimental data are available. Thus the application of OpenSim to the analysis of more abstract data is not easily possible and would call for the availability of additional high resolution data and the precise modeling of ground contact forces. \ No newline at end of file diff --git a/doc/conclusion.tex b/doc/conclusion.tex index 0f2ebba..ced517c 100644 --- a/doc/conclusion.tex +++ b/doc/conclusion.tex @@ -1,23 +1,16 @@ \chapter{Conclusion} \label{ch:conclusion} -\TODO{Conclusion about work done} +Musculoskeletal modeling and simulation is an instrumental tool in understanding muscle function involved in +biped walking. However creating these simulations at an anatomically realistic level is impossible without +proper experimental data. If phenomena shall be studied at a more abstract and fundamental level, the models +and the input need to be abstracted accordingly. -Musculoskeletal modeling has been instrumental in understanding joint torque and muscle function -during movements based on the physics of the musculoskeletal system. Because generating de novo -muscle-actuated simulations of movement based on performance criteria is challenging as a result -of both muscle redundancy and the stiffness of the solution space, particularly for unstable tasks -such as walking \cite{Ting2012} +The OpenSim software was used to analyze the feasibility of such an abstract-level musculoskeletal simulation +using locomotor primitives patterns. However the focus of usage for OpenSim currently is in areas where +adequate and precise experimental data is available or can easily be generated. -Thus OpenSim with the provided and used muscle-skeleton models is in its current state -not feasible for the simulation of tasks such as those that were subject of this project. - -\begin{itemize} - \item Precise EMG data from an actual subject (no averaged an processed patterns) would be needed in order to achieve realistic behavior in simulation with OpenSim. - \item Focus of OpenSim more on clinical applications where such data is available. - \item A more abstract musculoskeletal model of a human would need to be developed in OpenSim, for such simulations, but is outside of the focus of this project. The OpenSim API would provide a decent foundation for developing such a model from scratch. - \item Ground contact forces would need to be incorporated (and provided by the study the project is based on). - \item Muscles were removed from the model ``arbitrarily'' (because no data is available for them), consequences for behavior of model unclear -\end{itemize} - -More abstract models such as the ones used by Geyer and Herr \cite{Geyer2010}, Marques et al. \cite{Marques2012a, Marques2012} or Wang et al. \cite{Wang2012} might be more suitable. \ No newline at end of file +More abstract -- bottom-up built -- models such as the ones used by Geyer and Herr \cite{Geyer2010}, Marques et +al. \cite{Marques2012a, Marques2012} or Wang et al. \cite{Wang2012} are more suitable for these kinds of +analyses. Such an abstracted model could possibly also be built inside OpenSim, however the tools it currently +provides to achieve this are limited in their functionality. \ No newline at end of file diff --git a/doc/discussion.tex b/doc/discussion.tex index e5a814a..0b3b029 100644 --- a/doc/discussion.tex +++ b/doc/discussion.tex @@ -1,10 +1,59 @@ \chapter{Discussion} \label{ch:discussion} -\TODO{Discuss the few results obtained. With a lot of interpretation an alternating movement of the legs could be seen. Problems with data (missing ground contact forces), OpenSim more suited for analysis of precise data acquired in subject studies. Ground contact forces key to investigate walking.} +The results acquired in the few experiments that were conducted didn't match the initial expectations of the +project. Rather they showed the difficulties of working with OpenSim given only the locomotor primitive +patterns as input data for the simulation. Since several insights into the working of OpenSim, its advantages +and disadvantages were gained during the course of this project, mainly an evaluation of the OpenSim software +shall be discussed here. + +\section{Simulation using Locomotor Primitives} + +The unrealistic behavior in the simulation can be attributed to three main reasons: lack of ground contact +forces, scaling of the muscle activation and lack of joint constraints. As can be seen in the results for the +exerted muscle forces, by scaling the muscle activations accordingly, at least the muscle forces can be limited +to an appropriate range. One reason for the ``twiddling'' with the scaling of the activation is that OpenSim +doesn't specify a unit for the muscle activation. Rather it's a normalized value in the range $0.0$--$1.0$, but +the exact mapping of measured values to this range is unclear and no documentation could be found. Also the +muscle's default activation parameter was found to play a crucial role in the behavior of the simulation. As +with the muscle activation, unfortunately no documentation of the proper settings of this parameters could be +found with OpenSim. + +The lack of data for ground contact forces most probably is the main reason for the problems with respect to +the simulation. Ground contact forces were found to be crucial for the control of walking and the patterns of +muscle activation involved in walking \cite{Ivanenko2002}. By not incorporating them in the simulation which is +conducted using muscle activation data that was generated when ground contact forces were present, the +experimental setup is not appropriately reproduced and the outcome of the simulation can become unrealistic. \section{Evaluation of OpenSim} -\TODO{Describe what OpenSim is targeted at, how common studies done with it are done. How does it differ from what we intended to do? What would we need to provide in order to work with OpenSim for this project? Would it even be possible? What's the effort needed?} +OpenSim is a great tool to conduct biomechanics experiments -- when experimental data is available. Its usage +thus mainly is in areas where such data can be directly gathered. Its community provides a wide array of +musculoskeletal models and tools for analyzing simulations. However in order to use these appropriately, the +availability of experimental data -- namely marker trajectories, ground reaction forces and electromyography +measurements -- is key. The experimental protocols suggested to be used with OpenSim\footnote{\url{http:// +simtk-confluence.stanford.edu:8080/display/OpenSim/Collecting+Experimental+Data}} are rather strict e.g. with +respect to the measurement of forces. Up to now OpenSim is mainly used in biomechanics or clinical studies +(e.g. to compare the effects of different treatments to pathological gaits). In all of these cases subject data +were available. + +In this project however a lot of abstractions and simplifications (e.g. removing muscles, applying combined activations to certain muscles, neglecting ground contact forces) were made to a model whose purpose is to +represent the human legs as anatomically correct and realistic as possible. Furthermore abstracted data was +applied to it. This is not the indented usage of these models and the effects are unknown and cannot be +predicted when lacking the proper knowledge about its exact working. Furthermore the models used are very +sensitive to the type of muscle activation data that was provided (i.e. as could be seen in the random data +tests). Unfortunately also the scales and units used for the input data and model parameters (e.g. joint parameters) is not always clearly disclosed. + +OpenSim's open-source nature makes it possible to build models and simulations from ground up. The API provided +as part of Simbody and OpenSim provide a good base for physics-based simulations of multi-body systems. It can +be extended and adjusted if needed. Implementing e.g. custom simulation controllers or custom muscle models can +relatively easily be added and just need to implement the according interfaces, as it was done in this project. +However documentation on most of OpenSim's internals is rather scarce or only available in the source code. + +Building an abstract model from ground up is not easy and the supporting tools provided by OpenSim are very +rudimentary and are mostly targeted at changing existing models shipped with OpenSim. However if an abstract +model shall be built -- given the experiences gained in this project -- it should be done bottom-up rather than +top-down. -Short-comings of OpenSim with respect to documentation, source code is often the only available source of documentation for specific questions. Scales and units for certain parameters of muscles and other model componenents (e.g. joints) not disclosed. Glitches in joint behavior (over-bending of joints, certain parameters unpredictably leading to failure in simulation). \ No newline at end of file +Thus OpenSim with the provided muscle-skeleton models is in its current state not directly feasible for +simulations lacking suitable experimental data. \ No newline at end of file diff --git a/doc/images/locomotor_primitives_reported_forces.png b/doc/images/locomotor_primitives_reported_forces.png new file mode 100644 index 0000000..e144afd Binary files /dev/null and b/doc/images/locomotor_primitives_reported_forces.png differ diff --git a/doc/images/locomotor_primitives_sequence.png b/doc/images/locomotor_primitives_sequence.png new file mode 100644 index 0000000..6befdec Binary files /dev/null and b/doc/images/locomotor_primitives_sequence.png differ diff --git a/doc/images/locomotor_primitives_sequence.tif b/doc/images/locomotor_primitives_sequence.tif new file mode 100644 index 0000000..ed3c7c3 Binary files /dev/null and b/doc/images/locomotor_primitives_sequence.tif differ diff --git a/doc/images/simple_leg_sequence.png b/doc/images/simple_leg_sequence.png new file mode 100644 index 0000000..11ffddc Binary files /dev/null and b/doc/images/simple_leg_sequence.png differ diff --git a/doc/images/simple_leg_sequence.tif b/doc/images/simple_leg_sequence.tif new file mode 100644 index 0000000..694361b Binary files /dev/null and b/doc/images/simple_leg_sequence.tif differ diff --git a/doc/images/simpleleg_sim_0.png b/doc/images/simpleleg_sim_0.png deleted file mode 100644 index 58906cf..0000000 Binary files a/doc/images/simpleleg_sim_0.png and /dev/null differ diff --git a/doc/images/simpleleg_sim_1.png b/doc/images/simpleleg_sim_1.png deleted file mode 100644 index 296c64d..0000000 Binary files a/doc/images/simpleleg_sim_1.png and /dev/null differ diff --git a/doc/images/simpleleg_sim_2.png b/doc/images/simpleleg_sim_2.png deleted file mode 100644 index 162832f..0000000 Binary files a/doc/images/simpleleg_sim_2.png and /dev/null differ diff --git a/doc/images/simpleleg_sim_3.png b/doc/images/simpleleg_sim_3.png deleted file mode 100644 index b7721a7..0000000 Binary files a/doc/images/simpleleg_sim_3.png and /dev/null differ diff --git a/doc/introduction.tex b/doc/introduction.tex index 8b1d210..ba1a67a 100644 --- a/doc/introduction.tex +++ b/doc/introduction.tex @@ -1,10 +1,10 @@ \chapter{Introduction} -Controlling and learning to control movements of a many degree of freedom system such as the human body is a non-trivial task. How does the central nervous system (CNS) choose from the infinitely many possibilities of how to achieve a certain movement? This problem is famously known as Bernstein's problem \cite{Bernstein1967}. Understanding how natural agents such as humans and animals control their movements is one of the fundamental questions in embodied artificial intelligence \cite{Pfeifer1999,Pfeifer2006} +Controlling and learning to control movements of a many degree of freedom system such as the human body is a non-trivial task. How does the central nervous system (CNS) choose from the infinitely many possibilities of how to achieve a certain movement? This problem is famously known as Bernstein's problem \cite{Bernstein1967}. Understanding how natural agents such as humans and animals control their movements is one of the fundamental questions in embodied artificial intelligence \cite{Pfeifer1999,Pfeifer2006} but also holds insights for the field of robotics. -Complex movements of humans (and other animals such as cats \cite{Torres-Oviedo2006}) are thought to develop based on a simple, rudimentary set of building blocks -- commonly called motor primitives or muscle synergies -- during ontogenetic development \cite{Flash2005, Hart2010}. These primitives are defined by the pattern of simultaneous activation of different muscles and are then combined in order to achieve a certain movement or pose. +Complex movements of humans (and other animals such as cats \cite{Torres-Oviedo2006}) are thought to develop based on a simple, rudimentary set of building blocks -- commonly called motor primitives or muscle synergies -- during ontogenetic development \cite{Flash2005, Hart2010, Ivanenko2006, Ting2012}. These primitives are defined by the pattern of simultaneous activation of different muscles and are then combined in order to achieve a certain movement or pose. -Contrary to the hitherto belief that these innate simple reflexes get completely replaced by more complex ones with age, a recent publication by Dominici et al. \cite{Dominici2011} shows that certain basic patterns of muscle activation involved in biped walking are instead retained and are being built upon during development in order to achieve a variety of walking movement behaviors. The study measured EMG activity of 24 muscles (the same 12 muscles in each leg) simultaneously in neonates\footnote{Even though neonates of course can't walk by themselves, stepping can be evoked when they're being held upright and by gently pushing the infant along the pathway. However this behavior typically disappears 4 to 6 weeks after birth.} ($\sim{}3$ days old), toddlers (11-14 months old), preschoolers (22-48 months old) and adults (25-40 years old). Additionally foot pressure was recorded as well as the limb kinematics. +Contrary to the hitherto belief that these innate simple reflexes get completely replaced by more complex ones with age, a recent publication by Dominici et al. \cite{Dominici2011} shows that certain basic patterns of muscle activation involved in biped walking are instead retained and are being built upon during development in order to achieve a variety of walking movement behaviors. The study measured EMG activity of 24 muscles (the same 12 muscles in each leg) simultaneously in neonates\footnote{Even though neonates of course can't walk by themselves, stepping can be evoked by holding them upright and then gently pushing the infant along the pathway. However this behavior typically disappears 4 to 6 weeks after birth.} ($\sim{}3$ days old), toddlers (11-14 months old), preschoolers (22-48 months old) and adults (25-40 years old). Additionally foot pressure was recorded as well as the limb kinematics. This project thesis attempts to investigate the role these locomotor primitives play in the development of human bipedal walking by building a musculoskeletal simulation model in the OpenSim software \cite{Delp2007} and then applying the basic activation patterns at different stages of human development (neonate, toddlers, preschooler and adults) identified by Dominici et al. \cite{Dominici2011} to it in order to achieve a walking behavior. Thereby we hope to understand the basis of the development of biped walking in order to possibly be applied to humanoid robots, especially anthropomimetic robots \cite{Wittmeier2013} which more closely mimic the internal structure of the human body with respect to muscles, tendons, bones and joints. diff --git a/doc/masterproject_klauser.pdf b/doc/masterproject_klauser.pdf new file mode 100644 index 0000000..a97d8d1 Binary files /dev/null and b/doc/masterproject_klauser.pdf differ diff --git a/doc/methods.tex b/doc/methods.tex index c29522d..8758443 100644 --- a/doc/methods.tex +++ b/doc/methods.tex @@ -3,36 +3,72 @@ \section{Simulation Software} -Simulation is performed using the OpenSim\footnote{Not to be confused with the 3D virtual world software OpenSimulator, which is sometimes also abbreviated as OpenSim.} software \cite{Delp2007}. OpenSim is an open-source platform for modeling and analyzing neuromusculoskeletal structures and simulating their dynamic movement behavior. It's developed and maintained at the NIH Center for Biomedical Computation at Stanford University and is distributed as part of the Simtk.org repository. - -In OpenSim, a simulation typically consists of a musculoskeletal model and a controller. The model can either be built individually or one of the models provided with OpenSim can be used. The model contains any combination of rigid bodies (such as bones), joints (simple or complex), constraints, springs, dampers, muscles and other actuators. A model is then assigned a controller which defines its dynamic behavior during simulation. There are predefined controllers available for forward dynamics (drive simulation with given muscle excitations), inverse kinematics (position model in simulation according to experimental markers and coordinate data at each time step) and inverse dynamics (determine generalized forces at each joint responsible for a given movement). - -The software provides an extensive C++ application programming interface (API) in order to develop custom simulation controllers. Additionally it contains a graphical user interface (GUI) used to visualize models and simulations, edit models and muscle excitation data, as well as to access predefined simulation controllers. The GUI can be used independently from the simulation and the visualization is done using the data previously generated in the offline simulation. - -The controllers provided with the OpenSim distribution turned out to require a very specific data collection as its input. These types of data are typically generated during experiments using motion capturing systems and force plates. In the context of this project, no such data was or is available or could easily be acquired. Instead we rather rely on relatively sparse muscle activation data and have no ground force measurements available. Also we are interested in general principles of movement rather than the detailed analysis of the walking behavior of an individual subject. - -Thus a custom controller making use of the provided OpenSIM C++ API was developed which actuates the model based on the basic activation patterns from \cite{Dominici2011}. This approach also allows to specifically tune the simulation behavior to the requirements of this project and the data available. - -For all tasks performed as part of this project, version 3.0 of OpenSim was used as provided on \url{http://www.simtk.org}. The source version was used and built on Ubuntu Linux according to the provided instructions. Since the Linux version of OpenSim does not provide the GUI, the Windows version of OpenSim was used for visualization. +Simulation is performed using the OpenSim\footnote{Not to be confused with the 3D virtual world software +OpenSimulator, which is sometimes also abbreviated as OpenSim.} software \cite{Delp2007}. OpenSim is an open- +source platform for modeling and analyzing neuromusculoskeletal structures and simulating their dynamic +movement behavior. It's developed and maintained at the NIH Center for Biomedical Computation at Stanford +University and is distributed as part of the Simtk.org repository. + +In OpenSim, a simulation typically consists of a musculoskeletal model and a controller. The model can either +be built individually or one of the models provided with OpenSim can be used. The model contains any +combination of rigid bodies (such as bones), joints (simple or complex), constraints, springs, dampers, +muscles and other actuators. A model is then assigned a controller which defines its dynamic behavior during +simulation. There are predefined controllers available for forward dynamics (drive simulation with given +muscle excitations), inverse kinematics (position model in simulation according to experimental markers and +coordinate data at each time step) and inverse dynamics (determine generalized forces at each joint +responsible for a given movement). + +The software provides an extensive C++ application programming interface (API) in order to develop custom +simulation controllers. The SimTK multi body dynamics API (Simbody) is used as the basic physics engine. +Additionally OpenSim contains a graphical user interface (GUI) used to visualize models and simulations, edit +models and muscle excitation data, as well as to access predefined simulation controllers. The GUI can be used +independently from the simulation and the visualization is done using the data previously generated in the +offline simulation. + +The controllers provided with the OpenSim distribution turned out to require a very specific data collection +as their input. This type of data is typically generated during biomechanics experiments, where muscle activations are measured using electromyography (EMG), the kinematics are tracked using a motion +capturing system and force plates are used to register ground contact forces. + +In the context of this project, no such data was available or could easily be acquired. Instead we rather +relied on relatively sparse muscle activation patterns and have no ground force measurements or tracked +kinematic paths available. Also the interest is in general principles of movement rather than the detailed +analysis of the walking behavior of an individual subject. + +Thus a custom controller making use of the provided OpenSIM C++ API was developed which actuates the model +based on the basic activation patterns extracted from \cite{Dominici2011}. This approach also allows to +specifically tune the simulation behavior to the requirements of this project and the data available. + +For all tasks performed as part of this project, version 3.0 of OpenSim was used as provided on +\url{http://www.simtk.org}. The source version was used and built on Ubuntu Linux according to the provided +instructions. Since the Linux version of OpenSim does not provide the GUI, the Windows version of OpenSim was +used for visualizations. \section{Musculoskeletal Model} \subsection{Simple Leg Model} \label{subsec:simple_leg_model} -In order to get familiar with modeling and controlling models in OpenSim, a simple model of a leg was created. It consists of three rigid bodies -- pelvis, femur and tibia -- and two joints -- hip and knee. The pelvis was fixed in space, the femur attached to it and the tibia attached to the femur. The model can be actuated using two muscles: Rectus Femoris and Biceps Femoris, which are an agonist/antagonist pair of muscles. They're attached to the pelivs on one end and to the femur and the tibia on the other end respectively. See figure \ref{fig:screenshot_opensim_simpleleg} for a visualization of the model. +In order to get familiar with modeling and controlling models in OpenSim, a simple and abstracted skeletal +model of a human leg was created. It consists of three rigid bodies -- \emph{pelvis}, \emph{femur} and +\emph{tibia} -- and two joints -- hip and knee. The pelvis was fixed in space, the femur attached to it using +a pin joint (one degree-of-freedom) and the \emph{tibia} attached to the \emph{femur} using a pin joint. The +model can be actuated using two muscles: \emph{Rectus Femoris} (anterior) and \emph{Biceps Femoris} +(posterior), which are in this abstract model an agonist/antagonist pair of muscles and represent the largest +muscles in the human lower extremity. They're attached to the pelvis on one end and to the femur and the tibia +on the other end respectively. See figure \ref{fig:screenshot_opensim_simpleleg} for a visualization of the +model. \begin{figure}[htb!] -\label{fig:screenshot_opensim_simpleleg} \centering -\includegraphics[scale=0.6]{images/screenshot_opensim_simpleleg.png} -\caption{Visualization of the SimpleLeg model} +\includegraphics[scale=0.8]{images/screenshot_opensim_simpleleg.png} +\caption{Visualization of the SimpleLeg model. The topmost bone (\emph{pelvis}) is fixed in space, with the \emph{femur} and \emph{tibia} attached using pin joints. Actuation is achieved using the \emph{Rectus Femoris} (right) and \emph{Biceps Femoris} (left) muscles.} +\label{fig:screenshot_opensim_simpleleg} \end{figure} \subsection{Two-legged Model} \label{subsec:two_legged_model} -In the study by Dominici et al. \cite{Dominici2011} EMG activity of the following set of 12 muscles was recorded for each leg of the subject: +In the study by Dominici et al. \cite{Dominici2011} the EMG activity of a selected set of 12 muscles was recorded for each leg of the subject. The following muscles were included in the publication: \begin{itemize} \item \emph{Tibialis Anterior (TA)} @@ -51,16 +87,16 @@ In the study by Dominici et al. \cite{Dominici2011} EMG activity of the followin The musculoskeletal model to be used thus had to incorporate all these muscles for each leg, so the proper activation patterns could be applied. -Several musculoskeletal models of human legs are already provided with OpenSim 3.0. For this project the \emph{Gait2354} model was used as a basis. This is a 23-degree-of-freedom model of the musculoskeletal system of the human leg. By default it contains 54 musculo-tendon actuators and represents a subject with a height of 1.8 m and a mass of 75.16 kg\footnote{More information on the \emph{Gait2354} model and its kinematic and dynamic properties as well as references to the studies the model is based on can be found on \url{http://simtk-confluence.stanford.edu:8080/display/OpenSim/Gait+2392+and+2354+Models}}. See figure \ref{fig:screenshot_opensim_gait2354} for a visualization of the model. Since the model represents an adult subject, the adult data from \cite{Dominici2011} was of primary interest. However OpenSim provides means to scale a given musculoskeletal model to different sizes, so the model could possibly be adapted to represent a neonate, toddler or preschooler respectively. +Several musculoskeletal models of human legs are already provided with OpenSim 3.0. For this project the \emph{Gait2354} model was used as a basis. This is a 23-degree-of-freedom model of the musculoskeletal system of the human leg. By default it contains 54 musculo-tendon actuators and represents a subject with a height of 1.8 m and a mass of 75.16 kg\footnote{More information on the \emph{Gait2354} model and its kinematic and dynamic properties as well as references to the studies the model is based on can be found on \url{http://simtk-confluence.stanford.edu:8080/display/OpenSim/Gait+2392+and+2354+Models}}. See figure \ref{fig:screenshot_opensim_gait2354} for a visualization of the model. Since the model represents an adult subject, the adult data from \cite{Dominici2011} was of primary interest. However OpenSim provides means to scale a given musculoskeletal model to different sizes, so the model could eventually be adapted to represent a neonate, toddler or preschooler respectively. \begin{figure}[htb!] -\label{fig:screenshot_opensim_gait2354} \centering \includegraphics[scale=1.0]{images/screenshot_opensim_gait2354.png} \caption{Visualization of the \emph{Gait2354} model} +\label{fig:screenshot_opensim_gait2354} \end{figure} -The model was then adapted in order to only in its final version incorporate only the muscles for which data was available. Table \ref{table:muscles_correspondence} shows the muscles from the study and the \emph{Gait2354} musculoskeletal model respectively and how their activation was determined from the study data in the final model. Some of the muscles (the \emph{Hamstring} and \emph{Gluteus Maximus} specifically) were measured as one in the study, but the muscles were present individually in the model. They were left as is and later on the controller was adapted to distribute the entire activation among the individual muscles (see \ref{subsec:locomotor_primitives_controller} for details). Furthermore for some of the muscles in the original \emph{Gait2354} model no activation data was available from the study, thus they were removed in the model for this project. The table shows only the muscles for one leg, the muscles of second leg were handled correspondingly. The leg to which each muscle in the model belongs, is signified by it having a postfix \emph{\_l} for the left leg, or a postfix \emph{\_r} for the right leg. +The model was then adapted in order for its final version to only incorporate the muscles for which publication data was available. Table \ref{table:muscles_correspondence} shows the muscles from the study and the \emph{Gait2354} musculoskeletal model respectively and how their activation was determined from the study data in the final model. Some of the muscles (the \emph{Hamstring} and \emph{Gluteus Maximus} specifically) were measured in combination for the study, but the muscles were present individually in the model. They were left as is and later on the controller was adapted to distribute the entire activation among the individual muscles (see \ref{subsec:locomotor_primitives_controller} for details). Furthermore for some of the muscles in the original \emph{Gait2354} model no activation data was available from the study, thus the respective muscles were removed from the model. The table shows only the muscles for one leg, the muscles of the second leg were handled correspondingly. The leg to which each muscle in the model belongs, is signified by it having a postfix \emph{\_l} for the left leg, or a postfix \emph{\_r} for the right leg. \begin{center} \begin{longtable}{llll} @@ -132,41 +168,43 @@ Tibialis Posterior & tib\_post & - & removed from model \\ [1ex] \hline \\ [-1.5ex] Vastus Intermedius & vas\_int & - & removed from model \\ [1ex] \hline \\ [-1.5ex] -\caption{Muscles in the Gait2354 model and their correspondents in \cite{Dominici2011}.} +\caption{Muscles in the \emph{Gait2354} model and their correspondents in \cite{Dominici2011}.} \end{longtable} \end{center} -Because no data was available for the \emph{External Oblique (OE)} muscle -- one of the muscles which connects the torso to the pelvis -- the torso was removed from the model because otherwise only the back muscles, the \emph{Erector Spinae (ES)} would connect the torso to the pelvis, but would have no antagonist. Therefor OE and ES were removed, resulting in a total of 16 muscles per leg (32 total) in the final version of the model. +Because no data was available for the \emph{External Oblique (OE)} muscle -- one of the muscles which connects the torso to the pelvis -- the torso was removed from the model entirely, because otherwise only the back muscles -- the left and right \emph{Erector Spinae (ES)} -- would connect the torso to the pelvis, but would have no antagonist. Therefor \emph{OE} and \emph{ES} were removed, resulting in a total of 16 muscles per leg (32 total) in the final version of the model. -The joints were left as in the original \emph{Gait2354 models}. For initial experiments the ``joint'' connecting the pelvis to the world was fixed in space using the \emph{WeldJoint} joint type of OpenSim. +The joints were left as in the original \emph{Gait2354 models}. For initial experiments the ``joint'' +connecting the pelvis to the world was fixed in space using the zero-degree-of-freedom \emph{WeldJoint} joint +type of OpenSim. Figure \ref{fig:screenshot_opensim_locomotorprimitives} shows the resulting musculoskeletal model. \begin{figure}[htb!] -\label{fig:screenshot_opensim_locomotorprimitives} \centering \includegraphics[scale=1.0]{images/screenshot_opensim_locomotorprimitives.png} \caption{Visualization of the final musculoskeletal model used for the project.} +\label{fig:screenshot_opensim_locomotorprimitives} \end{figure} \section{Muscle Model} \subsection{Thelen Muscle Model} -For all muscles in the model, the Thelen mathematical muscle model \cite{Thelen2003} was used. This model is the default muscle model used by OpenSim 3.0\footnote{Note that OpenSim version up to 2.4 included numerical errors in the implementation. This deprecated version of the muscle model is still available in the software in the C++ class \texttt{OpenSim::Thelen2003Muscle\_Deprecated} but shouldn't be used for actual simulations.} and is available in the C++ class \texttt{OpenSim::Thelen2003Muscle}. This muscle model is employed in all musculoskeletal models provided with OpenSim. +For all muscles in the model, the Thelen mathematical muscle model \cite{Thelen2003} was used. This model is the default muscle model used by OpenSim 3.0\footnote{Note that up to version 2.4 OpenSim contained numerical errors in the implementation of the Thelen muscle model. This deprecated version is still available in the software in the C++ class \texttt{OpenSim::Thelen2003Muscle\_Deprecated} but shouldn't be used for actual simulations.} and is available in the C++ class \texttt{OpenSim::Thelen2003Muscle}. This muscle model is employed in all musculoskeletal models provided with OpenSim. The Thelen muscle model is based on the well-known and widely used Hill-type muscle-tendon model \cite{Hill1922,Hill1970,Shadmehr2005,Zajac1989}, but incorporates adjustments in order to better account for age-related changes in the muscle properties. \begin{figure}[htb!] -\label{fig:hill_muscle_model} \centering \includegraphics[scale=2.0]{images/hill_muscle_model.pdf} -\caption{Schematic of the Hill-type mathematical muscle model.Illustration taken from \cite{Shadmehr2005}} +\caption{Schematic of the Hill-type mathematical muscle model. Illustration taken from \cite{Shadmehr2005}} +\label{fig:hill_muscle_model} \end{figure} -In the Thelen as well as in the Hill model, the muscle-tendon complex of an actuator consists of three components: a \emph{contractile element (CE)}, a \emph{passive element (PE)}, and a \emph{series elastic element (SEE or SE)}. See figure \ref{fig:hill_muscle_model} for an illustration, where $T$ is the tension, $K_{SE}$ the spring constant of the \emph{SE}, $K_{PE}$ the sprain constant of the \emph{PE}, $b$ the damping coefficient (of the \emph{CE}) and $A$ the active force of the muscle. +In the Thelen as well as in the Hill model, the muscle-tendon complex of an actuator consists of three components: a \emph{contractile element (CE)}, a \emph{passive element (PE)}, and a \emph{series elastic element (SEE or SE)}. See figure \ref{fig:hill_muscle_model} for an illustration, where $T$ is the tension, $K_{SE}$ the spring constant of the \emph{SE}, $K_{PE}$ the spring constant of the \emph{PE}, $b$ the damping coefficient of the \emph{CE} and $A$ the active force of the muscle. -The following parameters are used to characterize each Thelen-type muscle: maximum isometric force, optimal muscle fiber length, tendon slack length, maximum contraction velocity, and pennation angle. During simulation, the muscle force is calculated using two states: the activation value and the muscle fiber length. +The following parameters are used to characterize each Thelen-type muscle: maximum isometric force, optimal muscle fiber length, tendon slack length, maximum contraction velocity, and pennation angle. During simulation, the muscle force is calculated using two states: the activation value and the muscle fiber length. All these parameters were left intact for the adjusted model. \section{Simulation Controller} @@ -174,32 +212,60 @@ The following parameters are used to characterize each Thelen-type muscle: maxim The \emph{Simple Leg Controller} was used in order to create a simulation using the Simple Leg model described in \ref{subsec:simple_leg_model} in order to get familiar with the OpenSim workflow. -It implements a behavior similar to the stretch reflex (\emph{myotatic reflex}): An increase of muscle length causes the muscle spindles to be stretched, leading them to increase neural activity. This increases the activity of alpha motor neurons, which cause the muscle fibers to contract in order to regain the original muscle length. Using a second set of neurons, the antagonist muscle is caused to relax \cite{Bear2007}. +It implements a behavior similar to the stretch reflex (\emph{myotatic reflex}): An increase of muscle length +causes the muscle spindles to be stretched, leading them to increase neural activity. This increases the +activity of alpha motor neurons, which cause the muscle fibers to contract in order to regain the original +muscle length. Using a second set of neurons, the antagonist muscle is caused to relax \cite{Bear2007}. + +OpenSim provides access to the lengthening speed of each muscle during simulation. This was used in the +controller to deduce increases in muscle length and cause the corresponding reflex. \subsection{Locomotor Primitives Controller} \label{subsec:locomotor_primitives_controller} -All the predefined controllers available with OpenSim require a specific set of input data typically resulting from biomechanics studies using EMG, tracker systems and force plates. Since such data was not available in the context of this project, a custom controller for the simulation of the musculoskeletal model was developed in order to account for the limited availability of data. - -The \emph{Locomotor Primitives Controller} allows to actuate the model described in section \ref{subsec:two_legged_model} using only the activation patterns for its 32 muscles (16 per leg). The data can be fed into the controller using the \emph{OpenSim Storage file format} (file extension \texttt{.sto}), which is the standard format to store and read data in OpenSim\footnote{The file format is explained in \url{http://simtk-confluence.stanford.edu:8080/display/OpenSim/Storage+(.sto)+Files}, but there are some subtleties to the be considered when generating \texttt{.sto} files outside of OpenSim. Thus looking at the implementation of the class \texttt{OpenSim::Storage} is suggested when generating \texttt{.sto} files using external scripts.}. - -All the activation data from the storage file are read into the controller and are accessible using the column name, which corresponds to the muscle name according to table \ref{table:muscles_correspondence} with the proper postfix. The controller checks the availability of activation data for all muscles for the entire simulation time and is then able to retrieve the corresponding activation for each muscle at each time step. - -The time step resolution in OpenSim by default is dynamic and determined by the integrator used. For this project a Runge-Kutta integrator in the implementation of \texttt{SimTK::RungeKuttaMersonIntegrator} was used. This is the suggested default integrator in OpenSim. - -Since the time step is not constant when using this integration method, OpenSim allows to interpolate the data read from the storage file in order to provide activation values at all possible time steps during the simulation. - -For most of the muscles in the model, the \emph{Locomotor Primitives Controller} applies the activation value directly. For some special cases however (see column ``Activation Usage'' in table \ref{table:muscles_correspondence}), the muscle activation is split among a set of muscles, because they were measured in combination. Furthermore, the controller allows to scale the activation for each muscle individually by a constant value. +All the predefined controllers available with OpenSim require a specific set of input data typically resulting +from biomechanics studies using EMG, tracker systems and force plates. Since such data was not available in +the context of this project, a custom controller for the simulation of the musculoskeletal model was developed +in order to account for the limited availability of data. + +The \emph{Locomotor Primitives Controller} allows to actuate the model described in section +\ref{subsec:two_legged_model} using only the activation patterns for its 32 muscles (16 per leg). The data can +be fed into the controller using the \emph{OpenSim Storage file format} (file extension \texttt{.sto}), which +is the standard format to store and read data in OpenSim\footnote{The file format is explained in \url{http:// +simtk-confluence.stanford.edu:8080/display/OpenSim/Storage+(.sto)+Files}, but there are some subtleties to the +be considered when generating \texttt{.sto} files outside of OpenSim. Thus looking at the implementation of +the class \texttt{OpenSim::Storage} is suggested when generating \texttt{.sto} files using external scripts.}. + +All the activation data from the storage file are read into the controller and are accessible using the column +name, which corresponds to the muscle name according to table \ref{table:muscles_correspondence} with the +proper postfix. The controller checks the availability of activation data for all muscles for the entire +simulation time and is then able to retrieve the corresponding activation for each muscle at each time step. + +The time step resolution in OpenSim by default is dynamic and determined by the integrator used. For this +project a Runge-Kutta integrator in the implementation of \texttt{SimTK::RungeKuttaMersonIntegrator} was used. +This is the suggested default integrator in OpenSim. + +Since the time step is not constant when using this integration method, OpenSim allows to interpolate the data +read from the storage file in order to provide activation values at all possible time steps during the +simulation. The total duration of the simulation can be specified as a parameter to the controller, so +interpolation of the data points is also crucial for that matter. + +For most of the muscles in the model, the \emph{Locomotor Primitives Controller} applies the activation value +directly. For some special cases however (see column ``Activation Usage'' in table +\ref{table:muscles_correspondence}), the muscle activation is split among a group of muscles, because they +were measured in combination. Furthermore, the controller allows to scale the activation for each muscle +individually by a constant value. \section{Data Preparation} \begin{figure}[htb!] -\label{fig:dominici_patterns_graphs} \centering -\includegraphics[scale=1.0]{images/dominici_patterns_graphs.pdf} -\caption{Muscle activation pattern curves for the 24 muscles (EMG profiles averaged over all subjects). Taken from \cite{Dominici2011}.} +\includegraphics[width=\textwidth]{images/dominici_patterns_graphs.pdf} +\caption{Muscle activation pattern curves for the 24 muscles (EMG profiles averaged over all subjects). + Illustration extracted from \cite{Dominici2011}} +\label{fig:dominici_patterns_graphs} \end{figure} -Since the original data from the publication by Dominici et al. \cite{Dominici2011} could not be used, the respective pattern curves had to be extracted from the paper (see figure \ref{fig:dominici_patterns_graphs}). This also lead to the fact, that neither data on ground contact forces nor kinematic tracker data was available. +Since the original data from the publication by Dominici et al. \cite{Dominici2011} could not be used, the respective pattern curves had to be extracted from the paper (see figure \ref{fig:dominici_patterns_graphs}). This also lead to the fact, that neither data on ground contact forces nor kinematic tracker data was available, since they're not published. In order to gather the activation data from the publication, a graph digitizer software (GraphClick by Arizona Software) was used. The data was then preprocessed by applying a curve interpolation in order to allow arbitrary, linearly spaced sampling of data points later on in the OpenSim model controller. Curve interpolation was done in a Python script using the \emph{NumPy} and \emph{SciPy} toolboxes. \ No newline at end of file diff --git a/doc/results.tex b/doc/results.tex index 8a2e5a1..21814e4 100644 --- a/doc/results.tex +++ b/doc/results.tex @@ -3,18 +3,67 @@ \section{Reflexes in Simple Leg Model} -\TODO{Describe results} +The simulation of the \emph{Simple Leg Model} led to the expected stretch reflex behavior. Given an initial +activation of the \emph{Rectus Femoris}, the \emph{Biceps Femoris} was caused to shorten proportional to the +speed of the shortening of the \emph{Rectus Femoris}. Since this will again cause \emph{Rectus Femoris} to be +shortened, the reflex is triggered in the other direction. For initial tests, the knee joint in the model was +fixed in all but one degree of freedom (pin joint) in order to permit flexion and tension. Unfortunately this +didn't lead to a realistic behavior with respect to the movement of the lower leg, as the knee joint remained +fixed during the entire simulation (see figure \ref{fig:simpleleg_sequence} for a sequence of two stretch +cycles). However since this simulation served as a test of the principal methods and the results were +satisfying enough, this problem was not further investigated. -\section{Random Data} +\begin{figure}[htb!] +\centering +\includegraphics[width=\textwidth]{images/simple_leg_sequence.png} +\caption{Sequence of the \emph{SimpleLeg} model during simulation of two cycles of the stretch reflex.} +\label{fig:simpleleg_sequence} +\end{figure} -In order to validate the principal working of the two-legged musculoskeletal model (see section \ref{subsec:two_legged_model}) and the corresponding controller (see section \ref{subsec:locomotor_primitives_controller}), random data resembling muscle activation patterns was generated for all the 53 muscles of the model. +\section{Random Data in Two-Legged Model} -\TODO{Add example plot} +In order to validate the principal working of the two-legged musculoskeletal model (see section +\ref{subsec:two_legged_model}) and to test the corresponding controller (see section +\ref{subsec:locomotor_primitives_controller}), random data resembling muscle activation patterns was generated +for all the 32 muscles of the model. In the initial version, the data contained discontinuities and the +activation values were generally too high. As a consequence, the simulation of the model took very long or in +some situations didn't complete at all. -Problems with random data: discontinuities, not fitted to muscle model and musculoskeletal model. Simulation time thus very high (or even unable to simulate). Smoothing of data needed, smaller activations. Works, but leads to unrealistic behavior. +Thus the script used to generate the data was adjusted in order to smoothen the data and provide smaller +activation values. With this second set of data, the simulation succeeded, but of course showed unrealistic +behavior. Since the simulation now completed with any type of generated pattern, the model and controller were +considered ready for usage with the real data. -\section{Locomotor Primitives Study Data} +\section{Locomotor Primitive Patterns in Two-Legged Model} -\TODO{Add example plot} +Only results for an adult subject was obtained using the locomotor primitive patterns (rightmost two columns in figure \ref{fig:dominici_patterns_graphs}). The simulation was run for $1.5$, $2.0$, $2.5$ and $3.0$ seconds respectively. The locomotor patterns were scaled to the according timespan. The visualization of a part of the simulation run is shown in figure \ref{fig:locomotor_primitives_sequence}. -\TODO{Add example image sequence of simulation} \ No newline at end of file +It can be seen from the sequence that the muscle activations lead to an unrealistic movement of the legs (knees +bending backwards, twisting of the feet). Part of the effect can be attributed to the improper scaling of the +muscle activations. Thus different scalings factors were tried ($0.25$, $0.33$, $0.5$, $0.66$ and $0.75$) but +with all of them the basic improper movement pattern remained. + +\begin{figure}[htb!] +\centering +\includegraphics[width=\textwidth]{images/locomotor_primitives_sequence.png} +\caption{Sequence of the two-legged \emph{LocomotorPrimitives} model. Only the first $1.2$ seconds are + visualized. Total duration of simulation: $3.0$ seconds.} +\label{fig:locomotor_primitives_sequence} +\end{figure} + +When looking at the exerted muscle forces for a simulation run with the muscle activations scaled by $0.25$ +(figure \ref{fig:locomotor_primitives_reported_forces}), the activations are only partially reflected. For the +\emph{Biceps Femoris-Long Head} and the \emph{Semimembranosus} (which are both part of the hamstring muscle +group, see section \ref{subsec:two_legged_model}) the reported forces match the muscle activation patterns (see +figure \ref{fig:dominici_patterns_graphs}) quite closely, while in the case of the \emph{Soleus} and the +\emph{Rectus Femoris}, the patterns don't directly reflect the activations. Overall the exerted forces don't +show unrealistically high values. + +\begin{figure}[htb!] +\centering +\includegraphics[width=\textwidth]{images/locomotor_primitives_reported_forces.png} +\caption{Muscle forces exerted on the model during simulation for the muscles \emph{Soleus (soleus)}, + \emph{Rectus Femoris (rect\_fem)}, \emph{Semimembranosus (semimem)} and \emph{Biceps Femoris-Long Head + (bifemlh)} in both legs. X-axis is time from $0.0$--$3.0$ seconds, Y-axis is force in $N$.} +\label{fig:locomotor_primitives_reported_forces} +\end{figure} \ No newline at end of file diff --git a/doc/title.tex b/doc/title.tex index 966e7f9..43565a4 100644 --- a/doc/title.tex +++ b/doc/title.tex @@ -9,12 +9,12 @@ \begin{center} \includegraphics[width=0.6\textwidth]{images/uzh_ifi_logo_e_pos.pdf}\\ - \vspace*{1.5ex} +% \vspace*{1.5ex} \ \\ - Artificial Intelligence Laboratory +% Artificial Intelligence Laboratory +% \ \\ \ \\ - \ \\ - \vspace*{1.5ex} +% \vspace*{1.5ex} \HRule \vspace*{10ex} @@ -65,9 +65,14 @@ % \end{minipage} Tobias Klauser \\ + Winterthur, Switzerland \\ Registration Number: 05-535-927 \\ + + \vspace*{4ex} + + Written at the Artificial Intelligence Laboratory \\ - \vspace*{8ex} + \vspace*{4ex} \emph{Supervized by:} \\ Prof. Dr. Rolf Pfeifer \\ -- cgit v1.2.3-54-g00ecf