Nimue platform

Diese Seite ist nicht editierbar. Sie können den Quelltext sehen, jedoch nicht verändern. Kontaktieren Sie den Administrator, wenn Sie glauben, dass hier ein Fehler vorliegt.
======Process Configuration======
=====General features=====
This component arranges the processing of multiple measurement trials by the [[modellingcomponents:mathengine|Math Engine]] based on a [[modellingcomponents:mathengine#model_description|Model description]]. Typically data trial are saved in a single folder, in the filesystem. For each trial, more than one file, with the same name but different suffixes can exist. 

E.g. there is a 001.c3d file, which includes timeseries of marker positions of a 3d motion analysis system, a 001.mpg file with a synchronous measured video, 001.enf (Vicon file type) with meta data and so on. The file types are free configurable.

How this session of trials is processed, is defined by the **XML Process Configuration file**. The principle processing arrangement steps are the following:

  - The trials are grouped based on arbitray information readable from the files of each trial. E.g. in the case of a Vicon session the .enf file includes meta data, which can be used to descide if a trial is a calibration trial or a motion trial. Also the file names themselfes can be used as a criterium.
  - The session can include special files with parameters so called "model parameter files", which are used by the model for processing of the trials, e.g these files can include subject specific anthropometric data like foot length,  age of a patient or what ever. These parameters are called Model Parameters. From which files in which formats model parameters are loaded depends on the used iTrialSet-Implementation. Different implementations exist, e.g. for Sessions determined by Vicon Workstation.
  - A labelset XML file is referenced for each group, which defines which timeseries must be read to process.
  - A model XML file must be referenced for each group, which defines the calculation steps. This results in new calculated timeseries.
  - Switches can be defined to switch on/off different calculation steps defined in the model, depending on the processed trial. These switches are independed from the group definition for maximum flexibility.
  - A labelset for the output must be defined for each group. This defines, which timeseries and single parameters should be written to an output file. Also the file format of the output file can be configured. And it is possible to create more than one output file for each trial or also only one output file for all of the (input) trials. Specialized Output handling can be plugged in by implementation of so called OutputHandler.

All of the configurations that are needed for processing sessions of trials is bundled to a so called project. An application can include one or more fix such projects or it can be modelling tool for user defined projects.

===== Project folder structure =====
A project corresponds to a folder with subfolders, which include different configuration files inside. Also the output of the processing can be saved inside this project folder structure.

^ sub-folder name ^ Description |
^ cmlproject | Internally used to save project configurations in a file named as cmlproject.properties (.properties file format) and to recognize the folder as Nimue projects folder. This folder is typically not shown to the user. |
^ datasets | One or more files which includes a set of paths to the sessions to process. One file is defined as main dataset file. |
^ jobs | One or more process configuration files. One file is defined as main process configuration file. |
^ models | One or more model description XML files. |
^ labelsets | One or more XML labelset configuration files |
^ constants | One or more files with constants referenced by the model. (.properties file format). |
^ output | Files which include the output of the processing, typically in the .d3d file format. |
^ classes | optional folder which can include plugged in output handlers for third-party file formats |
^ mfiles | optional folder which can includes additional matlab function files (.m files) |
^ sheets | One or more XML configuration files to define sheets of plot for different groups of trials. |
^ views | One ore more XML configuration files to define 3d visualizations. |
^ masks | One or more XML configuration files to configure the visualization of the trials meta data. This corresponds to the eclipse database .enf configuration files from Vicon Workstation. |
^ normdata | One or more .d3d-files including normal data for the project. |

In some folder a main file is defined. In the GUI these main files are shown with bold face. 

=====Process Config XML: Default Values=====

The folder **jobs** can include one or more **XML process configuration files**. Always one file is marked as the main file. This file defines all of the processing steps. The main structure of such a file looks as follow:

<code xml>
<?xml version="1.0" encoding="iso-8859-1" ?>

<CalcMLProcessConfig name="UEGeneral2 HUX + Cluster"
                     version="1.0"
                     defaultmodel="UEGeneral2+Cluster_without-SHO-markers.xml"
                     defaultmarkerset="Input.xml"
                     defaultconstants="constants.properties"
                     defaultview="StickFigure.xml"
                     useEventsFromSessionMetaDataFile="false">
...
</CalcMLProcessConfig>

</code>

The root element **<CalcMLProcessConfig>** can include default values for some properties:

^ property name 	^ Description |
^ defaultmodel | Name of a model file which is used, if for a CalcGroup no model file is defined. |
^ defaultmarkerset | Name of markerset file which is used, if for a CalcGroup no markerset file is defined. |
^ defaultconstants | Name of  constants file, which is used, if for a CalcGrop no constants file is defined. |
^ defaultsheet | Name of file with a sheet definition. It is used if data determined by different CalcGroups should be visualized in a sheet. |
^ defaultview4d | Name of file with a view4d definition. It is used, if data determined by different CalcGroups shoulb be visualiszed or a CalcGroup does not define it. |
^ defaultnormaldata | Name of the file with default normal data. For the structure of this file have a look at the explaination of the Process Configuration. |

=====Process Config XML: Groups of trials=====
The <CalcGroup>-element defines a group of trials. One or more groups can be defined. It is recommended to define a minimum of one groups. If no group is explicitly defined, a default group is automatically created which includes all trials.

Attributes of the **<CalcGroup>**-element:

^ name 	^ Default ^ Description  |
^ name | | Name of this group. Do not used any "_" inside a calcgroup name because files with include this character are not shown in the "Data"-Tab. |   
^ sheet | | A Sheet definition which is used to plot sheets of time series of data assigned to this group. |
^ view | | A View 4d defintion which is used to visualize time series of data assigned to this group. | 
^ normaldatagroup | | If normaldata based on an other CalcGroup should be used. If this attribute is not set, it is assumed that the group on his self is the normdata group. | 
^ normaldata | (not implemented yet) | Explicit name of a normal data file if the default is not sufficient. | 
^ calibrationstep | -1 | (not implemented yet) Used if this group is used for calibration. -1 if it is not used. Small numbers, starting with 0, the numbers must be different for different groups. If there is no group with calibrationstep != -1, then single trials can be processes. The defined order cauld be used instead of the implicit order in the <CalcGroups> sequence attribute. Oder besser statt dessen zusätzliche Attribute in <CalcGroups> definieren? ja! calibrationsequence zusätzlich einführen |

    
To define which trials are assigned to a group the element <identification> is used. It is possible to assign a trial to different groups but this is not recommended.

====Identification of trials:====
The <identification>-element includes one or more <entry>-Elements. Each of these elements describe a condition. The key-attribute is the name of a property which is attached to the trial e.g. in the case of Vicon data it is the name of a property saved in the trials corresponding .enf-file. There are some special additional keys which exist independed from the third-partys measurement data files:

The keys and the values are case sensitive. Have a look at the following example:

<code xml>
    <identification>
       <entry key="CLASS">Static_Cal"</entry>
       <entry key="SIDE">bds|links|rechts|Side R|Side L|Both"</entry>
    </identification>
</code>

The propertys value are defined as content of the <entry>-element. It is a regular expression.
A trial is assigned to the group of trials, if the conditions defined by all <entry>-elements are achieved.

====Process Configurations:====
The <process> element allows to declare properties, which can be read to configure the read of input data from specific files. E.g. for reading data from c3d-files the properties EVENTS, FORCES and POINTS are used. Set the property value to "false" to inhibit to read the specific data, e.g

<code xml>
    <process>
     <entry key="EVENTS">false</entry>
     <entry key="FORCES">false/entry>
    </process>
</code>

With the key="inputpath" the input file can be specified for testing purposes:

<code xml>
   <entry key="inputpath">rawdata/ABC000.r3d</entry>
</code>

^ Name ^ Values ^  Description |
^ EVENTS | true or false  | If set to false, no events are read. | 
^ POINTS | true of false | If set to false, no position data is read. |
^ FORCES | true of false | If set to false, not force plat data is read. |
^ inputpath | path to the data set e.g. inside a r3d-file | If set to a path, than this is used to find the input dataset. if the attribute is not used per default the dataCollections are searched in the "rawdata" group.|
^ outputpath | path to the data set e.g. inside a d3d-file | If set to a path, than this path is created to store the dataset. (This functionality is not yet implemented completely in the MotionDataStorageHandler class.) |


====Input:====

Attributes of the **<input>**-element:
^ Attribute ^ Default ^ Description ^ Required |
^ labelset | Labelset as defined with the attribute "inputmarkerset" of the element <CalcMLProcessConfiguration>.| The labelset which defines which parts of the data should be read to be processed. | No |

====Output:====

Attributes of the **<output>**-element:
^ Attribute ^ Default ^ Description ^ Required |
^ labelset | | The labelset which defines which parts of the data should be written to the output and also meta data e.g. the mime type and the file suffix. | Yes |
^ mimetype | (deprecated) | Mimetype to define the output file format. This is set normally inside the labelset file. | No |
^ filesuffix | | suffix of the outputfile normally this is defined inside the labelset (deprecated) | No |
^ dir | | dir of the input session of trials 	Absolute path of the directory, where the outfile should be written. | No |
^ name | | Can be used to define the name of the output file. (currently not used) | No |
^ append | false | If set to true the data is appended to file if is exists. | No |
^ handlerclass | | Use a customer defined output handler class. (changes are needed to be able to use more than one handler class) | No |

If the **<output>**-element is not set, than all time series and parameters are written to a file located in the input trials session folder. If a file exists this file is overwritten (append="false").
        
The **<output>**-element can have children  <code xml><entry key="A">a</entry></code>to control specific properties of the **output handler**.


===MotionDataOutputHandler===
To write output to a file a so called "handlerclass" has to be defined. Such a class implements a so called OutputHandler. The default and most complex OutputHandler is "de.orat.motionDataConverter.MotionDataOutputHandler". It saves so called d3d-files.

The following properties are read as <entry>-elements as children of the surrunding <output>-element:

^ Attribute ^ Default ^ Description |
^ fileName | <session id>.d3d | Output file name inclusive suffix, e.g. "abc.aaa". If the attribute is used the default value is overwritten |
^ overwrite | false | If set to "true" for the first calcgroup configuration only has the effect for before saving the data of the first group an existing d3d-file is deletete before. It makes no sense to use this attribute in combination with seperateGroupFiles="true". |
^ outputDir | The session folder currently processed. | Absolute path to the folder, where to save the output. |
^ trialsGroupName | "singleTrials" | Name of the child group inside the output file, where to save the trials. |
^ renameExistingTimeSeries | false | If set to true, existing data timeseries are renamed instead overwritten with the current ones. |
^ seperateGroupFiles | false | If set to true for every calcgroup a seperate output file is written. The name of the corresponding output files has the format <session id>#<calcgroup name>.d3d. |

If this outputhandler is used for more than one calcgroup, the data of all calcgroups are saved per default into a single d3d-file. For each calcgroup a child group with the name of the calcgroup is created. Inside each of these children a child group with default name "singleTrials" is created and inside this group the data of the single trials are saved. 

===TXTTableOutputHandler===

^ Attribute ^ Default ^ Description |
^ header | | |
^ filename | | |
=====Process Config XML: Model Switches=====
With a switch, the calculation of a modelling element can be switched of or on. One ore more switches can be defined. In the model elements, with the attribute "includes" or "calibrateIncludes" a list of switches can be set. If one or more of these switches are available, the calculation is done. This allows e.g. calculation only for specific trial types.

A on-switch is created only, if all defined keys are available in the trial and if the values of all keys matches the regular expression as defined as content of the <entry>-element.

<code xml>
 <Switches used="static_calibrate dynamic">
      <switch name="static_calibrate">
          <entry key="TRIALCATEGORY">static_calibrate</entry>
      </switch>
      ...
</code>

In the above example the switch "static_calibrate" is set only, if the trial includes a key with the name "TRIALCATEGORY" with the value "static_calibrate". If there are additional <entry>-elements, all of them are tested and only, if all matches, the switch is created.

The content of the <entry>-element is used as a regular expression. E.g. "static_calibrate|static_calibrate2" includes an or-assignment.

=====Process Config XML: Model Parameters=====
=====Process Config XML: Normal Data Definition=====
=====Process Config XML: Example fragment=====

<code xml>
        <?xml version="1.0" encoding="iso-8859-1" ?>
        <!-- DOCTYPE CalcMLConfig PUBLIC "-//CalcML//DTD CalcML 1.0//EN" "http://www.orat.de/CalcML/dtds/calcML-ProcessConfig_1.0.dtd"-->

        <CalcMLProcessConfig name="Test"
                            defaultmodel="codman_model.xml"
                            defaultmarkerset="codman_markerset.xml"
                            defaultconstants="constants.xml"
                            defaultsheet="defaultSheetDef.xml">
         
        <CalcGroups sequence="static_calibrate static_calibrate_shoulder static dynamic">
         
        <!-- Kathegorien fuer Calibrations-Zwecke -->
         
        <CalcGroup name="static_calibrate">
         <identification>
             <entry key="CLASS">Static_Cal"</entry>
             <entry key="SIDE">bds|links|rechts|Side R|Side L|Both"</entry>
         </identification>

         <process>
             <entry key=""></entry>
         </process>

         <input labelset="" 
             eventssource="TRIAL=default|sessionMetaDataFile|trialMetaDataFile"/>
         <output labelset="" append="false">
             <entry key="A">a</entry>
         </output>
         </CalcGroup>
         
         <CalcGroup name="static_calibrate_shoulder">
            <identification>
                <entry key="CLASS">Static_Cal</entry>
                <entry key="SIDE">bds|links|rechts|Side R|Side L|Both</entry>
         </identification>
         </CalcGroup>
         
         ...

         <!--category name="dynamic">
         <identification>
         <CLASS>Dynamic|ShoulderAnte/Retro|ShoulderAbd/Add</CLASS>
         <SIDE>bds|links|rechts|Side R|Side L|Both</SIDE>
         </identification>
         </category-->
         
         <!--category name="static">
         <identification>
         <CLASS>Static|Static_cal</CLASS>
         <SIDE>bds|links|rechts|Side R|Side L|Both</SIDE>
         </identification>
         </category-->
         
         </CalcGroups>
         
         <Switches used="static_calibrate static_calibrate_shoulder left right static dynamic">
         
         <switch name="static_calibrate">
         <entry key="TRIALCATEGORY">static_calibrate</entry>
         </switch>
         <switch name="static_calibrate_shoulder">
         <entry key="TRIALCATEGORY">static_calibrate_shoulder</entry>
         </switch>
         
         <switch name="left">
         <entry key="SIDE">bds|links|Side L|Both</entry>
         </switch>
         <switch name="right">
         <entry key="SIDE">bds|rechts|Side R|Both</entry>
         </switch>
         
         <switch name="static">
         <entry key="TRIALCATEGORY">static</entry>
         </switch>
         <switch name="dynamic">
         <entry key="TRIALCATEGORY">dynamic</entry>
         </switch>
         
         </Switches>
         
         <ModelParameters>
         <input filename="" filetype="" needed="a b c">
         <defaultValues>
         <entry key="abc">100</entry>
         </defaultValues>
         </input>
         <output filename="" filetype=""/>
         </ModelParameters> 
         
        </CalcMLProcessConfig> 
</code>
====Branding====

Some default values of the process configuration can be changed by branding, for a custom specific application:

<code xml>
    <folder name="ProcessConfig">
    	<folder name="defaults">
        		<file name="output">
    			<attr name="mimeType" stringvalue="text/x-hd-asc"/>
      			<attr name="fileSuffix" stringvalue="out"/>
        		</file>
      	</folder>
    </folder>
</code>

=====Labelsets=====
LabelSets are defined in the folder "labelsets" in the projects folder and can be used to select data to read or write.

====General structure====
The structure of a labelset file looks like follow:

<code xml>
<?xml version="1.0" encoding="UTF-8"?>
<LabelSet name="Default Input - Vicon Objects and Markers">
    
  <Labels name="ROS" baseFile="true">

          <Filter name="zero2nan"/>
          
          <!-- Objects -->

          <Label name="ThoraxQ" mathType="QUATERNION_D"/>
          <Label name="ThoraxP" mathType="COLUMN_VECTOR_3D"/>
          ...
  </Labels>
  
  <!-- Mapping of label names after reading or before writing to a file -->
  <LabelsMap>
          <Label from="TargetAngleRPS0" to="Angle0"/>
  </LabelsMap>
  ...
...
</code>

====File formats====
A labelset can be used to define the input or output file format. To do this to the element <Labels> the attributes "mimeType" to define the file format and "fileSuffix" to define the file suffix have to be added. 

<code xml>
<?xml version="1.0" encoding="UTF-8"?>
<LabelSet name="Empty labelset">
  <Labels mimeType="application/motiondata+mat" fileSuffix="mat"/>
</LabelSet>
</code>

The following file formats are available:

^ Mime Type ^ Description  ^
^ application/motiondata+mat | A matlab fileformat. |
^ text/motiondata | default motiondata file format. Used with several suffixes: .d3d, .r3d, .rtde, .v3d ... |
^ text/x-json | json |
^ text/x-csv  | |
====Timeseries properties====
Properties of the timeseries can defined by attributes of the <Label>-elements:


^ Attribute name ^ Description  ^ optional ^
^ name | Name the label | no |
^ mathType | The math type of the referenced data. Default is Vector3d | yes |
^ differentiations | The number of differentations to be calculated. For each differentiation a prefix "d" is added to the timeserie name. | yes |

The following math types are available:

^ Math type name ^ Description  ^
^ REAL | double timeserie |
^ COLUMN_VECTOR_3D | timeserie of Vector3d objects |
^ MATRIX_3X3D | timeserie of Matrix3d objects |
^ QUATERNION_D | timeserie of quaternion objects | 
^ STRING | |
^ CELL_STRING | |
^ COMPLEX | | 
^ INTEGER | |
^ UNKNOWN | | 
^ TYPELESS | |
====Filtering====
The labelset can be used to define a filter chain. Currently this filter chain is used only during loading of trial in the MultiTrialFileWrapper (module EasyCalc) and MultiTrialFileProvider (module MultiTrialFile API).

  - First of all, the filters defind as children-elements of the data labels are executed in its definition sequence. 
  - Second the filters, which are defind as children of a <Labels>-element
  - and last the filters, which are defined as children of the <LabelSet>-element

Independend from the filter chain, calculation of differentiation defined by the "differentiations"-attribut is execute always, if data is saved or loaded.

====Mapping timeseries names====
If timeseries are read from files to be processed or written to files after processing, their names can be mapped to an other name based on the labelset configuration.

If the corresponding label in the labelset defines derivatives, than the derivatives which start with "d" or "dd" for second order are automatically mapped to without an explicit mapping for the derivative. This feature is implemented yet for reading from files with d3d-file-format only.