The IO Module in CREDO is for Input/Output to various forms of data important for analysis and testing of StGermain codes.
These are primarily related to the core data formats that StG-based codes such as Underworld:
This module is for accessing, manipulating and writing out XML files stored in the StGermain data format.
It currently does not create a new Object-oriented representation of XML docs themselves, but most functions operate on an ElementTree object representative of an element in an open Stg XML document.
Simple utility func to add the Namespace prefix that LXML adds to element tag names when it parses in from a file
Flatten a list of provided XML files and optionally also cmdLineOverrides (string), using the StGermain FlattenXML tool.
| Returns: | the file name of the newly created flattened file. |
|---|
Create a new empty StGermain model XML file (can be merged with other model files).
| Returns: | a tuple of the new xml doc (Element Tree), and the root node of the new doc. |
|---|
Checks the “type” of a StGermain data node element. Here, we deal with 3 possibilities:
Gets the node at a current context. If context is a struct, treats nodeSpecStr as a named element. If context is a list, nodeSpecStr must be a list specifier.
Returns the element node (in etree form) of a particular list element that’s a child of the given elNode with given listName. If a node with the given name not found, returns none.
From a given specification of a node in a StGermain model file (eg plugins[0].Context), return the element to operate on.
Returns the element node (in etree form) of a particular Param parameter that’s a child of the given elNode with given paramName. If a node with the given name not found, returns none.
Gets the value of a parameter from a StGermain XML model file that’s a child of the given elNode, with the given paramName. The value is cast according to the castFunc, which should be a standard Python casting function, e.g. ‘int’, ‘double’ or ‘bool’.
Returns the element node (in etree form) of a particular struct element that’s a child of the given elNode with given structName. If a node with the given name not found, returns none.
Indent an XML file in xml.etree format, useful for pretty printing.
Credit to http://infix.se/2007/02/06/gentlemen-indent-your-xml
Navigate a document, based on a remaining StGermain command-line style element specification. Returns currNode, nodeName of the final entry in the hierarchy
Simple utility func to remove the Namespace prefix that LXML adds to element tag names when it parses in from a file
Set the “MergeType” of an XML node: usually for writing new nodes in an XML designed to over-ride existing XML of a model.
Converts a string (eg from a param in XML) to a Python Bool and returns this, using same idiom as in StGermain.
(See Dictionary_Entry_Value_AsBool() in Dictionary_Entry_Value.c in StGermain/Base/IO).
Write XML to merge a given component to the components list - and return new comp elt
Writes a particular parameter, with name of paramName, val of paramVal, to the open XML file at position specified by parentNode.
Write a Stg XML List structure, made up purely of (unnamed) parameters
Writes a set of parameters, with name:value mappings as specified by paramsDict, to the open XML file at position specified by parentNode.
Write a given StGermain xmlDoc to the file given by filename
Necessary because the xml.etree doesn’t include a pretty_print functionality by default (unlike lxml)
This module is for properly formatting StGermain command line operations.
Return the command line string to override a certain parameter path and value.
Return the cmd line string to apply a given PETSc solver options file.
A Module for convenient access to StGermain FrequentOutput files.
Primary construct is the FreqOutput class, which once constructed has numerous methods to access data from a FrequentOutput file.
A simple class to store information about a frequent output file, and make it accessible. Once passed a filename of a FrequentOutput file in it’s constructor, has methods to get information and values from that named file.
The FrequentOutput file can be either “cached” into memory using the populateFromFile() function so subsequent access is quick, or else calling an access function directly such as getRecordDictAtStep() will automatically populate the data cache for you behind the scenes.
Key attributes:
Bool, states whether the class’s data structures have been populated from file, shouldn’t be modified externally.
dict, mapping header names in the FrequentOutput to the column number they occupy in the file.
Returns the highest timestep number that has information recorded for it in the associated FrequentOutput file.
Get all records of timestep information from the associated FrequentOutput file.
Records are stored in an array, where each entry is of the form of a list of floats of the records at that timestep. Thus likely needs to be used in conjunction with other functions to access the data itself by header name, ie the self.headerColMap.
Saves this as self.records, and returns a reference to it. (Also populates the self._finalTimeStep attribute.)
Gets the closest value and timestep to the given value.
Get the column number in the record data structure/FrequentOutput file itself - associated with a particular header name.
Utility function for doing comparison operations on the records list, e.g. the max or minimum - where cmpFunc is a single operator
Read the headers from the associated FrequentOutput file, populate attr:headerColMap, and return the names of the headers as a list.
get the Maximum of the records for a given header, including the timestep at which that minimum occured.
gets the Mean of the records for a given header.
Note
this is provided for convenience. If user wants to do more complex statistical operations, use the getValuesArray, then process this directly using stats functions/libraries.
get the Minimum of the records for a given header, including the timestep at which that minimum occured.
Gets the record (in raw form, see getAllRecords) at a given timestep, and returns.
Utility wrapper function to get a dictionary of records in the FreqOutput at the final timestep - see getRecordDictAtStep .
For a given timestep tstep, looks up the records for that timestep and returns a dictionary of “headername”:recordVal mappings, where headername is the name of each header in the FrequentOutput file, and recordVal is the value of that property at the requested timestep.
Gets the record number in the FrequentOutput file of a given timestep. E.g. in a FrequentOutput file where values were saved every 5 timesteps, then the 15th timestep will map to the 3rd record.
Utility function for doing comparison operations on the records list, e.g. the max or minimum - where reduceFunc can operate on the whole records list at once, and support the “key” syntax to pick correct field out of tuples for comparison.
Note
This has been written to allow both standard Python ‘reduction ops’ like max() and min(), and also more complex operators defined in this module, or by the user.
Returns an array of all timestep numbers that have records saved in the associated FrequentOutput file.
Note
the “range” parameter is not yet operational and should be ignored for now.
Calculate a map of timestep number of model, to record number in the FrequentOutput file - stores this, and returns a reference to it.
This is important especially if the FrequentOutput has been sampled from the model at a timstep frequency less than 1.
Gets the values of a property given by ‘headerName’, at a specified timestep ‘tstep’.
Returns an array of all values over time for the property defined by “headerName” in the associated FrequentOutput file.
Note
the “range” parameter is not yet operational and should be ignored for now.
Plot the value of property given by ‘headerName’, against parameter ‘depName’, which defaults to ‘Timestep’.
Note
Use of this function requires the Python library matplotlib to be installed.
The argument “show” enables whether the graph is to be immediately shown interactively, and “save” if it should be saved. If “save” is true, the “path” parameter determines the path the resulting plot file will be saved under.
This function will read all essential data from the FrequentOutput file associated with the class into data structures in memory, for fast subsequent access. Saves the fact that this has occurred so it doesn’t neeed to be repeated in future.
Print the maximum and minimum values of all fields in the frequent output.
Utilitiy function for use with attr:~.FreqOutput.getReductionOp: Gets the value at a given timestep.
| Parameters: | target – the target simulation time. |
|---|
Utilitiy function for use with attr:~.FreqOutput.getReductionOp: Gets the value at a given timestep.
| Parameters: | target – the target timestep. |
|---|
Utilitiy function for use with attr:~.FreqOutput.getReductionOp: Gets the entry where the value of chosen observable (eg VRMS) is closest to a target value.
| Parameters: |
|
|---|
A utility function designed to pass to attr:~.FreqOutput.getReductionOp for getting the first value from a frequent output list.
A utility function designed to pass to attr:~.FreqOutput.getReductionOp for getting the last value from a frequent output list.
A Module for dealing with StGermain “Convergence” files, i.e. records of comparison between a set of Fields and either reference or Analytic solutions, produced by the FieldTester Component.
The module is not fully object-oriented, and in the current design allows for the possibility of a set of cvg files in one path that should all be referenced. (e.g. the see function genConvergenceFileIndex()).
It provides a similar, though not identical, interface to the credo.io.stgfreq module.
The format of CVG files is space-separated, of the form:
Header lines, which may be repeated throughout the file:
#Res FieldName1 FieldName2 ...
Values lines, which are of the form:
<len scale> <field value 1> <field value 2>
For example:
#Res TemperatureField1 TemperatureField2
1.000000e-01 6.16e-03 5.5e-2
#Res TemperatureField1 TemperatureField2
1.000000e-01 6.14e-03 5.4e-2
#Res TemperatureField1 TemperatureField2
1.000000e-01 6.12235812e-03 5.3e-2
Bases: exceptions.IOError
An exception for specifying problems reading an Underworld convergence file.
A simple class to store info about what fields map to what convergence files. Currently implicit is the name of the Field, this is usually handled by storing CvgFileInfos in a dictionary, with the keys being Field names.
The filename (as a string) that the cvg info is stored in.
A dictionary mapping for each degree of freedom of a field, the column number it is stored in in the cvg file.
Returns a dictionary relating field names to CvgFileInfo classes, after reading all .cvg files in the given path.
Checks that “steps” specified is valid for a given cvgFile (Python File), and if so converts it into a list of step numbers within the range specified. into a tuple of start and end values. If steps isn’t valid, raises assertion.
“steps” can be of the format:
For the given CvgFileInfo and step number, returns a list indexed by dof number of the error of each dof in that step.
For a given cvgFileInfo, get the errors in the specified dof from the specified file, indexed primarily by Dof. The ‘steps’ arg can be either ‘all’ (for all timesteps), ‘last’, or a tuple specifying range (see getCheckStepsRange() for more). If only one step result is asked for, the dofs are returned as a simple 1D array, otherwise they’re returned as a list.
For a given CvgFileInfo, return the errors in the cvgFileInfo’s specified file - for the timestep range specified by steps, indexed primarily by Timestep. The steps arg can be either ‘all’ (for all timesteps), ‘last’, or a tuple specifying range (see getCheckStepsRange() for more). If only one step result is asked for, the dofs are returned as a simple 1D array.
For a given cvg Filename, return the ‘resolutions’ (length scale) for the given set of steps, where “steps” is of the form documented in getCheckStepsRange().
This module is for accessing and working with StGermain-related paths.
Checks a whole set of XML input files exist, and raises an IOError if one of them doesn’t. See checkXMLInputFileExists().
Check if an XML input file exists in either the standard XML path, or relative to the current directory. Raises an IOError if not.
Check through the given input file list, and for any that aren’t found relative to either the local directory or the StGermain standard path, convert them to be relative to the given callingPath
| Returns: | new list of adjusted XML files |
|---|
Get the path of StGermain binaries (given by env variable STG_BINDIR).
Returns the path that StGermain standard XML files are stored in once installed (and is automatically searched within by StGermain when input files are specified on either the command line, or in include statements).
For a given executable name (eg “StGermain”), return the full path name of that executable (in the path given by the STG_BINDIR env variable).
return the full path to the main StGermain executable (in the path given by the STG_BINDIR env variable)
Move all files with extension fileExt from startPath to targetPath - automatically over-writing any existing files with identical names.