InventoryManager: Export Inventory Data
Overview
An important function of InventoryManager is to enable access to inventory data in a range of formats. This supports integration with other forest information systems as well as analysis using custom tools and processes. When enabled, export of inventory data will typically be accessed from a tap on the sub-menu labeled ‘Export’ or ‘Cruise’. Inventory Export supports two file formats (Excel and SQLite) and is highly configurable via range of options and an export configuration file. Some organizations use multiple export configuration files in order to tailor data exports for different uses (e.g., one for custom data quality assessment in Excel, another for export to FVS for growth and yield analysis, and a third for export to TCruise for compilation and reporting).
The general process for Inventory Export is to select a set of stands to be exported, select an export configuration file, configure other aspects of the export (including how to handle data stored in fields with coded value domains, selection of target systems like FVS or TCruise, etc.), then run the export. The Inventory Export process begins by exporting the selected stands, then it exports all plots within those stands, then all trees within those plots and where appropriate, all logs within those trees.
Selecting Data for Export
The Inventory Export process typically begins in the Map by searching for and selecting a set of Stands to be exported. Typical scenarios include searching for all stands that have a status of ‘Cruised’ or ‘Checked’, with the possible addition of spatial parameters like a specific property or region. See https://masonbruce.atlassian.net/wiki/spaces/MD/pages/307953750 for more information on searching for Stands to export.
Export Configuration File
To export cruise data, an export configuration file stored in CSV format (*.csv) is used to translate and map attribute fields from the cruise data model to the data model required by the compilation software. Use the Choose File button to browse your system and load a config file. The config file can be created in Excel and is a lookup table that matches the tables and fields of the Esri data model with the desired output tabs and fields in the exported Excel file.
The columns in the config CSV file are as follows:
table - the name of the table (spatial or attribute) in the Esri data model.
index - the order the field should be outputed in the exported file (1-n).
esriName - the name of the field in the Esri data model. Note that this should be the full Esri fieldname using correct case, underscores etc., not the field’s ‘alias’
exportName - the name of the field in the output Excel file.
defaultValue - a default value which will be written if the incoming field does not exist or there is no value on an incoming record.
dataType - one of TEXT,INT,DOUBLE indicating the data type. This field is optional, and is currently only used when exporting for TCruise. Use INT for integer or date fields, use double for decimal fields (floating point, double precision floating point) and TEXT for everything else.
The config file enables the following functionality:
Excluding fields - if a field is present in the Esri data model which is not desired in the export (e.g., OBJECTID) it can be left out of the config file and will not be exported.
Reordering fields - if fields in the config file are listed in a different order than in the Esri data model, they will be exported in the order of they are listed in the config file, not the order they occur in the data model.
Renaming fields - the export field name is independent from the Esri field name, thus a field named ‘DBH’ can be exported as ‘diameter’.
Adding fields - if a field is not present in the Esri data model, but is needed in the export file as an input to another process, it can be inserted in the desired field order in the export file. If a default value is provided for that field, that value will be inserted for all output records. An example might be the field ‘Measurement Year’, which could be included at the Stand level in the export configuration file with a default value set to the current year. In this scenario, the Stands tab in the exported Excel file would contain a column called ‘Measurement Year’ and each Stand record would contain the current year in that column, even though that field the Esri data model.
Removing fields - if a field is in the Esri data model but you do not wish to include it in the export, simply omit the entire row from the export file.
Export Functions
To support specific data manipulation tasks during export, InventoryManager Export includes a small number of functions that can be run during the export to transform data into the desired format. The table below lists the available functions. In each case, the function would be entered in the ‘esriName’ column of the export config to indicated that the source is a function rather than simply copying the value from an existing field in the ArcGIS data model.
Function | Description | Example |
---|---|---|
ADD | Adds the values of 2 fields together. If one or both fields are text then fields are concatenated. | ADD(Inventory_Year+Measurement_Frequency) - This example will add the values of Inventory_Year and Measurement_Frequency to calculate the year for the next inventory. |
AUTOINCREMENT | Auto-numbers output rows | AUTOINCREMENT - No parameters or parenthesis needed |
BASALAREA | Calculates the Basal Area of a tree (in square feet) using the equation (DBH)^2 * 0.005454154; | BASALAREA(DBH) - could calculate the Basal Area of a tree. Assumes that DBH is in inches and BA is desired in square feet. |
DIVIDE | Divide the value of one field by the value of another field | DIVIDE(Total_Height,DBH) - this example could calculate the height to diameter ratio for each tree to be used to detect outliers. |
EXCLUDE | Exclude records based on some attribute value. Note, this has only been implemented for Trees so far. | EXCLUDE(Cut_Leave:Cut) - this example will exclude any trees for which the ‘Cut_Leave’ field contains a value of ‘Cut’ |
FVSGROUPS | Calculate the ‘Groups’ field in the ‘FVS_StandInit' table of the FVS SQLite database export. The groups value will always contain the phrase 'All_Stands’. Adding this function can include additional groups based on hard-coded values or the values of any field(s) | FVSGROUPS(variant=SN;Cover_Type={Cover_Type};) - this example would include the following groups:
|
LOOKUP | Lookup values from list - if the export value should come from a list of allowed values, but that list does not match the Coded Value Domain list for that field, a custom lookup table can be provided in the config file. | LOOKUP(Cruise_Type:S=0;C=1) - This example will transform the values of the field ‘Cruise_Type’ in the Trees table (with domain values of S for standard cruise and C for check cruise) needs to match the values needed in the RECTYP field used by TCruise. It will replace all instances of ‘S' with the value 0, and all instances of 'C' with the value 1. LOOKUP(Length:0.5=8;1=16;1.5=24;) - This example could be used to translate between log lengths recorded in 1/2 logs (e.g., 0.5, 1, 1.5,…) to feet (e.g., 8, 16, 24,…). |
MAX | Return the maximum value of the selected attribute values | MAX(Defect_1;Defect_2;Defect_3) - this example could calculate the maximum amount of defect when tree defect is recorded per third of each tree. |
MIN | Return the minimum value of the selected attribute values | MIN(Reference_Tree_Distance_1;Reference_Tree_Distance_2) - this example could calculate the distance to the closest reference tree for a plot. |
MULTIPLY | Multiply the values of two fields together | MULTIPLY(Plot_Length,Plot_Width) - this example could calculate the area of a rectangular plot in square feet when the Plot_Length field and the Plot_Width fields are entered by a cruiser in feet. |
PLOTS. | Lookup value from an existing or new field in the Plots layer | PLOT.Plot_Size - this example could populate a Strata field in an output Plots table with the Strata value of that Plot’s parent Stand. |
POWER | Calculate the power (exponent) of an attribute value | POWER(Plot_Width;2) - this example could calculate the area of a square plot when the plot width is stored in an attribute field. Note that this would not convert units, so this would only be appropriate if the width was in Feet and you want the area in Square Feet. |
SQRT | Calculate the square root of of an attribute value | SQRT(Area) - this example could calculate the width of a square plot when the plot area is stored in an attribute field. Note that this would not convert units, so this would only be appropriate if the area was in Square Feet and you want the width in Feet. |
STANDS. | Lookup value from an existing or new field in the Stands layer | STANDS.Strata - this example could populate a Strata field in an output Plots table with the Strata value of that Plot’s parent Stand. |
SUBTRACT | Subtract the values of 1 field from the value of another field. | SUBTRACT(Inventory_Year-Origin_Year) - this example could calculate a stand ‘Age’ field as the origin subtracted from the cruise year. |
TREES. | Lookup value from an existing or new field in the Trees layer | TREES.Species- this example could populate a Species field in an output Trees table with the Species value of that Log’s parent Tree. |
YEAR | Return the 4 character year value for a date that is stored using an Esri Date field (unix timestamp) | YEAR(Cruise_Date) - this example would return the inventory year when Cruise_Date field stores an Esri Date value for the date of the cruise (typically at Stand or Plot level) Note this is just the calendar year, not a tree growth year. |
Stands To Export
In addition to the original selection of stands described above, the stands to be exported can be further constrained using the Stands to Export setting on the Inventory Export tab.
Choose the category of stands to be exported from the drop-down menu.
Selected stand(s) - will export all of the stands that are currently selected in the map.
Cruised stands - will only select those stands where the status field has been set to cruised (requires that the system has been properly configured to identify the status attribute field and the values that qualify as cruised (e.g., ‘Cruised’ and ‘Checked’)
All cruised (not checked) stands - similar to above, but will omit stands that have been checked
All checked stands - similar to above but includes only those stands that have been checked
Stands updated on/after… - will apply a starting date constraint on the last_modified_date field in order to only export the most recent data. An example of this would be if you need to export only those stands that were updated in the last 2 weeks. When using this option the date field below must be set to a valid date.
Export Codes Or Descriptions
For attribute fields that use coded value domains, it is possible to export either the codes or their descriptions. For example, if the species field uses 2-letter species codes, selecting Export codes would export the code ‘DF’ for Douglas fir, while selecting Export descriptions would export ‘Douglas fir’. This is a global setting and thus will use either codes or descriptions for all fields for all tables. It is typical to use descriptions when performing ad hoc review as they may be easier to interpret when scanning for data quality issues. When exporting data for other forest information systems, however, it is often necessary to export the codes for compatibility.
Calculate Check Cruise Score
Check cruise score calculation can be performed during the Inventory Export process. Check cruise score calculation requires that a Check_Rules table is present in the data model and is populated with one or more check cruise score rules. These rules define the quality control tests that should be applied and the point deductions that should be calculated. For more information on creating Check Cruise Rules, see InventoryManager: Check Cruise Rules. For an overview
Below are some example check cruise score rules, written in a human readable format:
If species is incorrect, deduct 5 points
If the total height is off by more than 15%, deduct 3 points
If plot was not properly flagged, deduct 2 points
When check cruise score calculation is applied, the general logic is as follows:
Score calculations occur at the tree and plot levels
Tree level calculations are performed first
For each tree, the sum of its deductions is recorded, along with a comma separated list of tree rules that were violated
Tree deductions for all trees on a plot are summed together as one component of the plot deductions
Additional plot deductions are applied if plot level check score rules are violated
The final set of plot deductions (plot level + sum of tree level) and subtracted from 100 to achieve a plot score
For each plot, the plot score is recorded, along with a comma separated list of tree and plot rules that were violated
The plot level score is written back to the Plots table to the fields ‘Check_Score’ and ‘Check_Details’ (these fields must be included in the Plots table for this functionality
To enable check cruise score calculating during export, check the Calculate check cruise score checkbox prior to clicking Export. When the export completes, if data were successfully written back to the Plots table a notification window will display the results of this process.
FVS Export
The Forest Vegetation Simulator (FVS) is a free software application developed by the US Forest Service. It is commonly used by foresters to perform growth and yield modeling, but can also be used to model wildfire fuel or carbon. The most recent releases of FVS use a web-based interface, rather than the older desktop Suppose application, and imports data via a SQLite database file.
Inventory Export supports export of data for FVS by writing out an FVS compatible SQLite database file. In order for to produce a valid FVS export, however, it is critical to use an export configuration file that maps between the Esri data model fields and the FVS expected fields. Successful creation of an export configuration file for FVS requires knowledge of FVS inputs and your Esri data model. To help support the creation of export configuration files, MB&G can provide example files that have been used by others for FVS data export.
table | index | esriName | exportName | defaultValue |
Stands | 1 | Stand_ID | Stand_ID | 0 |
Stands | 2 | Stand_ID | Stand_CN | 0 |
Stands | 3 |
| Variant | SN |
Stands | 4 |
| Inv_Year | 2019 |
Stands | 5 | FVSGROUPS(variant=SN;Cover_Type={Cover_Type};) | Groups |
|
Stands | 6 |
| AddFiles |
|
Stands | 7 |
| FVSKeywords |
|
Stands | 8 |
| Region |
|
Stands | 9 |
| Forest |
|
Stands | 10 |
| District |
|
Stands | 11 |
| Compartment |
|
Stands | 12 |
| Location |
|
Stands | 13 | SUBTRACT(Inventory_Year-Origin_Year) | Age |
|
Stands | 14 |
| Aspect |
|
Stands | 15 |
| Slope |
|
Stands | 16 |
| ElevFt |
|
Stands | 17 | LOOKUP(Saw_Plot_Size:BAF10=10;BAF20=20) | Basal_Area_Factor |
|
Stands | 18 |
| Inv_Plot_Size | 100 |
Stands | 19 |
| Brk_DBH | 5 |
Stands | 20 |
| Num_Plots | 0 |
Stands | 21 |
| NonStk_Plots |
|
Stands | 22 | Acres | Sam_Wt |
|
Stands | 23 |
| Max_BA |
|
Stands | 24 |
| Max_SDI |
|
Stands | 25 |
| Site_Species |
|
Stands | 26 |
| Site_Index |
|
Stands | 27 |
| State | 37 |
Stands | 28 |
| County |
|
Stands | 29 |
| Fuel_Model |
|
Stands | 30 |
| Photo_Ref |
|
Stands | 31 |
| Photo_code |
|
Plots | 1 | Stand_ID | Stand_ID | 0 |
Plots | 2 | Stand_ID | Stand_CN | 0 |
Plots | 3 |
| StandPlot_ID | 0 |
Plots | 4 |
| StandPlot_CN | 0 |
Plots | 5 | Plot_ID | Plot_ID |
|
Plots | 6 |
| Variant | SN |
Plots | 7 |
| Inv_Year | 2014 |
Plots | 8 |
| Groups | All_Stands Variant=SN |
Plots | 9 |
| AddFiles |
|
Plots | 10 |
| FVSKeywords |
|
Plots | 11 | Latitude | Latitude |
|
Plots | 12 | Longitude | Longitude |
|
Plots | 13 | STANDS.Age | Age |
|
Plots | 14 |
| Aspect |
|
Plots | 15 |
| Slope |
|
Plots | 16 | Elevation | ElevFt |
|
Plots | 17 | LOOKUP(Saw_Plot_Size:BAF10=10;BAF20=20) | Basal_Area_Factor |
|
Plots | 18 |
| Inv_Plot_Size | 100 |
Plots | 19 |
| Brk_DBH | 5 |
Plots | 20 |
| Num_Plots |
|
Plots | 21 |
| NonStk_Plots |
|
Plots | 22 |
| Max_BA |
|
Plots | 23 |
| Max_SDI |
|
Plots | 24 |
| Site_Species |
|
Plots | 25 |
| Site_Index |
|
Plots | 26 |
| State | 37 |
Plots | 27 |
| County |
|
Plots | 28 |
| Fuel_Model |
|
Plots | 29 |
| Photo_Ref |
|
Plots | 30 |
| Photo_code |
|
Trees | 1 | EXCLUDE(Cut_Leave:1) | NA |
|
Trees | 2 | Stand_ID | Stand_ID | 0 |
Trees | 3 | Stand_ID | Stand_CN | 0 |
Trees | 4 |
| StandPlot_ID | 0 |
Trees | 5 |
| StandPlot_CN | 0 |
Trees | 6 | Plot_ID | Plot_ID | 0 |
Trees | 7 | Tree_ID | Tree_ID | 0 |
Trees | 8 | Tally | Tree_Count |
|
Trees | 9 |
| History |
|
Trees | 10 | Species | Species |
|
Trees | 11 | DBH | DBH |
|
Trees | 12 |
| DG |
|
Trees | 13 | Total_Height | Ht |
|
Trees | 14 | Crown_Ratio | CrRatio |
|
Trees | 15 |
| Damage1 | 25 |
Trees | 16 |
| Severity1 | 10 |
Trees | 17 |
| Damage2 |
|
Trees | 18 |
| Severity2 |
|
Trees | 19 |
| TreeValue |
|
Trees | 20 | Tree_Age | Age |
|
Trees | 21 |
| Slope |
|
Trees | 22 |
| Aspect |
|
When exporting data for FVS, use the appropriate export configuration file and check the box labeled Export for FVS (*.db).
This will produce a SQLite database suitable for use in the new version of FVS. The basic workflow for using this file within FVS is outlined below:
Open FVS (web-based version, not Suppose. Application will open in browser using 'loopback' URL 127.0.0.1)
Import Data Tab (upper left)
Browse for file
Install uploaded database
Runs Tab (upper left)
Click ‘New’ button
On ‘Groups’ tab, select ‘All_Stands’ item from list, then click ‘Add stands in selected groups’ button below lists
Time Tab (mid right)
Select start, stop and interval
Select Outputs Tab
Check boxes for ‘Tree Lists’ and ‘Inventory Statistics’ (and others for which you have input data, e.g., carbon, fuels, etc.)
*Run* Tab
Click ‘Save and Run’
Review output chart
Review units & values
Modify data, settings, etc. as needed
Example results in FVS for 5 Southern Loblolly Pine stands exported from InventoryManager using the Export for FVS option.
TCruise Export
Similar to FVS Export, InventoryManager can export for use in TCruise Pro desktop compilation software. It is necessary for TCruise Export to be enabled in InventoryManager, and for a TCruise ‘Template’ (*.tct) file to be stored in InventoryManager. Once these have been implemented, an export configuration file can be used to provide the appropriate information for populating a TCruise-compatible SQLite database file which includes the following tables: Tract, Strata, Plots, Trees.
The standard TCruise export uses the Stand_ID field as the STRATUMID that is stored in the Strata and Plots table of the export SQLite file. This results in 1 strata per Stand. To export a stratified cruise, in which stands are grouped into strata for analysis in TCruise, the ‘esriName’ for the output STRATUMID should be a fieldname that will have the same value for all Stands in a given strata. For example, if Stands have a Stand_ID and a Stratum_ID field and you want to perform a Stand-level analysis in TCruise, then the ‘esriName’ for the STRATUMID would be ‘Stand_ID’. For a stratified cruise, the ‘esriName’ for the Stand-level STRATUMID field would be ‘Stratum_ID' while the ‘esriName’ for the Plot-level STRATUMID field would be 'STANDS.Stratum_ID’ indicating that the plots STRATUMID should be populated by the Stratum_ID field found in the parent Stand.