This topic will be used to report updates to the Observations survey and toolbox. Follow the topic to receive announcements when new versions are released. For more information regarding the toolbox, survey, and Biodiversity Observation Data Standard, see this Solution.

NEWSFLASH: NatureServe  can now clone the (Observation, QC, and paged version) surveys and  related materials (dashboards, map, app) from the NatureServe AGOL  Organization to yours. See this solution for details. This saves tons of time and allows you to hit the ground running - collecting data in the field the same day!

Download/replace all files in the Observations.zip (v 1.4) file (updated 7/17/2025) to ensure you get all updated files. Or, download just the updated Observations Toolbox  (updated 7/17/2025). Following is a list of SOME of the changes:

Documentation:

• Observations_V14.docx: 
  • updated 10/17/2023
    
  • updated 5/22/2023 to fix bug preventing Lifestage, Count, Sex group from displaying appropriately
  • updated 8/2024 - minor clarification
  • updated 11/18/2024 - re: new Dataset input in tool 1
  • updated 4/15/2025 to update syntax for List within Dashboard
  • updated 4/29/2025 to reflect new EST ID mismatch check in Scripts 5 & 6 and new checkbox option in Script 9.
  • updated 6/30/2025 to include new step and query required to run in Biotics prior to running 8th step in Observations Toolbox, as necessitated by a recent upgrade to ArcGIS Pro
• Collect Data via Survey123.docx - updated 8/22/2022 to include information for using offline basemaps

Survey (v 1.4)

• 5/20/2025: removed NULL from manual elevation (copy remnant), hide Recording section except for animals, hide Specimen Collected and hence Specimen Identification section for Communities.
• 4/24/2025: updated to Survey123 v 3.22.49 and updated all "begin repeat" questions within the bind::esri:parameters from "allowUpdates=true" to "query allowUpdates=true". This enables data within repeat sections to be imported into the Inbox. Excluding "query" means the data within repeats would be overwritten.
• Updated to Survey123 v 3.17 - 5/22/2023
• Fixed bug preventing Lifestage, Count, Sex group from displaying appropriately - 5/22/2023
• Location Use Class not displayed if not applicable - previously showing for all animals, even when there was no LUC value to select - 7/11/2022
• Error displayed for duplicate Associated Species, preventing saving. - 7/11/2022
• Available via web app
• Increased performance and ease of updating data lists
• Option to sketch or use vertex draw methods for lines and polygons
• Detected & Dataset fields added to reflect Biotics
• QC fields added for use with Dashboard
• Removed Dataset and Reference sections as they are now accommodated within the Toolbox
• Added fields: elevation (auto-calculated for automatically recorded point location, editable if point location manually edited), specimen collected?, family (hidden), egt_uid (hidden) (updated 5/25/2022) - see this solution if you had a previous version of the survey for details on which fields to add.
• Changed Museum/Herbarium field to select_one dropdown list from open text (updated 6/7/2022)
  

Specimen Labels (Labels_Observations.docm)

• updated 6/7/2022 - altered to include Scientific Name author and use new County/Municipality and State/Province fields.

Toolbox (v 1.4)

• Observations BLP Pro.pyt:
  • updated 11/18/2024: added Dataset input to Script 1 to allow further filtering. Changed from DIG_COM to v_note to record globalid as need to crosswalk back from Visit record (NOT SF) to the Observation record. v_note overwritten with orig_vnote data in subsequent SQL statements.
  • updated 11/29/2023: added Locational Uncertainty Distance Unit as a parameter, increased EO_ID field length, removed additional 1 m buffer
    
  • updated  10/9/2023: FIXED problem, caused by updated version of Pro, causing false report of success resulting in no fields/data being added.
    
• Observations.atbx: 
  • 7/17/2025: Updated error handling in scripts 6 & 9 to check for fields before adding including upper(field)
  • 6/30/2025: A recent upgrade to ArcGIS Pro has stopped support of Oracle 12 (actually, any version below 19). This requires the user to manually query and export to an Excel file prior to running the 8th script within the Observations Toolbox. Both the documentation and script have been updated accordingly.
  • updated 5/22/2025: Script 10: fixed all records being assigned the same VISIT_ID & EO_ID
    
  • updated 5/21/2025: 
    • Script 9: Replaced ' in orig_vnote with '' so SQL statements won't fail; Summarized by EO_ID to prevent duplicate insert statements per ID for reference records
    • Script 10: fix loop to go through all feature classes - had previously just been overwriting portion so only seemed as though it applied to last layer, lines.
  • updated 4/29/2025: added EST ID mismatch check to Scripts 5 & 6 for existing Visit records. Added checkbox to Script 9 to allow user to affect SQL statements
  • updated 4/15/2025: removed parameters and added error handling in a couple tools
  • updated 4/14/2025: fixed EO_NEW table which was not populating OBJECT_ID value
  • updated 3/31/2025: added more error handling to Script 5
  • updated 3/20/2025: fixed Scripts 10, 11 & 12
    
  • updated 3/19/2025: fixed Script 9
    
  • updated 3/17/2025: fixed Script 8
    
  • updated 3/12/2025: fixed Script 5
  • updated 1/30/2025: added error handling to Script 5
    
  • updated 12/10/2024: updated 3 scripts which were failing due to MakeQueryTable giving a RASTER error
  • updated 8/9/2024: writing v_note data to orig_vnote field; populating v_note data with OBJECTID field for joining purposes - allows joining from Visit rather than SF (previously joined via DIG_COM) as can have multiple Visits per SF; updated SQL statements in Script 9 as dataset/global_id not appropriately updated in all instances.
  • updated 4/18/2024: accommodate possibility of not having ALL points, lines & polys; minor bug fixes, empty lists, and adding data to correct sf layer; accommodate use of Enterprise instead of AGOL as survey location.
    
• 1. Create GDBs and Files (updated  8/18/2022): provides option to link to the survey via AGOL directly via the logged in ArcGIS Pro user; if AGOL credentials are provided, a profile is created to store the credentials securely, for subsequent use.
  
• Added 8/18/2022: 12. Update hosted Feature Layer (AGOL) with Biotics IDs (VISIT_ID, SOURCE_FEATUE_ID, EO_ID). This data within the hosted feature layer will clearly communicate whether the data has been loaded into Biotics and if so, with the specific record it is associated. This could be used to create a direct link to the Biotics record, which would of course prompt for a Biotics login.
  • Prompts for which geodatabase will be used to populate the ArcGIS Online (AGOL) feature layer: Observations.gdb for newly processed data or Observations_MASTER.gdb if you have existing data for which there are not IDs in the feature layer in AGOL already. NOTE: running the script using the Observations_MASTER.gdb as the source of data will take longer (more data) and hence should only be run once. Subsequent runs should use the Observations.gdb as only the new/edited data will need to be populated within the AGOL feature layer.
  • Adds new data to the feature layer in AGOL
  • For data which has been edited and hence there could be multiple records, the most recent data according to DESTINATION_DATE is used.
    
• 1. Create GDBs and Files (updated 2/28/2022): improved attachment downloads to include spreadsheets so can easily crosswalk attachments (of any type) to survey response
• 1. Create GDBs and Files (updated 5/25/2022): fixed bug when downloading attachments from zip files. Improved it to include spreadsheets for crosswalking attachments, as available when providing AGOL credentials. Also added calculation of county, state, and QRCode URL for records with specimen_collected = 'Yes'.
• 1. Create GDBs and Files (updated 6/7/2022): changed to feature service which includes US Counties/States and Canadian Municipalities/Provinces
• 1. Create GDBs and Files (updated 11/18/2022): added Dataset input to allow for further filtering of data
• 3. Prep Data for Bulk Load Prep Toolbox (updated 5/25/2022): updated to export specimens_collected.csv containing records with specimen_collected = 'Yes', for use with Labels_Observations.docm to generate specimen labels.
• Write an Observations_config.csv file to record inputs and minimize input prompts for user. (updated 3/19/2022)
• Delete intermediate geodatabase created when downloading directly from ArcGIS Online (updated 3/30/2022)
• Connect directly to Biotics to query for Visit/SF/EO IDs to:
  • identify updated Observations via parent/globalid
  • indicate IDs of resulting records created/modified in Biotics to provide link between Observation and Biotics record
• Identify and generate SQL statements to add Visit record to an existing SF rather than create a new SF
• Identify and populate Bulk Replace geodatabase with data which should replace an existing SF (spatial feature and/or tabular data) rather than create a new one
• Populate destination table in Observations.gdb with Visit/SF/EO IDs & status (Included, Replaced, Modified)
• Check Observations_MASTER.gdb for records with same globalid/parentglobalid as those in Observations.gdb. Deletes any such records in MASTER.gdb, assuming that edits have been made, to prevent duplication. Appends all observations to Observations_MASTER.gdb
• 9. Write SQL Statements: Auto-generate SQL statements to populate following fields (until BX-4414 implemented) - updated 8/9/2024 as some dataset/global_id was not updated when appropriate.

  • SOURCE_FEATURES, where SOURCE_FEATURE_ID IS NOT NULL:- Data Sensitive (SOURCE_FEATURE.d_data_sensitive_id)
    - Data Sensitive Category (SOURCE_FEATURE.d_data_sensitive_category_id)
    - Data Sensitive Reasons (SOURCE_FEATURE.data_sensitive_reasons)
    - Dataset (SOURCE_FEATURE.d_dataset_id) - when both REPLACE_SF AND REPLACE_VISIT BECAUSE ONLY VISIT FOR SF
    - Global ID (SOURCE_FEATURE.global_id) - when both REPLACE_SF AND REPLACE_VISIT BECAUSE ONLY VISIT FOR SF
    
    VISIT, where SOURCE_FEATURE_ID IS NOT NULL:
    - Detected (Visit, once Detected field included in Biotics, see BX-2139)
    - Dataset (VISIT.d_dataset_id)
    - Global ID (VISIT.global_id)
    - ALL VISIT FIELDS - when REPLACE_VISIT IS NOT NULL - Visit Date
    
    EO, as appropriate:
    - Data Sensitive (EO.d_data_sensitive_id)
    - Data Sensitive Category (EO.d_data_sensitive_category_id
    - Data Sensitive Reasons (EO.data_sensitive_reasons
    - Condition of EO (EO.condition_of_eo)
    - Confidence Extent (EO.d_confidence_extent_id
    - Survey Date (EO.survey_date)
    - Last Observation Date (EO.last_obs_date)
    - First Observation Date (EO.first_obs_date
    EO_REFERENCE, where EO_ID IS NOT NULL:
    - Reference (EO_REFERENCE.reference_id)
    - Primary Reference (EO_REFERENCE.eo_reference_primary_ind) , according to EO_REF_PRIM data
    
    

• 6. Create, Replace, Add - altered to prevent errors when not all of the feature classes (point, line, poly) had data - 6/22/2023
• 8. Populate Destination table - altered to prevent errors when certain fields were not included in the survey - 6/22/2023

Download an empty zipped Observations_v14 geodatabase, should you wish to import data from other sources into the  geodatabase for use with the Observations toolbox, in order to Bulk  Create, Bulk Replace, Add Visits of SFs/EOs in Biotics. This can be done  using Append (ArcGIS Pro) (aka Esri's Simple Data Loader in ArcMap).

Observations ToolboxObservations ToolboxObservations_V14.docxObservations_V14.docx