# isciml tdem

## Overview

ISCIML is a command line tool designed to generate training data for building artificial intelligence inversion models for time domain electromagnetic (TDEM) data acquisition methods. The tool simulates electromagnetic field propagation through different materials and records the resulting signals at specified receiver locations, which can then be used to train machine learning models for geophysical inversion.

The tool supports both forward and adjoint modeling approaches, allowing users to generate comprehensive datasets that capture the complex relationship between subsurface properties and electromagnetic responses. By adjusting various parameters such as mesh configuration, material properties, transmitter characteristics, and temporal settings, users can create customized training datasets tailored to specific geological environments and survey configurations.

## Installation

Please email <support@s2labs.co> for isciml singularity image for  TDEM training data generation.&#x20;

## Usage

```bash
singularity exec ./container/isciml_tdem.sif isciml --mesh_path ./data/em61_cart_mesh_0p25.xdmf --material_path ./data/em61_flat_n_pipes_3.npz --receiver "Rx_0" --loop_z 0.25 --solver_type adjoint --receiver_locati
on 0.001 0.001 0.25 --r_values_path ./data/em61_cart_r_values.npz
```

| Option                | Type                    | Default                               | Description                                        |
| --------------------- | ----------------------- | ------------------------------------- | -------------------------------------------------- |
| `--mesh_path`         | TEXT                    | `mesh/mesh.xdmf`                      | Path to the mesh file.                             |
| `--material_path`     | TEXT                    | `input/material_properties.npz`       | Path to the material properties file.              |
| `--output_dir`        | TEXT                    | `./output`                            | Directory to save the output files.                |
| `--loop_dimensions`   | FLOAT FLOAT             | `1.0 1.0`                             | Transmitter loop parameters as a tuple (Lx, Ly).   |
| `--loop_z`            | FLOAT                   | `0.001`                               | Height of the transmitter loop.                    |
| `--current_amplitude` | FLOAT                   | `1.0`                                 | Current amplitude in the transmitter loop.         |
| `--receiver_location` | FLOAT FLOAT FLOAT       | `0.001 0.001 0.001`                   | Receiver location coordinates.                     |
| `--dt_list`           | FLOAT FLOAT FLOAT       | `1e-08 1e-07 1e-06`                   | Time step sizes to use in the simulation.          |
| `--time_steps`        | INTEGER INTEGER INTEGER | `500 500 1260`                        | Number of time steps to use in the simulation.     |
| `--dbz_dbt_key`       | TEXT                    | `dBz_dt`                              | Key for the dBz/dt data in the output solution.    |
| `--t_step_key`        | TEXT                    | `t_step`                              | Key for the time step data in the output solution. |
| `--r_values_path`     | TEXT                    | `input/r_values.npz`                  | Path to the file containing r values.              |
| `--adjoint_npz_path`  | TEXT                    | `output/results_adjoint_all.npz`      | Path to the output adjoint results file.           |
| `--mu_0`              | FLOAT                   | `1.2566370614359173e-06`              | Magnetic permeability in free space.               |
| `--time_gates`        | FLOAT FLOAT FLOAT FLOAT | `0.000256 0.000406 0.000706 0.001306` | Time gates for the simulation.                     |
| `--solver_type`       | \[forward\|adjoint]     | `forward`                             | Type of solver to use: 'forward' or 'adjoint'.     |
| `--receiver`          | TEXT                    | `all`                                 | Receiver to use in the simulation.                 |
| `--help`              | FLAG                    | N/A                                   | Show this message and exit.                        |

## Other utilities

## generate\_r\_values.py

`generate_r_values.py` is a utility script for calculating distance values (`r_values`) between mesh nodes and a specified receiver location in electromagnetic simulations. These distance values are essential for accurate modeling of electromagnetic field propagation and are used by the ISCIML simulation tool for time domain electromagnetic (TDEM) modeling.

### Command Line Options

| Option                | Type              | Required | Default     | Description                                                   |
| --------------------- | ----------------- | -------- | ----------- | ------------------------------------------------------------- |
| `--mesh_path`         | PATH              | Yes      | N/A         | Path to the mesh file containing the geometric information.   |
| `--receiver_location` | FLOAT FLOAT FLOAT | Yes      | N/A         | Coordinates of the receiver location (x, y, z).               |
| `--r_values_key`      | TEXT              | No       | "r\_values" | Key used to reference the distance values in the output file. |
| `--output_path`       | PATH              | Yes      | N/A         | Path where the output NPZ file will be saved.                 |
| `--help`              | FLAG              | No       | N/A         | Display help information and exit.                            |

### Usage Examples

#### Basic Usage

```bash
python generate_r_values.py --mesh_path mesh/simulation_mesh.xdmf --receiver_location 0.0 0.0 0.001 --output_path input/r_values.npz
```

This command will:

* Load the mesh from `mesh/simulation_mesh.xdmf`
* Calculate distances from each mesh node to the receiver at coordinates (0.0, 0.0, 0.001)
* Save the results to `input/r_values.npz` using the default key "r\_values"
* Load the mesh from `mesh/fine_mesh.xdmf`
* Calculate distances from each mesh node to the receiver at coordinates (1.0, 2.0, 0.05)
* Save the results to `input/custom_r_values.npz` using the key "distances\_receiver1"

The output file generated by this script is typically used as input to the ISCIML simulation tool via the `--r_values_path` option:

## generate\_npz\_mappings.py

### Overview

`generate_npz_mappings.py` is a utility script that generates a list of NPZ filenames and their corresponding receiver indices. It scans through a specified folder containing NPZ files and creates a mapping file that associates each file with its receiver information.

### Purpose

This script creates an index that maps simulation output files to their respective receiver locations, making it easier to organize and process multiple simulation results. This mapping is especially useful when working with large batches of TDEM simulations that involve multiple receiver positions.

### Command Line Options

| Option          | Type | Default           | Description                                                                                |
| --------------- | ---- | ----------------- | ------------------------------------------------------------------------------------------ |
| `--folder`      | TEXT | Current directory | Directory containing NPZ files to index.                                                   |
| `--output_file` | TEXT | `mappings.txt`    | Path where the output text file (containing filenames and receiver indices) will be saved. |
| `--help`        | FLAG | N/A               | Show help message and exit.                                                                |

### Usage

#### Basic Usage

```bash
singularity exec isciml_tdem.sif python3 /isciml/generate_npz_mappings.py --folder ./simulation_outputs --output_file mappings.txt
```

This command will:

* Scan all NPZ files in the `./simulation_outputs` directory
* Generate a mapping that associates each filename with its receiver index
* Save the mapping information to a text file named `mappings.txt`

## SLURM Array Job Script for ISCIML TDEM Simulations

### Overview

This SLURM batch script is designed to run multiple electromagnetic simulations in parallel using ISCIML with the Singularity container system. It processes a list of material paths and receiver indices stored in a mapping file, executing one simulation per array task.

### Script Details

```bash
#!/bin/bash
#SBATCH --ntasks=1
#SBATCH --time=2:00:00
#SBATCH --mem=4G
#SBATCH --array=0-2
#SBATCH --output=array_job_slurm_%A_%a.out
SIF_FILE=/path/to/isciml_tdem.sif
NPZ_MAPPINGS_FILE=/path/to/npz_mappings.txt
MESH_PATH=/path/to/mesh.xdmf
R_VALUES_PATH=/path/to/r_values.npz
OUTPUT_PATH=/path/to/output
START_INDEX=0
LINE=$((SLURM_ARRAY_TASK_ID+1 + START_INDEX)) 
MATERIAL_PATH=$(sed -n "${LINE}p" ${NPZ_MAPPINGS_FILE} | cut -f1)
RECEIVER_KEY=$(sed -n "${LINE}p" ${NPZ_MAPPINGS_FILE} | cut -f2 | tr -d ' ')
echo "Job started for ${MATERIAL_PATH} ${RECEIVER_KEY}"
singularity exec isciml_tdem.sif isciml --loop_dimensions 1.0 0.5 --mesh_path ${MESH_PATH} --material_path ${MATERIAL_PATH} --receiver "${RECEIVER_KEY}" --loop_z 0.25 --solver_type adjoint --receiver_location 0.001 0.001 0.251 --r_values_path ${R_VALUES_PATH} --output_dir ${OUTPUT_PATH}
echo "Job completed ... "
```

### Purpose

This script facilitates running multiple adjoint solver simulations across different material properties and receiver configurations using SLURM's array job functionality. Each array task processes a specific simulation defined by a line in the mapping file.

### SLURM Resource Allocation

| Directive  | Value                         | Description                             |
| ---------- | ----------------------------- | --------------------------------------- |
| `--ntasks` | 1                             | Allocates 1 CPU core per job            |
| `--time`   | 2:00:00                       | Sets a 2-hour time limit per job        |
| `--mem`    | 4G                            | Allocates 4GB of memory per job         |
| `--array`  | 0-2                           | Runs 3 array jobs (indices 0, 1, and 2) |
| `--output` | array\_job\_slurm\_%A\_%a.out | Sets the output file naming pattern     |

### Parameters

| Variable            | Description                                                                           |
| ------------------- | ------------------------------------------------------------------------------------- |
| `SIF_FILE`          | Path to the Singularity container image for ISCIML TDEM.                              |
| `NPZ_MAPPINGS_FILE` | Path to the text file containing material paths and receiver indices (tab-separated). |
| `MESH_PATH`         | Path to the mesh file (.xdmf) for the simulation domain.                              |
| `R_VALUES_PATH`     | Path to the pre-computed r-values NPZ file.                                           |
| `OUTPUT_PATH`       | Directory where simulation results will be saved.                                     |
| `START_INDEX`       | Offset value for array task indexing (default: 0).                                    |

### How It Works

1. The script uses the SLURM array task ID to determine which line of the mapping file to process.
2. It extracts the material path and receiver key from the specified line in the mapping file.
3. It runs the ISCIML simulation with these parameters using the Singularity container.
4. Output logs are saved with the naming pattern `array_job_slurm_%A_%a.out`, where:
   * `%A` is the master job ID
   * `%a` is the array task ID

### Usage

#### Submission Command

To submit this job with the predefined settings (3 array tasks):

```bash
sbatch path/to/this_script.sh
```

To modify the array range (for example, to run jobs for lines 5-15 in the mapping file):

```bash
sbatch --array=5-15 path/to/this_script.sh
```

#### Mapping File Format

The mapping file (`NPZ_MAPPINGS_FILE`) should be a tab-separated text file with each line containing:

* Material properties file path (first column)
* Receiver index/key (second column)

Example:

```
/path/to/material1.npz    receiver_0
/path/to/material2.npz    receiver_1
/path/to/material3.npz    receiver_2
```

### Simulation Configuration

The script runs ISCIML with the following fixed parameters:

* Loop dimensions: 1.0 x 0.5 units
* Loop height: 0.25 units
* Solver type: adjoint
* Receiver location: (0.001, 0.001, 0.251)

These parameters can be modified directly in the script as needed.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://isciml.s2labs.co/isciml-tdem.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.