UAV Forensic Toolkit

Desktop forensic toolkit for recovering, decoding, validating, and visualizing UAV telemetry logs with chain-of-custody logging, SHA-256 hashing, PDF reports, and machine-readable JSON/CSV artifacts.

1) Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python3 -m uav_forensic_toolkit

2) High-level workflow

1. Recover: copy telemetry files from the SD card (or its forensic copy) into a case directory; compute SHA-256.
2. Decode: convert DJI logs into a unified CSV schema.
3. Analyze: compute physics/timestamp/precision checks + ML anomaly scoring; generate reports.
4. Visualize: create an integrity-aware flight map and charts.

2.1) Complete System Flow

┌─────────────────────────────────────────────────────────────────┐
│                    UAV FORENSIC TOOLKIT FLOW                     │
└─────────────────────────────────────────────────────────────────┘

1. INITIALIZATION
   └─> User enters Operator ID + Case Directory
       └─> Click "Initialize Case" button
           └─> Creates: case_metadata.json, chain_of_custody.jsonl
           └─> Code: uav_forensic_toolkit/gui/main_window.py:86-105

2. RECOVER TAB
   └─> User selects Evidence Source Directory
       └─> (Optional) Enable deep recovery + select forensic image
       └─> Click "Run Recovery" button
           └─> Function: recover_logs_from_directory()
           └─> Code: uav_forensic_toolkit/core/recover.py
           └─> Outputs:
               • RecoveredLogs/ (copied files)
               • CarvedLogs/ (if deep recovery enabled)
               • metadata.json, metadata.csv
               • hash_verification.csv
               • recovery_report.pdf

3. DECODE TAB
   └─> User selects Input Log File (.txt/.dat/.csv)
       └─> (Optional) Provide dji-log Path
       └─> Click "Decode to Unified CSV" button
           └─> Function: cli_decode subprocess
           └─> Code: uav_forensic_toolkit/cli_decode.py
           └─> Outputs:
               • Decoded/decoded_flightlog.csv (unified schema)
               • Decoded/decoder_metadata.json
               • Decoded/decoder_report.pdf

4. ANALYZE TAB
   └─> User selects Decoded CSV
       └─> (Optional) Provide Labels CSV + Label column name
       └─> Click "Run Tampering Analysis" button
           └─> Function: analyze_tampering()
           └─> Code: uav_forensic_toolkit/core/analyze.py:44-330
           └─> Process:
               a) Load points from CSV (lines 62-94)
               b) Physics checks (lines 98-167):
                  • Speed limit: 20.0 m/s
                  • Acceleration limit: 5.0 m/s²
                  • Time gap threshold: 5.0 seconds
               c) GPS precision check (lines 169-179)
               d) ML anomaly detection (lines 181-236):
                  • IsolationForest model
                  • Features: dt_s, distance_m, speed_mps, accel_mps2,
                    alt_rate_mps, delta_lat, delta_lon, bearing_deg,
                    bearing_change_deg
                  • Threshold: 70th percentile (default 0.12)
               e) Evaluation metrics (if labels provided) (lines 243-282)
           └─> Outputs:
               • Analysis/anomaly_scores.csv
               • Analysis/tampering_report.json
               • Analysis/tampering_report.pdf

5. VISUALIZE TAB
   └─> User selects Decoded CSV + Tampering Report JSON
       └─> Click "Generate Flight Visualization" button
           └─> Function: build_visualization_artifacts()
           └─> Code: uav_forensic_toolkit/core/visualize.py:72-234
           └─> Process:
               a) Load points and tampering report (lines 91-96)
               b) Build integrity segments (lines 99-118):
                  • Green = authenticated
                  • Red = tampered
                  • Orange = missing data
               c) Generate KML (lines 125-126)
               d) Generate interactive map with folium (lines 128-146)
               e) Generate charts with matplotlib (lines 169-191):
                  • altitude_chart.png
                  • speed_chart.png
           └─> Outputs:
               • Visualization/flight_map.html (interactive map)
               • Visualization/flight_map.kml
               • Visualization/altitude_chart.png
               • Visualization/speed_chart.png
               • Visualization/flight_report.pdf
               • Visualization/flight_summary.json
    

2.2) Every Button Explained

2.3) Code Locations for Study

2.4) All Threshold Values

2.5) Machine Learning Models Explained

2.6) Labels Format and Meaning

row_idx,label
12,1
13,1
14,0
15,0

case_id,row_idx,label,timestamp,latitude,longitude,altitude
1,0,0,2024-01-01T10:00:00+00:00,37.7749,-122.4194,100.0
1,1,1,2024-01-01T10:00:01+00:00,37.7750,-122.4195,100.0

2.7) Graph Generation Locations

3) Case setup (top panel)

4) Recover tab (Component 1)

Goal: copy telemetry-related files into the case without modifying originals.

Evidence Source Directory (Browse)

Select the SD root folder (or a forensic copy). The recovery code automatically searches for FlightRecord / Flight Records directories and prefers them if found.

Run Recovery (button)

Copies supported extensions into RecoveredLogs/ and computes SHA-256 hashes.

Deep recovery (carving) (optional)

If the SD card contains deleted or corrupted logs that are not visible as normal files, you can enable deep recovery.

• Select a forensic image file (example extensions: .img, .dd, .raw, .dmg).
• If installed, the toolkit uses foremost to carve files from the image. Otherwise it uses a built-in printable-bytes fallback.
• Carved outputs are written under CarvedLogs/ with a confidence score and SHA-256 hashes.

Recovery outputs

• RecoveredLogs/
• CarvedLogs/ (only if deep recovery is enabled)
• metadata.json, metadata.csv
• hash_verification.csv
• recovery_report.pdf

5) Decode tab (Component 2)

Goal: convert DJI logs into a unified CSV schema used by the rest of the pipeline.

Input Log File (Browse)

Select a recovered log file (for DJI, typically DJIFlightRecord_YYYY-MM-DD_[HH-MM-SS].txt).

dji-log Path (optional)

If binary DJI FlightRecord decoding is needed, the toolkit calls the external dji-log tool (from dji-log-parser). Provide the executable path here if it is not in your PATH.

Decode to Unified CSV (button)

Runs decoding in a separate subprocess and writes:

• Decoded/decoded_flightlog.csv (unified schema)
• Decoded/decoder_metadata.json
• Decoded/decoder_report.pdf

Unified CSV schema

timestamp,latitude,longitude,altitude,speed,heading,source

source records the origin/decoder path (example: DJI_FLIGHTRECORD_BIN_DJI_LOG).

6) Analyze tab (Component 3)

Goal: detect suspicious segments using physics checks + timestamp checks + GPS precision + ML anomaly scoring.

Complete Analysis Process (Step-by-Step)

1. Load Data (lines 62-94)
   • Reads decoded CSV row by row
   • Parses timestamp, latitude, longitude, altitude, speed, heading
   • Stores raw GPS strings for precision analysis
   • Creates Point objects with index, timestamp, coordinates
2. Feature Extraction (lines 106-132)
   • For each consecutive point pair (i-1, i):
     • dt_s = time difference in seconds
     • distance_m = Haversine distance
     • speed_mps = distance / dt (m/s)
     • accel_mps2 = (speed - prev_speed) / dt
     • bearing_deg = direction of travel (0-360°)
     • bearing_change_deg = change in direction
     • alt_rate_mps = altitude change rate
     • delta_lat = latitude change
     • delta_lon = longitude change
3. Physics-Based Checks (lines 134-167)
   • Checks if dt_s ≤ 0 → "non_monotonic_timestamp"
   • Checks if dt_s ≥ 5.0 → "suspicious_gap"
   • Checks if speed_mps > 20.0 → "impossible_speed"
   • Checks if |accel_mps2| > 5.0 → "excessive_acceleration"
   • Marks both rows as "tampered" if any flag is set
4. GPS Precision Check (lines 169-179)
   • Counts decimal places in latitude/longitude strings
   • Finds mode (most common) precision
   • Flags rows with precision deviating by ≥2 decimal places
   • Adds "gps_precision_inconsistency" flag
5. ML Anomaly Detection (lines 181-236)
   • Builds feature matrix X from all feature_rows (9 features)
   • Trains IsolationForest on current data
   • Calculates anomaly scores for each segment
   • Normalizes scores to 0.0-1.0 range
   • Calculates threshold as 70th percentile (min 0.12)
   • Flags rows with score ≥ threshold as "ml_anomaly"
6. Evaluation Metrics (lines 243-282, if labels provided)
   • Loads ground truth labels from CSV
   • Compares predictions vs labels
   • Calculates TP, TN, FP, FN
   • Computes Accuracy, Precision, Recall, F1
7. Report Generation (lines 284-330)
   • Creates JSON report with all findings
   • Writes anomaly_scores.csv
   • Generates PDF report
   • Logs to chain of custody

Decoded CSV (Browse)

Select Decoded/decoded_flightlog.csv.

Labels CSV (optional) + Label column

If you have ground truth labels, provide them to compute metrics (accuracy/precision/recall/F1).

row_idx,label
12,1
13,1
14,0

Run Tampering Analysis (button)

Writes:

• Analysis/anomaly_scores.csv
• Analysis/tampering_report.json
• Analysis/tampering_report.pdf

How the ML works

• Model: anomaly scoring over segment-level features.
• Data source: the current case's decoded CSV.
• Features: timestamp interval (dt), distance, speed, acceleration, altitude rate, delta latitude/longitude, bearing, bearing change.
• Output: per-row anomaly score + per-row status (authenticated/tampered) in the reports.

7) Visualize tab (Component 4)

Goal: build an integrity-aware flight path and outputs.

Decoded CSV + Tampering Report JSON (Browse)

Select Decoded/decoded_flightlog.csv and Analysis/tampering_report.json.

Generate Flight Visualization (button)

Writes:

• Visualization/flight_map.html (interactive)
• Visualization/flight_map.kml
• Visualization/altitude_chart.png, Visualization/speed_chart.png
• Visualization/flight_report.pdf, Visualization/flight_summary.json

Visualization Process (Step-by-Step)

1. Load Data (lines 91-96)
   • Loads points from decoded CSV
   • Loads tampering report JSON
   • Extracts row_status dictionary
2. Build Integrity Segments (lines 99-118)
   • Groups consecutive points by status (authenticated/tampered/missing)
   • Detects gaps ≥ 5 seconds → "missing" status
   • Creates segments with color coding:
     • Green: authenticated (solid line)
     • Red: tampered (dashed line, pattern "10, 10")
     • Orange: missing data (dashed line, pattern "2, 8")
3. Generate KML (lines 125-126)
   • Creates KML file for Google Earth
   • Format: longitude,latitude,altitude
4. Generate Interactive Map (lines 128-146)
   • Uses folium library (Python wrapper for Leaflet.js)
   • Creates map centered on first point
   • Adds colored polylines for each segment
   • Adds start/end markers
   • Saves as HTML file
5. Generate Charts (lines 169-191)
   • Altitude Chart:
     • X-axis: Timestamp (Unix epoch)
     • Y-axis: Altitude (meters)
     • Library: matplotlib
     • Output: altitude_chart.png (10×3 inches)
   • Speed Chart:
     • X-axis: Timestamp
     • Y-axis: Speed (m/s)
     • Library: matplotlib
     • Output: speed_chart.png (10×3 inches)
6. Calculate Statistics (lines 153-164)
   • Total distance: Sum of all Haversine distances
   • Speed series: Calculated for each segment
7. Generate Reports (lines 195-225)
   • Creates flight_summary.json with metadata
   • Generates flight_report.pdf
   • Logs to chain of custody

Map Color Coding Explained

Embedded map view

The app embeds flight_map.html if Qt WebEngine is available; otherwise use the "Open Map in Browser" button.

Code: visualize_tab.py:73-83 - Uses QWebEngineView if available

8) Testing the analysis (CLI)

You can run analysis without the GUI:

source .venv/bin/activate
python3 -c "from pathlib import Path; from uav_forensic_toolkit.core.analyze import analyze_tampering; \
from uav_forensic_toolkit.core.custody import OperatorContext; \
print(analyze_tampering(Path('output/Decoded/decoded_flightlog.csv'), Path('output/Analysis'), OperatorContext('op','0.1.0'), Path('output/chain_of_custody.jsonl'), print))"

9) Testing using curl (local HTTP API)

This project includes an optional local HTTP API to run analysis and return JSON. It is intended for local testing only.

source .venv/bin/activate
python3 -m uav_forensic_toolkit.api_server

Then in another terminal:

curl -s http://127.0.0.1:8000/health

curl -s -X POST http://127.0.0.1:8000/analyze \
  -F "decoded_csv=@output/Decoded/decoded_flightlog.csv" \
  -F "operator_id=test_op" \
  -o analysis_result.json

The response is a JSON object containing paths to generated artifacts and key summary fields.

10) Model training (Random Forest)

The toolkit can train a supervised Random Forest model on a labeled CSV dataset and generate training reports and graphs.

Expected dataset format

The training script expects a CSV with at least these columns:

case_id,row_idx,label,timestamp,latitude,longitude,altitude

Extra columns are allowed and will be ignored by the trainer.

Train Random Forest (CLI)

source .venv/bin/activate
python3 -m uav_forensic_toolkit.cli_train_supervised \
  --model random_forest \
  --dataset-csv "path/to/labeled_dataset.csv" \
  --out-dir "output/ModelTraining_supervised_rf" \
  --max-segments 800000 \
  --test-size 0.2 \
  --seed 42 \
  --threshold 0.5

Training outputs

• training_report.json: metrics (accuracy, precision, recall, F1) + confusion matrix + artifact paths
• training_report.pdf: human-readable training summary
• supervised_model.joblib: trained Random Forest model bundle
• metrics_curve.png: accuracy/precision/recall/F1 vs threshold (trimmed to the chosen threshold)
• accuracy_curve.png, precision_curve.png, recall_curve.png, f1_curve.png: separate graphs
• precision_recall_curve.png: precision–recall curve
• confusion_matrix.png: confusion matrix heatmap

11) Reports and artifacts (PDF/JSON/CSV)

This toolkit produces both human-readable PDFs and machine-readable JSON/CSV artifacts.

Recover

• recovery_report.pdf: Recovery summary (including deep recovery if enabled)
• metadata.json, metadata.csv: Recovered file list + metadata
• hash_verification.csv: SHA-256 hashes for verification
• RecoveredLogs/: Copied files
• CarvedLogs/: Carved files (only when deep recovery is enabled)

Decode

• Decoded/decoder_report.pdf
• Decoded/decoder_metadata.json
• Decoded/decoded_flightlog.csv

Analyze

• Analysis/tampering_report.pdf
• Analysis/tampering_report.json: Includes per-row status and flags
• Analysis/anomaly_scores.csv: ML anomaly score per row

Visualize

• Visualization/flight_report.pdf
• Visualization/flight_summary.json
• Visualization/flight_map.html, Visualization/flight_map.kml
• Visualization/altitude_chart.png, Visualization/speed_chart.png

Chain of custody

• chain_of_custody.jsonl: One JSON event per line for every operation
• case_metadata.json: Case initialization metadata

12) Where to find outputs

Everything from the GUI is stored under your Case Directory (example: output/).

• RecoveredLogs/
• CarvedLogs/ (if deep recovery was enabled)
• Decoded/
• Analysis/
• Visualization/
• chain_of_custody.jsonl

13) Troubleshooting

• Segmentation fault on macOS: fixed by keeping background QRunnables alive until completion and using reportlab (pure Python) for PDF generation instead of Qt's QPdfWriter.
• Map shows "L is not defined": fixed by allowing local HTML to load remote Leaflet assets in Qt WebEngine (enabled LocalContentCanAccessRemoteUrls and LocalContentCanAccessFileUrls).
• Decode produces 0 rows: input may be binary or unsupported format; use dji-log for binary DJIFlightRecord. The DJI API key is hardcoded in the decoder.
• Binary DJIFlightRecord decoding fails: ensure the dji-log executable is in PATH or provide its path in the Decode tab. The tool requires the external dji-log binary from dji-log-parser.