Skip to content
Arxo Arxo

Examples and Report Walkthrough

Examples and Report Walkthrough

Section titled “Examples and Report Walkthrough”

This page walks through real finetuning_architecture sample artifacts and how to interpret results.

Sample Artifacts

Section titled “Sample Artifacts”

Bundled local sample outputs:

  • crates/arxo-engine/src/metrics/ai_observability/finetuning_architecture/samples/toy-finetune-workflow-report.json
  • crates/arxo-engine/src/metrics/ai_observability/finetuning_architecture/samples/toy-finetune-workflow-report.md

Sample project and config:

  • crates/arxo-engine/src/metrics/ai_observability/finetuning_architecture/samples/toy-finetune-workflow/
  • crates/arxo-engine/src/metrics/ai_observability/finetuning_architecture/samples/finetuning-architecture-config.yaml

Regenerate the JSON Sample

Section titled “Regenerate the JSON Sample”

From your project directory, run Arxo with the path to your fine-tuning project and config:

Terminal window
arxo analyze \
  --path /path/to/your/finetune-project \
  --config finetuning-architecture-config.yaml \
  --format json \
  --output report.json

How to Read the Report

Section titled “How to Read the Report”

1. Start with composite health

Section titled “1. Start with composite health”
  • finetuning_architecture.overall_finetuning_health
  • finetuning_architecture.reproducibility_score
  • finetuning_architecture.data_integrity_score
  • finetuning_architecture.safety_governance_score

These summarize whether the pipeline is broadly healthy before detector-level triage.

2. Triage detector families

Section titled “2. Triage detector families”
  • Reproducibility: base model pinning, run lineage, determinism envelope, checkpoint-eval linkage.
  • Data/eval: eval harness maturity, contamination risk, prompt/template-loss consistency, distillation integrity.
  • Safety/governance: artifact access, trust surface, privacy recordkeeping, provenance.
  • Operations: OOM controls, cost tracking, checkpoint hygiene, resume safety.

3. Use findings evidence

Section titled “3. Use findings evidence”

Review findings with rule_id and CodeSpan evidence to prioritize fixes in concrete files/lines.

4. Use pipeline diagnostics

Section titled “4. Use pipeline diagnostics”
  • pipeline_dag_depth
  • pipeline_cycle_count
  • pipeline_completeness_score
  • effect counts for GPU/database/storage in training files

These help explain operational topology and missing stages.

Example Triage Pattern: Risky Pipeline

Section titled “Example Triage Pattern: Risky Pipeline”

Signals:

  • eval_absence_score: low
  • dataset_contamination_score: low
  • checkpoint_eval_lineage_score: low
  • artifact_trust_surface_score: low

Action order:

  1. Add eval split and quality/safety metrics.
  2. Enforce split contamination controls and dedup checks.
  3. Link checkpoints to eval outcomes and rollback criteria.
  4. Harden artifact trust surface (safe_serialization, avoid unsafe trust paths).

Example Triage Pattern: Stable Pipeline

Section titled “Example Triage Pattern: Stable Pipeline”

Signals:

  • reproducibility_score: high
  • data_integrity_score: high
  • safety_governance_score: high
  • overall_finetuning_health: high

Action order:

  1. Keep baseline no-regression policies enabled in CI.
  2. Raise thresholds gradually for high-impact detector keys.
  3. Focus remediation on new findings only, not already green categories.

Practical Notes

Section titled “Practical Notes”
  • Low score without findings usually means weak evidence density; inspect central training/config files first.
  • Findings are best used as fix-entry points, while scores are better for release gates and trend tracking.
Section titled “Read Next”