starting dec 29

README.md

@@ -9,7 +9,70 @@ app_file: app.py
 pinned: false
 license: apache-2.0
 ---
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+## PestIDBot - Quick Reference
+
+### Environment
+```bash
+source ~/miniconda3/etc/profile.d/conda.sh && conda activate agllm-env1-updates-1
+```
+
+### Key Commands
+| Task | Command |
+|------|---------|
+| Build DB | `python app_database_prep.py` |
+| Run Eval | `python retrieval_evaluation.py` |
+| Run App | `python app.py` |
+| Deploy Dev | `git push space3 fresh-start:main` |
+| Deploy Prod | `git push space2 fresh-start:main` |
+
+### Git Remotes
+- `space2` → `git@hf.co:spaces/arbabarshad/agllm2` (production)
+- `space3` → `git@hf.co:spaces/arbabarshad/agllm2-dev` (dev)
+
+### Project Structure
+```
+├── app.py                      # Main Gradio app (deployed)
+├── app_database_prep.py        # Builds ChromaDB from PDFs + Excel
+├── retrieval_evaluation.py     # Runs 4-filter evaluation
+├── retrieval_evaluation_results.json  # Eval metrics output
+│
+├── agllm-data/
+│   ├── agllm-data-isu-field-insects-all-species/
+│   │   ├── *.pdf               # Insect IPM documents
+│   │   └── matched_species_results_v2.csv  # Species metadata
+│   ├── agllm-data-isu-field-weeds-all-species/
+│   │   ├── *.pdf               # Weed IPM documents
+│   │   └── matched_species_results_v2.csv  # Species metadata
+│   └── PestID Species.xlsx     # India & Africa data (sheets)
+│
+├── vector-databases-deployed/
+│   └── db5-agllm-data-isu-field-insects-all-species/  # ChromaDB output
+│
+├── species-organized/          # Analysis scripts & outputs
+│   ├── species_analysis.py     # Generates paper Figure 3
+│   └── species_table.tex       # LaTeX species table
+│
+└── writing/                    # Paper drafts
+```
+
+### Database Build Flow
+1. PDFs loaded from `agllm-data/` (insects + weeds)
+2. Metadata read from `matched_species_results_v2.csv` files
+3. Excel sheets (India, Africa) processed from `PestID Species.xlsx`
+4. Documents chunked (512 tokens, 10 overlap)
+5. Tagged with `matched_specie_X` + `region` metadata
+6. Stored in ChromaDB at `vector-databases-deployed/db5-*/`
+
+### Evaluation Filters (retrieval_evaluation.py)
+| Filter | P@5 | nDCG@5 |
+|--------|-----|--------|
+| No Filter | 0.82 | 0.72 |
+| Species Only | 0.99 | 0.89 |
+| Region Only | 0.83 | 0.73 |
+| Species + Region | **1.00** | **0.90** |
+
+---
 
 ## Git LFS Troubleshooting Notes
 
@@ -44,11 +107,3 @@ This repository encountered several Git LFS issues during setup. Here's a summar
 *   Pushing branches with problematic LFS history to a fresh remote can fail. Starting the remote with a clean, history-free branch is a workaround.
 *   When adding LFS tracking for existing binary files via `.gitattributes`, ensure the commit correctly converts files to LFS pointers. `git add --renormalize .` after updating `.gitattributes` and *before* committing is often necessary.
 *   Double-check `.gitignore` if expected files or directories are missing after a `git add .`.
-
-
-while running in claude code : 
-source ~/miniconda3/etc/profile.d/conda.sh && conda install -c conda-forge numpy
-ate agthinker
-
-run command like example:  source ~/miniconda3/etc/profile.d/conda.sh && conda activate agllm-env1-updates-1 &&         │
-│   python whatebverscriptis.py

app_database_prep.py

@@ -228,6 +228,11 @@ africa_splitted_documents = process_excel_sheet(
 splitted_documents = pdf_splitted_documents + india_splitted_documents + africa_splitted_documents
 
 
+print("pdf_splitted_documents", len(pdf_splitted_documents))
+print("india_splitted_documents", len(india_splitted_documents))
+print("africa_splitted_documents", len(africa_splitted_documents))
+
+
 # print(splitted_documents[0]) # Original print statement - commented out as we print chunks above
 print("=== Combined Processing Done ===") # Adjusted print statement
 print(f"Total documents after combining PDF, India, and Africa sources: {len(splitted_documents)}")

retrieval_evaluation_results.json

@@ -1,45 +1,45 @@
 {
   "no_filter": {
     "precision@1": {
-      "mean": 0.61,
-      "std": 0.4877499359302879,
+      "mean": 0.59,
+      "std": 0.49183330509431744,
       "count": 100
     },
     "precision@3": {
-      "mean": 0.82,
-      "std": 0.38418745424597095,
+      "mean": 0.76,
+      "std": 0.42708313008125254,
       "count": 100
     },
     "precision@5": {
-      "mean": 0.84,
-      "std": 0.36660605559646725,
+      "mean": 0.82,
+      "std": 0.38418745424597095,
       "count": 100
     },
     "ndcg@1": {
-      "mean": 0.61,
-      "std": 0.4877499359302879,
+      "mean": 0.59,
+      "std": 0.49183330509431744,
       "count": 100
     },
     "ndcg@3": {
-      "mean": 0.7359487605714332,
-      "std": 0.38022493138147806,
+      "mean": 0.6946394630357184,
+      "std": 0.41495405707705707,
       "count": 100
     },
     "ndcg@5": {
-      "mean": 0.7441240542245126,
-      "std": 0.3685408287782305,
+      "mean": 0.7182888689781796,
+      "std": 0.3848500116841757,
       "count": 100
     }
   },
   "species_only": {
     "precision@1": {
-      "mean": 0.71,
-      "std": 0.4537620521815371,
+      "mean": 0.73,
+      "std": 0.4439594576084623,
       "count": 100
     },
     "precision@3": {
-      "mean": 0.97,
-      "std": 0.17058722109231983,
+      "mean": 0.98,
+      "std": 0.13999999999999999,
       "count": 100
     },
     "precision@5": {
@@ -48,57 +48,57 @@
       "count": 100
     },
     "ndcg@1": {
-      "mean": 0.71,
-      "std": 0.4537620521815371,
+      "mean": 0.73,
+      "std": 0.4439594576084623,
       "count": 100
     },
     "ndcg@3": {
-      "mean": 0.8661859507142915,
-      "std": 0.23310162928115066,
+      "mean": 0.8811859507142915,
+      "std": 0.2136019453378135,
       "count": 100
     },
     "ndcg@5": {
-      "mean": 0.8739230068589822,
-      "std": 0.2094424760171824,
+      "mean": 0.8850544787866369,
+      "std": 0.2007226726424171,
       "count": 100
     }
   },
   "region_only": {
     "precision@1": {
-      "mean": 0.62,
-      "std": 0.48538644398046393,
+      "mean": 0.61,
+      "std": 0.4877499359302879,
       "count": 100
     },
     "precision@3": {
-      "mean": 0.83,
-      "std": 0.375632799419859,
+      "mean": 0.77,
+      "std": 0.4208325082500163,
       "count": 100
     },
     "precision@5": {
-      "mean": 0.86,
-      "std": 0.34698703145794946,
+      "mean": 0.83,
+      "std": 0.375632799419859,
       "count": 100
     },
     "ndcg@1": {
-      "mean": 0.62,
-      "std": 0.48538644398046393,
+      "mean": 0.61,
+      "std": 0.4877499359302879,
       "count": 100
     },
     "ndcg@3": {
-      "mean": 0.7459487605714332,
-      "std": 0.373834218916114,
+      "mean": 0.7083301655000039,
+      "std": 0.4110942789611411,
       "count": 100
     },
     "ndcg@5": {
-      "mean": 0.7584308198052464,
-      "std": 0.3552188974398061,
+      "mean": 0.7319795714424648,
+      "std": 0.37983366654728035,
       "count": 100
     }
   },
   "species_and_region": {
     "precision@1": {
-      "mean": 0.72,
-      "std": 0.4489988864128729,
+      "mean": 0.75,
+      "std": 0.4330127018922193,
       "count": 100
     },
     "precision@3": {
@@ -107,23 +107,23 @@
       "count": 100
     },
     "precision@5": {
-      "mean": 0.99,
-      "std": 0.09949874371066199,
+      "mean": 1.0,
+      "std": 0.0,
       "count": 100
     },
     "ndcg@1": {
-      "mean": 0.72,
-      "std": 0.4489988864128729,
+      "mean": 0.75,
+      "std": 0.4330127018922193,
       "count": 100
     },
     "ndcg@3": {
-      "mean": 0.877495248250006,
-      "std": 0.21470277973614038,
+      "mean": 0.8898766531785769,
+      "std": 0.2091728695998248,
       "count": 100
     },
     "ndcg@5": {
-      "mean": 0.8813637763223514,
-      "std": 0.20196444998865976,
+      "mean": 0.8980519468316562,
+      "std": 0.18024378480878206,
       "count": 100
     }
   }

species-organized/PestID Species - Organized.xlsx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb0ceab28dfca471472fcc9e4631100826dc5ea2d3fdf2b78c215b859938eb61
+size 27466

species-organized/species_analysis.py

@@ -0,0 +1,513 @@
+"""
+Species Analysis Script
+=======================
+Analyzes pest species data from PestID Species - Organized.xlsx
+Generates: LaTeX table, multi-panel visualization, and statistics summary
+
+Author: AgLLM Project
+Date: 2025-10-26
+"""
+
+import pandas as pd
+import numpy as np
+import matplotlib.pyplot as plt
+from matplotlib.gridspec import GridSpec
+from matplotlib.patches import Circle
+from matplotlib.patches import Patch
+from pathlib import Path
+
+# Set font parameters (matching reference style)
+plt.rcParams.update({
+    'font.family': 'Arial',
+    'font.size': 14,
+    'axes.labelsize': 14,
+    'axes.titlesize': 15,
+    'xtick.labelsize': 12,
+    'ytick.labelsize': 12,
+    'legend.fontsize': 13,
+})
+
+# Define color palette
+COLORS = ['#2E86AB', '#A23B72', '#F18F01', '#C73E1D', '#6A994E', '#BC4B51', '#5B8E7D', '#F4A259']
+REGION_COLORS = {
+    'US': '#2E86AB',
+    'Africa': '#F18F01',
+    'India': '#6A994E'
+}
+
+# File paths
+SCRIPT_DIR = Path(__file__).parent
+DATA_FILE = SCRIPT_DIR / 'PestID Species - Organized.xlsx'
+OUTPUT_TABLE = SCRIPT_DIR / 'species_table.tex'
+OUTPUT_PLOT_PDF = SCRIPT_DIR / 'species_analysis.pdf'
+OUTPUT_PLOT_PNG = SCRIPT_DIR / 'species_analysis.png'
+OUTPUT_STATS = SCRIPT_DIR / 'species_statistics.txt'
+
+
+def load_and_prepare_data():
+    """Load data from all sheets and create consolidated DataFrame"""
+    print("Loading data from Excel file...")
+
+    # Read all sheets
+    us_df = pd.read_excel(DATA_FILE, sheet_name='US')
+    africa_df = pd.read_excel(DATA_FILE, sheet_name='Africa')
+    india_df = pd.read_excel(DATA_FILE, sheet_name='India')
+
+    # Add region column
+    us_df['Region'] = 'US'
+    africa_df['Region'] = 'Africa'
+    india_df['Region'] = 'India'
+
+    # Standardize column names across sheets
+    # US: Species, Common Name, Tag, Accuracy
+    # Africa: Common Name, Species, Tag, Accuracy, IPM Info, Excluded Link
+    # India: Common Name, Species, Tag, Accuracy, IPM Info
+
+    # Reorder columns for US to match others
+    us_df = us_df[['Region', 'Species', 'Common Name', 'Tag', 'Accuracy']]
+    us_df['IPM Info'] = None
+    us_df['Excluded Link'] = None
+
+    # Reorder columns for Africa
+    africa_df = africa_df[['Region', 'Species', 'Common Name', 'Tag', 'Accuracy', 'IPM Info', 'Excluded Link']]
+
+    # Reorder columns for India (no Excluded Link)
+    india_df = india_df[['Region', 'Species', 'Common Name', 'Tag', 'Accuracy', 'IPM Info']]
+    india_df['Excluded Link'] = None
+
+    # FIX: India uses decimal accuracy (0.59-0.95) instead of percentage
+    # Multiply India accuracy by 100
+    india_df['Accuracy'] = india_df['Accuracy'] * 100
+
+    # Concatenate all DataFrames
+    df_consolidated = pd.concat([us_df, africa_df, india_df], ignore_index=True)
+
+    # Add helper column for IPM availability
+    df_consolidated['Has_IPM'] = df_consolidated['IPM Info'].notna()
+
+    print(f"✓ Loaded {len(df_consolidated)} species from 3 regions")
+    print(f"  - US: {len(us_df)} species")
+    print(f"  - Africa: {len(africa_df)} species")
+    print(f"  - India: {len(india_df)} species")
+
+    return df_consolidated
+
+
+def generate_latex_table(df):
+    """Generate LaTeX table with all species data"""
+    print("\nGenerating LaTeX table...")
+
+    # Create simplified table for LaTeX (without full IPM text)
+    df_table = df[['Region', 'Species', 'Common Name', 'Tag', 'Accuracy', 'Has_IPM']].copy()
+
+    # Format accuracy
+    df_table['Accuracy'] = df_table['Accuracy'].apply(lambda x: f"{x:.1f}" if pd.notna(x) else "—")
+
+    # Format Has_IPM as Yes/No
+    df_table['Has_IPM'] = df_table['Has_IPM'].apply(lambda x: "Yes" if x else "No")
+
+    # Replace NaN in Tag
+    df_table['Tag'] = df_table['Tag'].fillna("—")
+
+    # Sort by Region, then Tag, then Accuracy
+    df_table = df_table.sort_values(['Region', 'Tag', 'Accuracy'], ascending=[True, True, False])
+
+    # Start building LaTeX code
+    latex_code = []
+    latex_code.append("% LaTeX Table: Species Analysis")
+    latex_code.append("% Requires: \\usepackage{booktabs, longtable}")
+    latex_code.append("")
+    latex_code.append("\\begin{longtable}{llllrr}")
+    latex_code.append("\\toprule")
+    latex_code.append("\\textbf{Region} & \\textbf{Species} & \\textbf{Common Name} & \\textbf{Tag} & \\textbf{Accuracy (\\%)} & \\textbf{IPM Info} \\\\")
+    latex_code.append("\\midrule")
+    latex_code.append("\\endfirsthead")
+    latex_code.append("")
+    latex_code.append("\\multicolumn{6}{c}")
+    latex_code.append("{\\tablename\\ \\thetable\\ -- \\textit{Continued from previous page}} \\\\")
+    latex_code.append("\\toprule")
+    latex_code.append("\\textbf{Region} & \\textbf{Species} & \\textbf{Common Name} & \\textbf{Tag} & \\textbf{Accuracy (\\%)} & \\textbf{IPM Info} \\\\")
+    latex_code.append("\\midrule")
+    latex_code.append("\\endhead")
+    latex_code.append("")
+    latex_code.append("\\midrule")
+    latex_code.append("\\multicolumn{6}{r}{\\textit{Continued on next page}} \\\\")
+    latex_code.append("\\endfoot")
+    latex_code.append("")
+    latex_code.append("\\bottomrule")
+    latex_code.append("\\endlastfoot")
+    latex_code.append("")
+
+    # Add data rows
+    for idx, row in df_table.iterrows():
+        # Escape special LaTeX characters
+        species = str(row['Species']).replace('_', '\\_').replace('&', '\\&')
+        common_name = str(row['Common Name']).replace('_', '\\_').replace('&', '\\&')
+
+        latex_code.append(f"{row['Region']} & \\textit{{{species}}} & {common_name} & {row['Tag']} & {row['Accuracy']} & {row['Has_IPM']} \\\\")
+
+    latex_code.append("")
+    latex_code.append("\\end{longtable}")
+
+    # Write to file
+    with open(OUTPUT_TABLE, 'w') as f:
+        f.write('\n'.join(latex_code))
+
+    print(f"✓ LaTeX table saved to: {OUTPUT_TABLE}")
+    print(f"  Contains {len(df_table)} species")
+
+
+def create_visualization(df):
+    """Create comprehensive multi-panel visualization"""
+    print("\nCreating visualization...")
+
+    # Create figure with GridSpec layout (2 rows × 2 columns)
+    fig = plt.figure(figsize=(14, 10))
+    gs = GridSpec(2, 2, figure=fig, hspace=0.35, wspace=0.35)
+
+    # 1. Species Count by Region (Top Left) - with Insect/Weed breakdown
+    ax1 = fig.add_subplot(gs[0, 0])
+
+    # Get insect and weed counts by region
+    tag_by_region = pd.crosstab(df['Region'], df['Tag'])
+    tag_by_region = tag_by_region.reindex(['US', 'Africa', 'India'])
+
+    insects = tag_by_region['insect'].values if 'insect' in tag_by_region.columns else [0, 0, 0]
+    weeds = tag_by_region['weed'].values if 'weed' in tag_by_region.columns else [0, 0, 0]
+
+    x = range(len(tag_by_region))
+
+    # Create stacked bars
+    bars1_insects = ax1.bar(x, insects, label='Insect', color=COLORS[0], alpha=0.8, edgecolor='black', linewidth=0.5)
+    bars1_weeds = ax1.bar(x, weeds, bottom=insects, label='Weed', color=COLORS[2], alpha=0.8, edgecolor='black', linewidth=0.5)
+
+    ax1.set_xticks(x)
+    ax1.set_xticklabels(tag_by_region.index)
+    ax1.set_ylabel('Number of Species')
+    ax1.set_title('Species Count by Region')
+    ax1.legend(loc='upper right', fontsize=11)
+
+    # Add total count labels on top
+    for i, region in enumerate(tag_by_region.index):
+        total = insects[i] + weeds[i]
+        ax1.text(i, total + 2, str(int(total)), ha='center', va='bottom',
+                fontsize=12, fontweight='bold')
+
+    # 2. Accuracy Distribution by Region - Box Plot (Top Right)
+    ax2 = fig.add_subplot(gs[0, 1])
+
+    # Prepare data for box plot
+    accuracy_data = []
+    labels = []
+    colors_box = []
+    for region in ['US', 'Africa', 'India']:
+        region_acc = df[df['Region'] == region]['Accuracy'].dropna()
+        if len(region_acc) > 0:
+            accuracy_data.append(region_acc)
+            labels.append(region)
+            colors_box.append(REGION_COLORS[region])
+
+    bp = ax2.boxplot(accuracy_data, tick_labels=labels, patch_artist=True,
+                     medianprops=dict(color='red', linewidth=2),
+                     whiskerprops=dict(linewidth=1.5),
+                     boxprops=dict(linewidth=1.5),
+                     showfliers=True)
+
+    # Color the boxes
+    for patch, color in zip(bp['boxes'], colors_box):
+        patch.set_facecolor(color)
+        patch.set_alpha(0.7)
+
+    ax2.set_ylabel('Accuracy (%)')
+    ax2.set_title('Accuracy Distribution by Region')
+    ax2.grid(True, alpha=0.3, axis='y')
+    ax2.set_ylim(35, 105)
+
+    # 3. Species Overlap - Venn Diagram (Bottom Left)
+    ax3 = fig.add_subplot(gs[1, 0])
+
+    # Calculate species overlap
+    us_species = set(df[df['Region'] == 'US']['Species'].str.lower().str.strip())
+    africa_species = set(df[df['Region'] == 'Africa']['Species'].str.lower().str.strip())
+    india_species = set(df[df['Region'] == 'India']['Species'].str.lower().str.strip())
+
+    us_only = len(us_species - africa_species - india_species)
+    africa_only = len(africa_species - us_species - india_species)
+    india_only = len(india_species - us_species - africa_species)
+    us_africa = len((us_species & africa_species) - india_species)
+    us_india = len((us_species & india_species) - africa_species)
+    africa_india = len((africa_species & india_species) - us_species)
+    all_three = len(us_species & africa_species & india_species)
+
+    # Create professional 3-circle Venn diagram
+    ax3.set_xlim(0, 4)
+    ax3.set_ylim(-0.5, 3.5)  # Extended lower bound to prevent cutoff
+    ax3.set_aspect('equal')
+    ax3.axis('off')
+
+    # Circle parameters for proper overlap
+    radius = 1.0
+    # Positions chosen to create good overlaps
+    circle_us = Circle((1.2, 1.8), radius, color=REGION_COLORS['US'], alpha=0.4,
+                       linewidth=2, edgecolor=REGION_COLORS['US'], fill=True)
+    circle_africa = Circle((2.8, 1.8), radius, color=REGION_COLORS['Africa'], alpha=0.4,
+                          linewidth=2, edgecolor=REGION_COLORS['Africa'], fill=True)
+    circle_india = Circle((2.0, 0.7), radius, color=REGION_COLORS['India'], alpha=0.4,
+                         linewidth=2, edgecolor=REGION_COLORS['India'], fill=True)
+
+    ax3.add_patch(circle_us)
+    ax3.add_patch(circle_africa)
+    ax3.add_patch(circle_india)
+
+    # Add labels for regions (outside circles, avoiding overlap)
+    ax3.text(0.4, 2.9, 'US', fontsize=13, fontweight='bold', color=REGION_COLORS['US'], ha='center')
+    ax3.text(3.6, 2.9, 'Africa', fontsize=13, fontweight='bold', color=REGION_COLORS['Africa'], ha='center')
+    ax3.text(2.0, -0.45, 'India', fontsize=13, fontweight='bold', color=REGION_COLORS['India'], ha='center')
+
+    # Add counts in appropriate regions
+    # US only (left)
+    ax3.text(0.7, 1.8, str(us_only), fontsize=14, fontweight='bold', ha='center', va='center')
+
+    # Africa only (right)
+    ax3.text(3.3, 1.8, str(africa_only), fontsize=14, fontweight='bold', ha='center', va='center')
+
+    # India only (bottom)
+    ax3.text(2.0, 0.3, str(india_only), fontsize=14, fontweight='bold', ha='center', va='center')
+
+    # US & Africa (top middle)
+    ax3.text(2.0, 2.1, str(us_africa), fontsize=13, fontweight='bold', ha='center', va='center',
+             bbox=dict(boxstyle='round,pad=0.3', facecolor='white', alpha=0.7))
+
+    # US & India (left-bottom)
+    ax3.text(1.4, 1.0, str(us_india), fontsize=13, fontweight='bold', ha='center', va='center',
+             bbox=dict(boxstyle='round,pad=0.3', facecolor='white', alpha=0.7))
+
+    # Africa & India (right-bottom)
+    ax3.text(2.6, 1.0, str(africa_india), fontsize=13, fontweight='bold', ha='center', va='center',
+             bbox=dict(boxstyle='round,pad=0.3', facecolor='white', alpha=0.7))
+
+    # All three (center)
+    ax3.text(2.0, 1.4, str(all_three), fontsize=13, fontweight='bold', ha='center', va='center',
+             bbox=dict(boxstyle='round,pad=0.3', facecolor='yellow', alpha=0.6))
+
+    ax3.set_title('Species Overlap Across Regions', fontsize=15, pad=10)
+
+    # 4. Accuracy Range Distribution (Bottom Right)
+    ax4 = fig.add_subplot(gs[1, 1])
+
+    # Create accuracy bins
+    df_acc = df.dropna(subset=['Accuracy']).copy()
+    bins = [0, 50, 70, 85, 100]
+    labels_bins = ['0-50%', '50-70%', '70-85%', '85-100%']
+    df_acc['Accuracy_Range'] = pd.cut(df_acc['Accuracy'], bins=bins, labels=labels_bins, include_lowest=True)
+
+    range_counts = df_acc['Accuracy_Range'].value_counts().reindex(labels_bins)
+
+    bars4 = ax4.bar(range(len(range_counts)), range_counts.values,
+                   color=COLORS[3], alpha=0.8, edgecolor='black', linewidth=1)
+    ax4.set_xticks(range(len(range_counts)))
+    ax4.set_xticklabels(range_counts.index, rotation=0)
+    ax4.set_xlabel('Accuracy Range')
+    ax4.set_ylabel('Number of Species')
+    ax4.set_title('Species by Accuracy Range')
+
+    # Add value labels
+    for i, val in enumerate(range_counts.values):
+        if pd.notna(val) and val > 0:
+            ax4.text(i, val + 1.5, str(int(val)),
+                    ha='center', va='bottom', fontsize=11, fontweight='bold')
+
+    # Add overall title
+    plt.suptitle('PestID Bot Knowledgebank Features', fontsize=18, fontweight='bold', y=0.98)
+
+    # Save figures
+    plt.savefig(OUTPUT_PLOT_PDF, dpi=300, bbox_inches='tight')
+    plt.savefig(OUTPUT_PLOT_PNG, dpi=300, bbox_inches='tight')
+
+    print(f"✓ Visualization saved:")
+    print(f"  - {OUTPUT_PLOT_PDF}")
+    print(f"  - {OUTPUT_PLOT_PNG}")
+
+
+def generate_statistics(df):
+    """Generate comprehensive statistics summary"""
+    print("\nGenerating statistics summary...")
+
+    stats = []
+    stats.append("=" * 80)
+    stats.append("PEST SPECIES ANALYSIS - STATISTICS SUMMARY")
+    stats.append("=" * 80)
+    stats.append("")
+
+    # 1. Overall counts
+    stats.append("1. OVERALL SPECIES COUNTS")
+    stats.append("-" * 40)
+    stats.append(f"Total species: {len(df)}")
+    stats.append("")
+    stats.append("By Region:")
+    for region in ['US', 'Africa', 'India']:
+        count = len(df[df['Region'] == region])
+        percentage = (count / len(df)) * 100
+        stats.append(f"  {region:10s}: {count:3d} species ({percentage:5.1f}%)")
+    stats.append("")
+
+    # 2. Tag distribution
+    stats.append("2. INSECT VS WEED DISTRIBUTION")
+    stats.append("-" * 40)
+
+    total_insects = len(df[df['Tag'] == 'insect'])
+    total_weeds = len(df[df['Tag'] == 'weed'])
+    stats.append(f"Overall:")
+    stats.append(f"  Insects: {total_insects} ({total_insects/len(df)*100:.1f}%)")
+    stats.append(f"  Weeds:   {total_weeds} ({total_weeds/len(df)*100:.1f}%)")
+    stats.append("")
+
+    stats.append("By Region:")
+    for region in ['US', 'Africa', 'India']:
+        region_df = df[df['Region'] == region]
+        insects = len(region_df[region_df['Tag'] == 'insect'])
+        weeds = len(region_df[region_df['Tag'] == 'weed'])
+        stats.append(f"  {region}:")
+        stats.append(f"    Insects: {insects}")
+        stats.append(f"    Weeds:   {weeds}")
+    stats.append("")
+
+    # 3. Accuracy statistics
+    stats.append("3. ACCURACY STATISTICS")
+    stats.append("-" * 40)
+
+    for region in ['US', 'Africa', 'India']:
+        region_df = df[df['Region'] == region]
+        acc = region_df['Accuracy'].dropna()
+
+        stats.append(f"{region}:")
+        if len(acc) > 0:
+            stats.append(f"  Mean:     {acc.mean():6.2f}%")
+            stats.append(f"  Median:   {acc.median():6.2f}%")
+            stats.append(f"  Std Dev:  {acc.std():6.2f}%")
+            stats.append(f"  Min:      {acc.min():6.2f}%")
+            stats.append(f"  Max:      {acc.max():6.2f}%")
+            missing = region_df['Accuracy'].isna().sum()
+            stats.append(f"  Missing:  {missing} ({missing/len(region_df)*100:.1f}%)")
+        else:
+            stats.append(f"  No accuracy data")
+        stats.append("")
+
+    # Overall accuracy
+    all_acc = df['Accuracy'].dropna()
+    stats.append("Overall (all regions):")
+    stats.append(f"  Mean:     {all_acc.mean():6.2f}%")
+    stats.append(f"  Median:   {all_acc.median():6.2f}%")
+    stats.append(f"  Std Dev:  {all_acc.std():6.2f}%")
+    stats.append(f"  Range:    {all_acc.min():.2f}% - {all_acc.max():.2f}%")
+    stats.append("")
+
+    # 4. IPM Info coverage
+    stats.append("4. IPM INFORMATION COVERAGE")
+    stats.append("-" * 40)
+
+    for region in ['US', 'Africa', 'India']:
+        region_df = df[df['Region'] == region]
+        with_ipm = region_df['Has_IPM'].sum()
+        total = len(region_df)
+        percentage = (with_ipm / total) * 100
+        stats.append(f"{region:10s}: {with_ipm:2d}/{total:2d} species ({percentage:5.1f}%)")
+
+    total_ipm = df['Has_IPM'].sum()
+    stats.append(f"{'Overall':10s}: {total_ipm:2d}/{len(df):2d} species ({total_ipm/len(df)*100:5.1f}%)")
+    stats.append("")
+
+    # 5. Top species by accuracy
+    stats.append("5. TOP 10 SPECIES BY ACCURACY")
+    stats.append("-" * 40)
+
+    top_10 = df.dropna(subset=['Accuracy']).nlargest(10, 'Accuracy')
+    for i, (idx, row) in enumerate(top_10.iterrows(), 1):
+        stats.append(f"{i:2d}. {row['Common Name']:30s} ({row['Species']:25s}) - {row['Accuracy']:5.1f}% [{row['Region']}]")
+    stats.append("")
+
+    # 6. Species with lowest accuracy
+    stats.append("6. BOTTOM 10 SPECIES BY ACCURACY")
+    stats.append("-" * 40)
+
+    bottom_10 = df.dropna(subset=['Accuracy']).nsmallest(10, 'Accuracy')
+    for i, (idx, row) in enumerate(bottom_10.iterrows(), 1):
+        stats.append(f"{i:2d}. {row['Common Name']:30s} ({row['Species']:25s}) - {row['Accuracy']:5.1f}% [{row['Region']}]")
+    stats.append("")
+
+    # 7. Species overlap analysis
+    stats.append("7. SPECIES OVERLAP ACROSS REGIONS")
+    stats.append("-" * 40)
+
+    us_species = set(df[df['Region'] == 'US']['Species'].str.lower().str.strip())
+    africa_species = set(df[df['Region'] == 'Africa']['Species'].str.lower().str.strip())
+    india_species = set(df[df['Region'] == 'India']['Species'].str.lower().str.strip())
+
+    overlap_us_africa = us_species & africa_species
+    overlap_us_india = us_species & india_species
+    overlap_africa_india = africa_species & india_species
+    all_three = us_species & africa_species & india_species
+
+    stats.append(f"US & Africa:      {len(overlap_us_africa)} species")
+    if len(overlap_us_africa) > 0:
+        for species in sorted(overlap_us_africa):
+            stats.append(f"  - {species}")
+
+    stats.append(f"\nUS & India:       {len(overlap_us_india)} species")
+    if len(overlap_us_india) > 0:
+        for species in sorted(overlap_us_india):
+            stats.append(f"  - {species}")
+
+    stats.append(f"\nAfrica & India:   {len(overlap_africa_india)} species")
+    if len(overlap_africa_india) > 0:
+        for species in sorted(overlap_africa_india):
+            stats.append(f"  - {species}")
+
+    stats.append(f"\nAll three regions: {len(all_three)} species")
+    if len(all_three) > 0:
+        for species in sorted(all_three):
+            stats.append(f"  - {species}")
+
+    stats.append("")
+    stats.append("=" * 80)
+    stats.append(f"Analysis completed on: 2025-10-26")
+    stats.append("=" * 80)
+
+    # Write to file
+    with open(OUTPUT_STATS, 'w') as f:
+        f.write('\n'.join(stats))
+
+    print(f"✓ Statistics saved to: {OUTPUT_STATS}")
+
+    # Also print to console
+    print("\n" + '\n'.join(stats[:50]))  # Print first 50 lines to console
+
+
+def main():
+    """Main execution function"""
+    print("=" * 80)
+    print("PEST SPECIES ANALYSIS")
+    print("=" * 80)
+    print()
+
+    # Load and prepare data
+    df = load_and_prepare_data()
+
+    # Generate outputs
+    generate_latex_table(df)
+    create_visualization(df)
+    generate_statistics(df)
+
+    print("\n" + "=" * 80)
+    print("ANALYSIS COMPLETE!")
+    print("=" * 80)
+    print("\nGenerated files:")
+    print(f"  1. {OUTPUT_TABLE.name} - LaTeX table")
+    print(f"  2. {OUTPUT_PLOT_PDF.name} - Visualization (PDF)")
+    print(f"  3. {OUTPUT_PLOT_PNG.name} - Visualization (PNG)")
+    print(f"  4. {OUTPUT_STATS.name} - Statistics summary")
+    print()
+
+
+if __name__ == '__main__':
+    main()

species-organized/species_statistics.txt

@@ -0,0 +1,121 @@
+================================================================================
+PEST SPECIES ANALYSIS - STATISTICS SUMMARY
+================================================================================
+
+1. OVERALL SPECIES COUNTS
+----------------------------------------
+Total species: 126
+
+By Region:
+  US        :  80 species ( 63.5%)
+  Africa    :  35 species ( 27.8%)
+  India     :  11 species (  8.7%)
+
+2. INSECT VS WEED DISTRIBUTION
+----------------------------------------
+Overall:
+  Insects: 65 (51.6%)
+  Weeds:   59 (46.8%)
+
+By Region:
+  US:
+    Insects: 44
+    Weeds:   36
+  Africa:
+    Insects: 10
+    Weeds:   23
+  India:
+    Insects: 11
+    Weeds:   0
+
+3. ACCURACY STATISTICS
+----------------------------------------
+US:
+  Mean:      89.69%
+  Median:    91.00%
+  Std Dev:   11.74%
+  Min:       40.00%
+  Max:      100.00%
+  Missing:  19 (23.8%)
+
+Africa:
+  Mean:      89.81%
+  Median:    95.00%
+  Std Dev:   10.83%
+  Min:       59.00%
+  Max:      100.00%
+  Missing:  8 (22.9%)
+
+India:
+  Mean:      80.00%
+  Median:    83.00%
+  Std Dev:   10.69%
+  Min:       59.00%
+  Max:       95.00%
+  Missing:  1 (9.1%)
+
+Overall (all regions):
+  Mean:      88.73%
+  Median:    90.00%
+  Std Dev:   11.67%
+  Range:    40.00% - 100.00%
+
+4. IPM INFORMATION COVERAGE
+----------------------------------------
+US        :  0/80 species (  0.0%)
+Africa    : 35/35 species (100.0%)
+India     : 10/11 species ( 90.9%)
+Overall   : 45/126 species ( 35.7%)
+
+5. TOP 10 SPECIES BY ACCURACY
+----------------------------------------
+ 1. Seedcorn beetle                (stenolophus lecontei     ) - 100.0% [US]
+ 2. Seedcorn maggot                (delia platura            ) - 100.0% [US]
+ 3. Hop Vine Borer                 (hydraecia immanis        ) - 100.0% [US]
+ 4. Barnyardgrass                  (echinochloa crus-galli   ) - 100.0% [US]
+ 5. common Cocklebur               (xanthium strumarium      ) - 100.0% [US]
+ 6. common Lambsquarters           (chenopodium album        ) - 100.0% [US]
+ 7. CommonWaterhemp                (amaranthus tuberculatus  ) - 100.0% [US]
+ 8. Gaint ragweed                  (ambrosia trifida         ) - 100.0% [US]
+ 9. Henbit (deadnettle)            (lamium amplexicaule      ) - 100.0% [US]
+10. Jimsonweed                     (datura stramonium        ) - 100.0% [US]
+
+6. BOTTOM 10 SPECIES BY ACCURACY
+----------------------------------------
+ 1. Annual ryegrass                (lolium multiflorum       ) -  40.0% [US]
+ 2. Spotted fireworm               (choristoneura parallela  ) -  44.0% [US]
+ 3. Cowpea aphid                   (Aphis craccivora         ) -  59.0% [Africa]
+ 4. Cowpea aphid                   (Aphis craccivora         ) -  59.0% [India]
+ 5. Spiraea Aphid                  (Aphis spiraecola         ) -  67.0% [Africa]
+ 6. Spiraea Aphid                  (Aphis spiraecola         ) -  67.0% [India]
+ 7. alfalfa weevil                 (hypera postica           ) -  73.0% [US]
+ 8. twospotted spider mite         (tetranychus urticae      ) -  73.0% [US]
+ 9. Corn ear borer                 (Helicoverpa armigera     ) -  74.0% [Africa]
+10. Corn ear borer                 (Helicoverpa armigera     ) -  74.0% [India]
+
+7. SPECIES OVERLAP ACROSS REGIONS
+----------------------------------------
+US & Africa:      2 species
+  - amaranthus tuberculatus
+  - cyperus esculentus
+
+US & India:       1 species
+  - spodoptera frugiperda
+
+Africa & India:   10 species
+  - aphis craccivora
+  - aphis spiraecola
+  - atherigona reversura
+  - drosophila suzukii
+  - euborellia annulipes
+  - halyomorpha halys
+  - helicoverpa armigera
+  - icerya purchasi
+  - nezara viridula
+  - spodoptera litura
+
+All three regions: 0 species
+
+================================================================================
+Analysis completed on: 2025-10-26
+================================================================================

species-organized/species_table.tex

@@ -0,0 +1,151 @@
+% LaTeX Table: Species Analysis
+% Requires: \usepackage{booktabs, longtable}
+
+\begin{longtable}{llllrr}
+\toprule
+\textbf{Region} & \textbf{Species} & \textbf{Common Name} & \textbf{Tag} & \textbf{Accuracy (\%)} & \textbf{IPM Info} \\
+\midrule
+\endfirsthead
+
+\multicolumn{6}{c}
+{\tablename\ \thetable\ -- \textit{Continued from previous page}} \\
+\toprule
+\textbf{Region} & \textbf{Species} & \textbf{Common Name} & \textbf{Tag} & \textbf{Accuracy (\%)} & \textbf{IPM Info} \\
+\midrule
+\endhead
+
+\midrule
+\multicolumn{6}{r}{\textit{Continued on next page}} \\
+\endfoot
+
+\bottomrule
+\endlastfoot
+
+Africa & \textit{Halyomorpha halys} & Brown Marmorated Stink Bug & insect & 95.0 & Yes \\
+Africa & \textit{Nezara viridula} & Green stink bug & insect & 88.0 & Yes \\
+Africa & \textit{Drosophila suzukii} & Spotted-winged Drosophila & insect & 86.0 & Yes \\
+Africa & \textit{Spodoptera litura} & Tobacco caterpillar & insect & 86.0 & Yes \\
+Africa & \textit{Atherigona reversura} & Shoot fly & insect & 84.0 & Yes \\
+Africa & \textit{Icerya purchasi} & Cottony cushion scale & insect & 82.0 & Yes \\
+Africa & \textit{Euborellia annulipes} & Ring-legged Earwig & insect & 79.0 & Yes \\
+Africa & \textit{Helicoverpa armigera} & Corn ear borer & insect & 74.0 & Yes \\
+Africa & \textit{Aphis spiraecola} & Spiraea Aphid & insect & 67.0 & Yes \\
+Africa & \textit{Aphis craccivora} & Cowpea aphid & insect & 59.0 & Yes \\
+Africa & \textit{Trianthema triquetrum} & Red Spinach & weed & — & Yes \\
+Africa & \textit{Trianthema portulacastrum} & Desert Horse Purslane & weed & — & Yes \\
+Africa & \textit{Cleome rutidosperma} & Purple spider flower & weed & — & Yes \\
+Africa & \textit{Cleome gynandra} & Spiderwisp & weed & — & Yes \\
+Africa & \textit{Cleome viscosa} & Asian Spider flower & weed & — & Yes \\
+Africa & \textit{Cleome spinosa} & Spiny Spider-Flower & weed & — & Yes \\
+Africa & \textit{Cleome aculeata} & Prickly Spiderflower & weed & — & Yes \\
+Africa & \textit{Cleome monophylla} & Singleleaf Spindlepod & weed & — & Yes \\
+Africa & \textit{Amaranthus viridis} & Green Amaranth & weed & 95.0 & Yes \\
+Africa & \textit{Cyperus entrerianus} & Deeproot Sedge & weed & 95.0 & Yes \\
+Africa & \textit{Cyperus esculentus} & Yellow nutsedge & weed & 95.0 & Yes \\
+Africa & \textit{Cyperus haspan} & Haspan flatsedge & weed & 95.0 & Yes \\
+Africa & \textit{Cyperus iria L.} & Rice flatsedge & weed & 90.0 & Yes \\
+Africa & \textit{Cyperus rotundus} & Purple Nutsedge & weed & 90.0 & Yes \\
+Africa & \textit{Medicago minima} & Little Bur-clover & weed & 90.0 & Yes \\
+Africa & \textit{Cyperus prolifer} & Dwarf papyrus & weed & 80.0 & Yes \\
+Africa & \textit{Amaranthus tuberculatus} & Tall waterhemp & weed & 100.0 & Yes \\
+Africa & \textit{Cyperus brevifolius} & Shortleaf flatsedge & weed & 100.0 & Yes \\
+Africa & \textit{Cyperus difformis} & Smallflower umbrella sedge & weed & 100.0 & Yes \\
+Africa & \textit{Cyperus mindorensis} & nan & weed & 100.0 & Yes \\
+Africa & \textit{Cleome houtteana} & Spider flower & weed & 100.0 & Yes \\
+Africa & \textit{Medicago falcata} & Yellow alfalfa & weed & 100.0 & Yes \\
+Africa & \textit{Medicago lupulina} & Black Medick & weed & 100.0 & Yes \\
+Africa & \textit{Medicago polymorpha} & Burr Medic & — & 95.0 & Yes \\
+Africa & \textit{Striga asiatica} & Witch weed & — & 100.0 & Yes \\
+India & \textit{Spodoptera frugiperda} & Fall Armyworm & insect & — & No \\
+India & \textit{Halyomorpha halys} & Brown Marmorated Stink Bug & insect & 95.0 & Yes \\
+India & \textit{Nezara viridula} & Green stink bug & insect & 88.0 & Yes \\
+India & \textit{Drosophila suzukii} & Spotted-winged Drosophila & insect & 86.0 & Yes \\
+India & \textit{Spodoptera litura} & Tobacco caterpillar & insect & 86.0 & Yes \\
+India & \textit{Atherigona reversura} & Shoot fly & insect & 84.0 & Yes \\
+India & \textit{Icerya purchasi} & Cottony cushion scale & insect & 82.0 & Yes \\
+India & \textit{Euborellia annulipes} & Ring-legged Earwig & insect & 79.0 & Yes \\
+India & \textit{Helicoverpa armigera} & Corn ear borer & insect & 74.0 & Yes \\
+India & \textit{Aphis spiraecola} & Spiraea Aphid & insect & 67.0 & Yes \\
+India & \textit{Aphis craccivora} & Cowpea aphid & insect & 59.0 & Yes \\
+US & \textit{chaetocnema pulicaria} & corn flea beetle & insect & — & No \\
+US & \textit{hypera zoilus} & clover leaf weevil & insect & — & No \\
+US & \textit{agromyza frontella} & alfalfa blotch leafminer & insect & — & No \\
+US & \textit{resseliella maxima} & soybean gall midge & insect & — & No \\
+US & \textit{aphis glycines} & soybean aphid & insect & — & No \\
+US & \textit{Damsel bugs} & Damsel bugs & insect & — & No \\
+US & \textit{Flower fly larvae} & Flower fly larvae & insect & — & No \\
+US & \textit{Ground beetles} & Ground beetles & insect & — & No \\
+US & \textit{Lacewings} & Lacewings & insect & — & No \\
+US & \textit{Lady beetles} & Lady beetles & insect & — & No \\
+US & \textit{Parasitoid wasps} & Parasitoid wasps & insect & — & No \\
+US & \textit{Pirate bugs} & Pirate bugs & insect & — & No \\
+US & \textit{Soldier beetles} & Soldier beetles & insect & — & No \\
+US & \textit{Podisus maculiventris} & Spined soldier bug & insect & — & No \\
+US & \textit{Tachinid flies} & Tachinid flies & insect & — & No \\
+US & \textit{empoasca fabae} & potato leafhopper & insect & 97.0 & No \\
+US & \textit{striacosta albicosta} & western bean cutworm & insect & 97.0 & No \\
+US & \textit{hypena scabra} & green cloverworm & insect & 96.0 & No \\
+US & \textit{agrotis ipsilon} & black cutworm & insect & 95.0 & No \\
+US & \textit{vanessa cardui} & painted lady & insect & 95.0 & No \\
+US & \textit{popillia japonica} & Japanese beetle & insect & 94.0 & No \\
+US & \textit{mythimna unipuncta} & armyworm & insect & 94.0 & No \\
+US & \textit{lygus lineolaris} & tarnished plant bug & insect & 92.0 & No \\
+US & \textit{colias eurytheme} & alfalfa caterpillar & insect & 91.0 & No \\
+US & \textit{microtechnites bractatus} & garden fleahopper & insect & 90.0 & No \\
+US & \textit{papaipema nebris} & stalk borer & insect & 90.0 & No \\
+US & \textit{sitona hispidulus} & clover root curculio & insect & 89.0 & No \\
+US & \textit{philaenus spumarius} & meadow spittlebug & insect & 89.0 & No \\
+US & \textit{dectes texanus} & dectes stem borer & insect & 88.0 & No \\
+US & \textit{ostrinia nubilalis} & European corn borer & insect & 88.0 & No \\
+US & \textit{cerotoma trifurcata} & bean leaf beetle & insect & 87.0 & No \\
+US & \textit{helicoverpa zea} & Tomato fruitworm & insect & 87.0 & No \\
+US & \textit{spodoptera ornithogalli} & yellowstriped armyworm & insect & 86.0 & No \\
+US & \textit{chrysodeixis includens} & soybean looper & insect & 83.0 & No \\
+US & \textit{spodoptera frugiperda} & fall armyworm & insect & 80.0 & No \\
+US & \textit{calomycterus setarius} & imported longhorned weevil & insect & 79.0 & No \\
+US & \textit{loxostege cereralis} & alfalfa webworm & insect & 79.0 & No \\
+US & \textit{odontota horni} & Soybean leaf miner & insect & 75.0 & No \\
+US & \textit{hypera postica} & alfalfa weevil & insect & 73.0 & No \\
+US & \textit{tetranychus urticae} & twospotted spider mite & insect & 73.0 & No \\
+US & \textit{choristoneura parallela} & Spotted fireworm & insect & 44.0 & No \\
+US & \textit{stenolophus lecontei} & Seedcorn beetle & insect & 100.0 & No \\
+US & \textit{delia platura} & Seedcorn maggot & insect & 100.0 & No \\
+US & \textit{hydraecia immanis} & Hop Vine Borer & insect & 100.0 & No \\
+US & \textit{solanum ptycanthum} & Eastern black nightshade & weed & — & No \\
+US & \textit{conyza canadensis} & Horseweed & weed & — & No \\
+US & \textit{kochia scoparia} & Kochia & weed & — & No \\
+US & \textit{sinapis arvensis} & Wild mustard & weed & — & No \\
+US & \textit{ambrosia artemisiifolia} & common Ragweed & weed & 95.0 & No \\
+US & \textit{stellaria media} & commonChickweed & weed & 95.0 & No \\
+US & \textit{equisetum arvense} & Field Horsetail & weed & 95.0 & No \\
+US & \textit{digitaria sanguinalis} & Large crabgrass & weed & 95.0 & No \\
+US & \textit{sida spinosa} & Prickly sida & weed & 95.0 & No \\
+US & \textit{cyperus esculentus} & yellow Nutsedge & weed & 95.0 & No \\
+US & \textit{helianthus annuus} & Common Sunflower & weed & 90.0 & No \\
+US & \textit{bromus tectorum} & Downy brome & weed & 90.0 & No \\
+US & \textit{setaria viridis} & Green foxtail & weed & 90.0 & No \\
+US & \textit{euphorbia dentata} & Toothed spurge & weed & 90.0 & No \\
+US & \textit{mirabilis nyctaginea} & wild Four-o’clock & weed & 90.0 & No \\
+US & \textit{setaria faberi} & Giant foxtail & weed & 85.0 & No \\
+US & \textit{eleusine indica} & Goosegrass & weed & 85.0 & No \\
+US & \textit{salsola tragus} & Russian thistle & weed & 85.0 & No \\
+US & \textit{sorghum bicolor} & Shattercane & weed & 85.0 & No \\
+US & \textit{setaria pumila} & Yellow foxtail & weed & 85.0 & No \\
+US & \textit{persicaria pensylvanica} & Pennsylvania smartweed & weed & 80.0 & No \\
+US & \textit{amaranthus palmeri} & Palmer amaranth & weed & 75.0 & No \\
+US & \textit{lolium multiflorum} & Annual ryegrass & weed & 40.0 & No \\
+US & \textit{echinochloa crus-galli} & Barnyardgrass & weed & 100.0 & No \\
+US & \textit{xanthium strumarium} & common Cocklebur & weed & 100.0 & No \\
+US & \textit{chenopodium album} & common Lambsquarters & weed & 100.0 & No \\
+US & \textit{amaranthus tuberculatus} & CommonWaterhemp & weed & 100.0 & No \\
+US & \textit{ambrosia trifida} & Gaint ragweed & weed & 100.0 & No \\
+US & \textit{lamium amplexicaule} & Henbit (deadnettle) & weed & 100.0 & No \\
+US & \textit{datura stramonium} & Jimsonweed & weed & 100.0 & No \\
+US & \textit{lactuca serriola} & Prickly lettuce & weed & 100.0 & No \\
+US & \textit{amaranthus retroflexus} & Redroot pigweed & weed & 100.0 & No \\
+US & \textit{equisetum hyemale} & Scouringrush & weed & 100.0 & No \\
+US & \textit{capsella bursa-pastoris} & Shepherd’s purse & weed & 100.0 & No \\
+US & \textit{abutilon theophrasti} & Velvetleaf & weed & 100.0 & No \\
+US & \textit{daucus carota} & Wild Carrot & weed & 100.0 & No \\
+
+\end{longtable}

vector-databases-deployed/db5-agllm-data-isu-field-insects-all-species/{8da9893a-19f6-48c6-bb16-8a169d9e166f → 0bdb47f3-00af-43ed-a2af-ae5a3eee5f98}/length.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b274da292d64f026adecde33133c35635f3faf9e38eee883d259dcf632c7729b
+oid sha256:11491cf0eac47e805aa1b059bb8d72b895d20b41d24581b6a4383eff57db12f5
 size 40000

vector-databases-deployed/db5-agllm-data-isu-field-insects-all-species/chroma.sqlite3

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:717b0646137d385b2777333886c81f41d57bae3261a881b66c728a21e465c29b
-size 5414912
+oid sha256:12653e79b55a19108699f56736a4d97a4ad00f3627d6504348862d911eaa1688
+size 5410816