segment unstructured import into batches of 10000 files to keep within buffer limits, structured imports should be fine as we are talking 1 sd card (24/7 its 16000 max)
restrict db exports to structured, or handle unstructured : Andrew Digby
(base) skraak ➤ ./skraak export dataset --db ./db/skraak.duckdb --id 4Sh8_7p1ocks --output db/export.duckdb
Error: failed to copy file_dataset: Constraint Error: Violates foreign key constraint because key "id: CwN4rQCeOs2J4lOvijhxa" does not exist in the referenced table
ingest my training datasets 
buy a drive to backup mac ~
include skraak metadata in written .data files at the end of relevant section BUT ONLY on insert to db at end, final review.
file level metadata: skraak_dataset_id, skraak_location_id, skraak_cluster_id, skraak_file_id, skraak_file_hash
label level metadata: skraak_segment_id, skraak_label_id
operator and reviewer in file metadata
make a psuedo schema for file and label metadata
Update tools could allow setting active to false??
look at nulls in schema, problem with updatde tool
Just the simplest clip viewer and sort in a tui, from db data not avianz data
Make segments from pred and save as json??? / insert into db
Make clip dynamicly in Julia and sort using model, get data from db, modify db weth changes
Img Viewer renders from db and files
User confirms
Update db, inc certainty 
Make export clips tool 
Edit selection tool to change start end. Frameing 
Make import avianz, add skraak metadata
Where do I add skraak metadata 
Make import freebird tool
Make import raven
##### NEED TO MERGE LABELS ###################
SELECT id, label, ebird_code, description, active
  FROM species
  WHERE ebird_code IN ('lotkoe1', 'liskiw1', 'grskiw1', 'okbkiw1', 'sobkiw1', 'nibkiw1');
┌──────────────┬──────────────────────┬────────────┬───────────────────────────────────────────────┬─────────┐
│      id      │        label         │ ebird_code │                  description                  │ active  │
│   varchar    │       varchar        │  varchar   │                    varchar                    │ boolean │
├──────────────┼──────────────────────┼────────────┼───────────────────────────────────────────────┼─────────┤
│ CNi9X-Hkz4-A │ Kiwi_Brown           │ sobkiw1    │ FreebirdCode: 109,  BiowebCode: 10857,  Com…  │ false   │
│ D_pepf6f08kn │ Kiwi_Great_Spotted   │ grskiw1    │ FreebirdCode: 110,  BiowebCode: 10860,  Com…  │ false   │
│ di-jxVg8tcVO │ Kiwi_Little_Spotted  │ liskiw1    │ FreebirdCode: 111,  BiowebCode: 10859,  Com…  │ false   │
│ dZLV3Ue916E- │ Kiwi_Nth_Is_Brown    │ nibkiw1    │ FreebirdCode: 112,  BiowebCode: 10856,  Com…  │ false   │
│ 74yg7hquUKT9 │ Kiwi_Okarito_Brown   │ okbkiw1    │ FreebirdCode: 113,  BiowebCode: 13151,  Com…  │ false   │
│ ac0AgauTMvMT │ Kiwi_spp             │ sobkiw1    │ FreebirdCode: 114,  BiowebCode: 29709,  Com…  │ false   │
│ 4JOC8g7OsLGW │ Cuckoo_long-tailed   │ lotkoe1    │ FreebirdCode: 35,  BiowebCode: 11248, Long-…  │ false   │
│ rqIcB4EkSy2G │ Long-tailed Koel     │ lotkoe1    │ NULL                                          │ true    │
│ eYZ6Ns7XNbwR │ Kiwi pukupuku        │ liskiw1    │ Kiwi pukupuku / Little Spotted Kiwi           │ true    │
│ TSwVuQdE0cEA │ Roroa                │ grskiw1    │ Roroa / Great Spotted Kiwi                    │ true    │
│ fDGmXv9c6xEn │ Rowi                 │ okbkiw1    │ Rowi / Okarito Brown Kiwi                     │ true    │
│ jqEQrViqRFFX │ S. Fiordland Tokoeka │ sobkiw1    │ Southern Fiordland Tokoeka                    │ true    │
│ ANhV8iZPfIh8 │ Haast Tokoeka        │ sobkiw1    │ NULL                                          │ true    │
│ 7LNNjRcptMev │ Kiwi                 │ nibkiw1    │ Kiwi-nui / North Island Brown Kiwi            │ true    │
├──────────────┴──────────────────────┴────────────┴───────────────────────────────────────────────┴─────────┤
│ 14 rows                                                                                          5 columns │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
on the duckdb cli How can i append this description
│ dZLV3Ue916E- │ Kiwi_Nth_Is_Brown    │ nibkiw1    │ FreebirdCode: 112,  BiowebCode: 10856,  Com…  │ false   │
to the following, and also overwrite the Kiwi label with Kiwi_Nth_Is_Brown
7LNNjRcptMev │ Kiwi                 │ nibkiw1    │ Kiwi-nui / North Island Brown Kiwi            │ true
then delete dZLV3Ue916E-
# Does not work.
UPDATE species
SET
    label = 'Kiwi_Nth_Is_Brown',
    description = description || ', ' || (
        SELECT description
        FROM species
        WHERE id = 'dZLV3Ue916E-'
    )
WHERE id = '7LNNjRcptMev';
DELETE FROM my_table
WHERE id = 'dZLV3Ue916E-';
time ./skraak calls from-preds --csv /media/david/SSD4/Twenty_Four_Seven/R620/2024-05-06/preds9_opensoundscape-multi-1.0_2025-07-22.csv > /media/david/SSD4/Twenty_Four_Seven/R620/2024-05-06/preds9_opensoundscape-multi-1.0_2025-07-22.json
time ./skraak calls from-preds --csv /media/david/SSD4/Twenty_Four_Seven/R620/2025-02-25/preds9_opensoundscape-multi-1.0_2025-07-22.csv > /media/david/SSD4/Twenty_Four_Seven/R620/2025-02-25/preds9_opensoundscape-multi-1.0_2025-07-22.json
./skraak calls classify \
        --folder /media/david/SSD4/Pomona/C05/2025-11-08/ \
        --reviewer David \
        --bind k=Kiwi \
        --bind d="Kiwi+Duet" \
        --bind f="Kiwi+Female" \
        --bind m="Kiwi+Male" \
        --bind n="Don't Know" \
        --bind p=Morepork \
        --color \
        --img-dims 224
./skraak calls classify \
        --folder /media/david/SSD4/Twenty_Four_Seven/R620/2024-05-06/ \
        --reviewer David \
        --bind k=Kiwi \
        --bind d="Kiwi+Duet" \
        --bind f="Kiwi+Female" \
        --bind m="Kiwi+Male" \
        --bind n="Don't Know" \
        --bind p=Morepork \
        --color
file,start_time,end_time,ausbit1,bluduc1,comcha,comred,dunnoc1,eurbla,eursta,fernbi1,grskiw1,gryger1,kea1,liskiw1,lotkoe1,morepo2,nezbel1,nezfan1,nezkak1,nezpig2,nezrob3,nibkiw1,okbkiw1,parake,pipipi1,riflem1,shbcuc1,silver3,sobkiw2,soioys1,soiwre1,sonthr1,spocra2,tomtit1,tui1,varoys1,weka1,yellow2,weta,saddle3
ausbit1       Australasian Bittern
bluduc1       Blue Duck                                                              
comcha        Common Chaffinch
comred        Redpoll (Common)
dunnoc1       Dunnock
eurbla        Eurasian Blackbird
eursta        European Starling
fernbi1       New Zealand Fernbird
grskiw1       Great Spotted Kiwi
gryger1       Gray Gerygone
kea1          Kea
liskiw1       Little Spotted Kiwi                                                    
lotkoe1       Long-tailed Koel                                                       
morepo2       Morepork
nezbel1       New Zealand Bellbird
nezfan1       New Zealand Fantail
nezkak1       New Zealand Kaka
nezpig2       New Zealand Pigeon                                                     
nezrob3       South Island Robin
nibkiw1       North Island Brown Kiwi
okbkiw1       Okarito Brown Kiwi
parake        parakeet sp.                                                           
pipipi1       Pipipi
riflem1       Rifleman
saddle3       South Island Saddleback
shbcuc1       Shining Bronze-Cuckoo
silver3       Silvereye
sobkiw2       Southern Brown Kiwi (South I.)
soioys1       South Island Oystercatcher
soiwre1       South Island Wren
sonthr1       Song Thrush
spocra2       Spotless Crake
tomtit1       Tomtit
tui1          Tui
varoys1       Variable Oystercatcher                                                 
weka1         Weka
yellow2       Yellowhammer
weta
bluduc1       Duck_Blue_Whio  Blue Duck
comcha        Chaffinch       Common Chaffinch
dunnoc1       Dunnock_Hedge_Sparrow   Dunnock
fernbi1       Fernbird        New Zealand Fernbird
fernbi1       Fernbird_Nth Is New Zealand Fernbird
fernbi1       Fernbird_Sth Is New Zealand Fernbird                                   
grskiw1       Kiwi_Great_Spotted      Great Spotted Kiwi
grskiw1       Roroa   Great Spotted Kiwi
gryger1       Warbler_Grey    Gray Gerygone
kea1  Kea     Kea
liskiw1       Kiwi pukupuku   Little Spotted Kiwi
liskiw1       Kiwi_Little_Spotted     Little Spotted Kiwi                            
lotkoe1       Cuckoo_long-tailed      Long-tailed Koel
lotkoe1       Long-tailed Koel        Long-tailed Koel                               
morepo2       Morepork        Morepork                                               
nezbel1       Bellbird        New Zealand Bellbird
nezbel1       Bellbird_Poor_Knights   New Zealand Bellbird                           
nezbel1       Bellbird_Three_Kings    New Zealand Bellbird
nezfan1       Fantail New Zealand Fantail
nezfan1       Fantail_Chatham_Is      New Zealand Fantail
nezfan1       Fantail_Nth_Is  New Zealand Fantail
nezfan1       Fantail_Sth_Is  New Zealand Fantail                                    
nezkak1       Kaka    New Zealand Kaka
nezrob3       Robin_Stewart_Is        South Island Robin
nezrob3       Robin_Sth_Is    South Island Robin
nibkiw1       Kiwi    North Island Brown Kiwi                                        
nibkiw1       Kiwi_Nth_Is_Brown       North Island Brown Kiwi
okbkiw1       Kiwi_Okarito_Brown      Okarito Brown Kiwi                             
okbkiw1       Rowi    Okarito Brown Kiwi
riflem1       Rifleman        Rifleman
riflem1       Rifleman_Nth_Is Rifleman                                               
riflem1       Rifleman_Sth_Is Rifleman                                               
saddle3       Saddleback_Sth_Is       South Island Saddleback                        
shbcuc1       Cuckoo_Shining  Shining Bronze-Cuckoo
silver3       Silvereye       Silvereye
sobkiw2       Kiwi_Tokoeka_Haast      Southern Brown Kiwi (South I.)                 
sonthr1       Thrush_Song     Song Thrush                                            
spocra2       Crake_Spotless  Spotless Crake
tomtit1       Tomtit  Tomtit
tomtit1       Tomtit_Auckland_Is      Tomtit
tomtit1       Tomtit_Chatham_Is       Tomtit
tomtit1       Tomtit_Nth_Is   Tomtit
tomtit1       Tomtit_Snares_Is_Black  Tomtit
tomtit1       Tomtit_Sth_Is   Tomtit                                                 
tui1  Tui     Tui
tui1  Tui_Chatham_Is  Tui
varoys1       Oystercatcher_Variable  Variable Oystercatcher
weka1 Weka_Buff       Weka
weka1 Weka_Nth_Is     Weka
weka1 Weka_Stewart_Is Weka
weka1 Weka_Western    Weka
weka1 Weka_spp        Weka                                                           
yellow2       Yellowhammer    Yellowhammer
Not in species table: ausbit1, comred, eurbla, eursta, nezpig2, parake, pipipi1, soioys1, soiwre1, weta
c=comcha        Chaffinch       Common Chaffinch
d=dunnoc1       Dunnock_Hedge_Sparrow   Dunnock
b=eurbla        Eurasian Blackbird
g=gryger1       Warbler_Grey    Gray Gerygone
k=kea1  Kea     Kea
l=lotkoe1       Long-tailed Koel        Long-tailed Koel
p=malpar2       Kakariki     Malherbe's Parakeet (Orange-fronted Parakeet)                             
m=morepo2       Morepork        Morepork                                               
nezbel1       Bellbird        New Zealand Bellbird
nezfan1       Fantail New Zealand Fantail
nezkak1       Kaka    New Zealand Kaka
nezpig2       New Zealand Pigeon
nezrob3       Robin_Sth_Is    South Island Robin
pipipi1       Pipipi          Brown Creeper
riflem1       Rifleman        Rifleman
saddle3       Saddleback_Sth_Is       South Island Saddleback                        
shbcuc1       Cuckoo_Shining  Shining Bronze-Cuckoo
silver3       Silvereye       Silvereye
sobkiw2       Kiwi_Tokoeka_Haast      Southern Brown Kiwi (South I.)
soioys1       South Island Oystercatcher                 
sonthr1       Thrush_Song     Song Thrush                                            
tomtit1       Tomtit  Tomtit                                          
tui1  Tui     Tui
varoys1       Oystercatcher_Variable  Variable Oystercatcher
yellow2       Yellowhammer    Yellowhammer

*.wav
*.WAV

xxh-tools/

.pi/

docs/
.pi/
*.parquet
*.jsonl
me.txt
example/

*.wav
*.WAV
*.parquet
*.jsonl