lemm-test-100 / backend /services /dataset_service.py

Commit History

Add dataset import & LoRA collection sync - Adds import_prepared_dataset() method to support ZIP dataset imports with name conflict resolution - Adds sync_on_startup() to download LoRAs from HF collection on app startup - Enhanced upload_lora() with training_config for proper metadata - Implements numeric suffix naming for conflicts (_1, _2, etc.)
17f5813

Gamahea commited on

Add persistent dataset detection for HF and user datasets across sessions
6d5bfcd

Gamahea commited on

Remove MusicCaps dataset (metadata only, no audio)
1329490

Gamahea commited on

Remove MTG-Jamendo dataset (not found on Hub)
f7f2f36

Gamahea commited on

Fix dataset IDs: remove Common Voice & FLEURS, fix FSD50K & LibriSpeech
bea0065

Gamahea commited on

Remove MSD (no audio), add FSD50K and Common Voice
6559aff

Gamahea commited on

Add Million Song Dataset (10K subset)
bf66672

Gamahea commited on

Remove broken datasets that don't exist on HuggingFace Hub
56e9a20

Gamahea commited on

Access audio file paths directly to bypass torchcodec
275de5b

Gamahea commited on

Fix audio decoding and duplicate method issue
7a66654

Gamahea commited on

Fix FFmpeg dependencies and improve audio decoding
384696b

Gamahea commited on

Remove VCTK dataset (uses unsupported loading script)
5f63c36

Gamahea commited on

Fix missing is_dataset_downloaded methods
fbc33ea

Gamahea commited on

Re-add LibriSpeech and implement download status tracking
623cee1

Gamahea commited on

Replace broken datasets with working alternatives
b595d0c

Gamahea commited on

Add dataset preparation feature for training
1015ddf

Gamahea commited on

Fix training error - validate HF datasets are prepared
7c2cd8e

Gamahea commited on

Fix dataset loading with Parquet-based datasets
b8b7922

Gamahea commited on

Fix dataset loading errors with Parquet-based datasets
1252ef3

Gamahea commited on

Upgrade to complete datasets with 11TB Pro storage
661fe20

Gamahea commited on

Fix dataset download errors with verified HuggingFace datasets
9a8320c

Gamahea commited on

Add HuggingFace datasets integration for LoRA training
d1b6f86

Gamahea commited on