Set up conda environment for LLM fine-tuning
Sets up a conda environment for LLM fine-tuning with PyTorch, Hugging Face tools, and quantization libraries.
/plugin marketplace add danielrosehill/linux-desktop-plugin/plugin install lan-manager@danielrosehillYou are helping the user set up a conda environment for LLM fine-tuning.
Create base environment
conda create -n llm-finetune python=3.11 -y
conda activate llm-finetune
Install PyTorch with ROCm
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0
Install core fine-tuning libraries
Hugging Face ecosystem:
pip install transformers
pip install datasets
pip install accelerate
pip install evaluate
pip install peft # Parameter-Efficient Fine-Tuning
pip install bitsandbytes # Quantization (may need special build for ROCm)
Training frameworks:
pip install trl # Transformer Reinforcement Learning
pip install deepspeed # Distributed training (if needed)
Install quantization and optimization tools
pip install optimum
pip install auto-gptq # GPTQ quantization
pip install autoawq # AWQ quantization
Install evaluation and monitoring tools
pip install wandb # Weights & Biases for experiment tracking
pip install tensorboard
pip install rouge-score # Text evaluation
pip install sacrebleu # Translation metrics
Install data processing tools
pip install pandas
pip install numpy
pip install scipy
pip install scikit-learn
pip install nltk
pip install spacy
Install specialized fine-tuning tools
pip install axolotl # LLM fine-tuning framework
pip install unsloth # Fast fine-tuning (if compatible with ROCm)
pip install qlora # Quantized LoRA
Install Jupyter for interactive work
conda install -c conda-forge jupyter jupyterlab ipywidgets -y
Create example fine-tuning script
~/scripts/llm-finetune-example.py with basic LoRA setupTest installation
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model
print(f"PyTorch: {torch.__version__}")
print(f"GPU available: {torch.cuda.is_available()}")
print("All libraries imported successfully!")
Provide a summary showing: