1
Clone the Repository
Clone the LitGPT GitHub repository to your local machine using git.
2
Install Dependencies
Install required Python packages via pip as specified in the repository documentation.
3
Download a Tokenizer
Use the command `litgpt download EleutherAI/pythia-160m --tokenizer_only True` to download the tokenizer for a model.
4
Pretrain a Model
Run a pretraining session with a command like `litgpt pretrain EleutherAI/pythia-160m --tokenizer_dir EleutherAI/pythia-160m --data TextFiles --data.train_data_path "custom_texts/" --train.max_tokens 10_000_000 --out_dir out/custom-model`.
5
Finetune with LoRA
Execute finetuning using LoRA with a command such as `litgpt finetune lora --config config_hub/finetune/phi-2/lora.yaml`.
6
Test the Model
Test the trained or finetuned model locally using `litgpt chat out/custom-model/final`.