Update README.md

README.md

@@ -36,6 +36,43 @@ which was the most upvoted comment. If you ask me, I did not expect it to be tha
 
 There are more examples I tried that were "approved" by the community, however it's not always the case and it depends on a lot more things that are out of the commenter's control, such as the popularity of the post and timing. In general, for most posts I've tried the comments generated were reasonable for my expectations.
 
+### How to Get Started with the Model
+
+Use the code below to get started with the model
+
+```python
+from peft import PeftModel, PeftConfig
+from transformers import AutoProcessor, LlavaForConditionalGeneration
+from PIL import Image
+import torch
+
+# Load the original model and processor (This parts downloads Pixtral-12B)
+model_id = "mistral-community/pixtral-12b"
+model = LlavaForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.float16, device_map="cuda") # Or "mps" for MacOS or "auto" for multiple Nvidia GPUs
+processor = AutoProcessor.from_pretrained(model_id)
+
+# Load the LoRA configuration (This parts downloads current repo)
+peft_config = PeftConfig.from_pretrained("AlexandrosChariton/reddit-pixtral-12B-Lora-v4")
+lora_model = PeftModel.from_pretrained(model, "AlexandrosChariton/reddit-pixtral-12B-Lora-v4")
+
+# You need a meme in image format to run the model
+image_path = "meme_image.png" # or webp, jpg, jpeg
+image_path = "seriously-how-are-you-doing-this-v0-n7drhacsba6e1.webp"
+meme_title = "I hate it when this happens to me"
+
+# Resize image to 512x512. This is not necessary but I used 512x512 during training
+image = Image.open(image_path).convert("RGB")
+image.thumbnail((512, 512))
+
+# Chat template is applied to the prompt manually here
+PROMPT = f"<s>[INST]Come up with a comment that will get upvoted by the community for a reddit post in r/memes. Provide the comment body with text and nothing else. The post has title: '{meme_title}' and image:\n[IMG][/INST]"
+inputs = processor(text=PROMPT, images=image, return_tensors="pt").to("cuda") # or "mps" for MacOS
+
+generate_ids = lora_model.generate(**inputs, max_new_tokens=650)
+output = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
+print(output)
+```
+
 ### Model Description
 I fine tuned the model using a sample of popular comments from posts that I extracted using Reddit's python API.
 I started with comments with > 10 upvotes and did some basic filtering, removing long and low quality comments based on my personal standards. About 3% of the model's parameters were trainable. I used 1.5k posts and 12k total comments as training data.
@@ -44,26 +81,9 @@ The prompt was pretty vanilla, without any carefully crafted prompt engineering
 You may notice that this repo has a v4 in the title, that's because I tried this training a number of times but I tried my best to filter the training data as much as possible, without greatly affecting the goal of getting upvotes.
 This decision was made after the previous versions ended up producing text that was pure misinformation or inappropriate. I thought it would be best to avoid extreme cases so I did not make the other versions public. Still, the text produced is likely to be weird (for the lack of a better word), given that the language model was exposed to popular Reddit comments. Also, I filtered out posts that one might consider controversial that were quite popular in the last few days.
 
-
-
 - **Developed by:** me
-- **Funded by [optional]:** me :(
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Funded by:** me :(
 
 ## Uses
 
-For fun and for limit testing text imprinted on images, commonly found in memes :)
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model
-Coming soon
-
-- PEFT 0.14.0
+For fun and for limit testing text imprinted on images, commonly found in memes :)