Update README.md
Browse files
    	
        README.md
    CHANGED
    
    | @@ -1,3 +1,68 @@ | |
| 1 | 
            -
            ---
         | 
| 2 | 
            -
            license: apache-2.0
         | 
| 3 | 
            -
             | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | 
|  | |
| 1 | 
            +
            ---
         | 
| 2 | 
            +
            license: apache-2.0
         | 
| 3 | 
            +
            pipeline_tag: mask-generation
         | 
| 4 | 
            +
            tags:
         | 
| 5 | 
            +
            - sam2
         | 
| 6 | 
            +
            ---
         | 
| 7 | 
            +
             | 
| 8 | 
            +
            # SAM2-Hiera-base-plus
         | 
| 9 | 
            +
             | 
| 10 | 
            +
            This repository contains base variant of SAM2 model. SAM2 is the state-of-the-art mask generation model released by Meta.
         | 
| 11 | 
            +
             | 
| 12 | 
            +
            ## Usage
         | 
| 13 | 
            +
             | 
| 14 | 
            +
            You can use it like below. First install packaged version of SAM2.
         | 
| 15 | 
            +
             | 
| 16 | 
            +
            ```bash
         | 
| 17 | 
            +
            pip install samv2 huggingface_hub
         | 
| 18 | 
            +
            ```
         | 
| 19 | 
            +
             | 
| 20 | 
            +
            Each model requires different classes to infer. 
         | 
| 21 | 
            +
             | 
| 22 | 
            +
            ```python
         | 
| 23 | 
            +
            from huggingface_hub import hf_hub_download
         | 
| 24 | 
            +
            from sam2.build_sam import build_sam2
         | 
| 25 | 
            +
            from sam2.sam2_image_predictor import SAM2ImagePredictor
         | 
| 26 | 
            +
             | 
| 27 | 
            +
            hf_hub_download(repo_id = "merve/sam2-hiera-base-plus", filename="sam2_hiera_base_plus.pt", local_dir = "./")
         | 
| 28 | 
            +
            sam2_checkpoint = "../checkpoints/sam2_hiera_base_plus.pt"
         | 
| 29 | 
            +
            model_cfg = "sam2_hiera_b+.yaml"
         | 
| 30 | 
            +
             | 
| 31 | 
            +
            sam2_model = build_sam2(config, ckpt, device="cuda", apply_postprocessing=False)
         | 
| 32 | 
            +
            predictor = SAM2ImagePredictor(sam2_model)
         | 
| 33 | 
            +
             | 
| 34 | 
            +
            # it accepts coco format
         | 
| 35 | 
            +
            box = [x1, y1, w, h]
         | 
| 36 | 
            +
            predictor.set_image(image)
         | 
| 37 | 
            +
             | 
| 38 | 
            +
            masks = predictor.predict(box=box,
         | 
| 39 | 
            +
            multimask_output=False)
         | 
| 40 | 
            +
            ```
         | 
| 41 | 
            +
             | 
| 42 | 
            +
            For automatic mask generation:
         | 
| 43 | 
            +
             | 
| 44 | 
            +
            ```python
         | 
| 45 | 
            +
            from huggingface_hub import hf_hub_download
         | 
| 46 | 
            +
            from sam2.build_sam import build_sam2
         | 
| 47 | 
            +
            from sam2.automatic_mask_generator import SAM2AutomaticMaskGenerator
         | 
| 48 | 
            +
             | 
| 49 | 
            +
            hf_hub_download(repo_id = "merve/sam2-hiera-base-plus", filename="sam2_hiera_base_plus.pt", local_dir = "./")
         | 
| 50 | 
            +
            sam2_checkpoint = "../checkpoints/sam2_hiera_base_plus.pt"
         | 
| 51 | 
            +
            model_cfg = "sam2_hiera_b+.yaml"
         | 
| 52 | 
            +
             | 
| 53 | 
            +
            sam2 = build_sam2(model_cfg, sam2_checkpoint, device ='cuda', apply_postprocessing=False)
         | 
| 54 | 
            +
             | 
| 55 | 
            +
            mask_generator = SAM2AutomaticMaskGenerator(sam2)
         | 
| 56 | 
            +
            masks = mask_generator.generate(image)
         | 
| 57 | 
            +
            ```
         | 
| 58 | 
            +
             | 
| 59 | 
            +
             | 
| 60 | 
            +
            ## Resources
         | 
| 61 | 
            +
             | 
| 62 | 
            +
            The team behind SAM2 made example notebooks for all tasks.
         | 
| 63 | 
            +
             | 
| 64 | 
            +
            - See [image predictor example](https://github.com/facebookresearch/segment-anything-2/blob/main/notebooks/image_predictor_example.ipynb) for full example on prompting.
         | 
| 65 | 
            +
             | 
| 66 | 
            +
            - See [automatic mask generation example](https://github.com/facebookresearch/segment-anything-2/blob/main/notebooks/automatic_mask_generator_example.ipynb) for generating all masks.
         | 
| 67 | 
            +
             | 
| 68 | 
            +
            - See [video object segmentation example](https://github.com/facebookresearch/segment-anything-2/blob/main/notebooks/video_predictor_example.ipynb)
         |