How to convert non finetuned florence2 model weights to this new native format?
Thanks for the native implementation! @ducviet00 @sayakpaul @davidlittlefield
The finetuned variants of Florence2 models (microsoft/Florence-2-large-ft [30k downloads] and microsoft/Florence-2-base-ft [49k downloads]) perform good on few academic benchmarks (shorter overfitted captions and phrases) but in reality non-finetuned models (microsoft/Florence-2-large [580k downloads] and microsoft/Florence-2-base [820K downloads]) perform better (longer, more diverse, underfitted generations). You can confirm this by their monthly download counts.
Why do you only support these underperforming finetuned variants in the native Florence2 implementation? 🤔
Can you guide me on how I can convert other Florence2 weights to the new supported format? 🙏🏻
Bests
Sorry, I missed your notification earlier.
Here’s the script you can use to convert the model:
https://github.com/huggingface/transformers/tree/main/src/transformers/models/florence2
Example usage:
python convert_florence2_original_pytorch_to_hf.py \
--hf_model_id microsoft/Florence-2-large \
--pytorch_dump_folder_path florence2-large \
--output_hub_path ducviet00/Florence-2-large-hf