simpleParadox/seed_0_git_causal_image_caption_only_standard_final_model_8_epochs

0.2B • Updated Aug 30, 2024 • 7 • 1

Note This is GIT model trained on the image-caption pairs only (50M tokens) using standard i.i.d training for 8 epochs.

simpleParadox/seed_0_git_causal_image_caption_only_curriculum_final_model_8_epochs

0.2B • Updated Aug 30, 2024 • 6

Note This is GIT model trained on the image-caption pairs only (50M tokens) using curriculum training for 8 epochs.

simpleParadox/seed_0_git_causal_initialize_with_text_image_caption_only_standard_final_model_8_epochs

0.2B • Updated Sep 2, 2024 • 9

Note This is GIT model first trained on the text-only dataset (50M tokens) using standard i.i.d training, for 20 epochs. Then the model training is continued on the image-caption pairs (50M tokens) using standard i.i.d training for 8 epochs. Total tokens seen by the model is 100M tokens.

simpleParadox/seed_0_git_causal_initialize_with_text_image_caption_only_curriculum_final_model_8_epochs

0.2B • Updated Sep 1, 2024 • 5

Note This is GIT model first trained on the text-only dataset (50M tokens) using standard i.i.d training, for 20 epochs. Then the model training is continued on the image-caption pairs (50M tokens) using curriculum training for 8 epochs. Total tokens seen by the model is 100M tokens.

simpleParadox/seed_0_flamingo_causal_image_caption_only_standard_final_model_8_epochs

0.3B • Updated Aug 31, 2024 • 10

Note This is Flamingo model trained on the image-caption pairs only (50M tokens) using standard i.i.d training for 8 epochs.

simpleParadox/seed_0_flamingo_causal_image_caption_only_curriculum_final_model_8_epochs

0.3B • Updated Aug 30, 2024 • 9

Note This is Flamingo model trained on the image-caption pairs only (50M tokens) using curriculum training for 8 epochs.

simpleParadox/seed_0_flamingo_causal_initialize_with_text_image_caption_only_standard_final_model_8_epochs

0.3B • Updated Sep 5, 2024 • 4

Note This is Flamingo model first trained on the text-only dataset (50M tokens) using standard i.i.d training, for 20 epochs. Then the model training is continued on the image-caption pairs (50M tokens) using standard i.i.d training for 8 epochs. Total tokens seen by the model is 100M tokens.

simpleParadox/seed_0_flamingo_causal_initialize_with_text_image_caption_only_curriculum_final_model_8_epochs

0.3B • Updated Sep 3, 2024 • 6

Note This is Flamingo model first trained on the text-only dataset (50M tokens) using standard i.i.d training, for 20 epochs. Then the model training is continued on the image-caption pairs (50M tokens) using curriculum training for 8 epochs. Total tokens seen by the model is 100M tokens.

simpleParadox/bert_pos_tagger_babylm_2024

0.1B • Updated Sep 13, 2024 • 6

Note This bert should only be used when ranking the image-caption for the ranking of the data. In other words, this is the scoring function that counts the number of nouns. This model should be not be run on the evaluation pipeline.