The multimodal models represent a significant step towards General Artificial Intelligence (AGI), a theoretical concept that aims for AI to perform any intellectual task by “thinking” with the versatility and breadth of a human being.

As a multimodal example, Visual Language Models (VLMs) are capable of processing both text and images, enabling a more versatile and human interaction.

World's First IDEFICS Adaptation

With this in mind, our research and development laboratory, Clibrain Labs, has carried out the world's first adaptation of IDEFICS (Hugging Face), an open-source reproduction of the most advanced visual language model to date, Flamingo (Google Deepmind).

This adaptation has been achieved through fine-tuning techniques, making use of a large-scale dataset of text and images comprising more than 14 million images.

Trained for Midjourney-Style Image Descriptions

As a result, the model is adapted to generate descriptions of images in the style of prompts from platforms such as Midjourney or Stable Difussion (text-to-image diffusion models).

This functionality improves the ability to communicate with image generation models (diffusion models), helping to understand how the model “describes” the images and use it as a starting point to generate similar images or modifications of the same.

The Midjourney platform offers similar functionality, but it is a paid service. By releasing it as open-source, we enable everyone to use it for free.

Commitment to the Open-Source Community

In line with our commitment to research and the open-source community, we have published the model on Hugging Face so that everyone can make use of it.

Along with this, we share the procedure and techniques that have been followed to train the model, as well as the steps to perform inference on the same.

You can find the adaptation of IDEFICS and the rest of our open-source models at hf.co/clibrain