608 264 697

Daniel van Strien PRO

davanstrien

https://danielvanstrien.xyz/

AI & ML interests

Machine Learning Librarian

Articles

Introducing Synthetic Data Workshop: Your Gateway to Easy Synthetic Dataset Creation

Jun 20

• 12

Data Is Better Together: A Look Back and Forward

Jun 20

• 18

Synthetic dataset generation techniques: generating custom sentence similarity data

May 23

• 14

Synthetic dataset generation techniques: Self-Instruct

May 15

• 6

Can we create pedagogically valuable multi-turn synthetic datasets from Cosmopedia?

May 7

• 7

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 59

Extracting Insights from Model Cards Using Open Large Language Models

Nov 27, 2023

Creating open machine learning datasets? Share them on the Hugging Face Hub!

Oct 30, 2023

Introducing IDEFICS: An Open Reproduction of State-of-the-art Visual Language Model

Aug 22, 2023

• 25

Huggy Lingo: Using Machine Learning to Improve Language Metadata on the Hugging Face Hub

Aug 2, 2023

The Hugging Face Hub for Galleries, Libraries, Archives and Museums

Jun 12, 2023

• 1

Introducing BERTopic Integration with Hugging Face Hub

May 31, 2023

• 4

Organizations

Posts 31

Post

555

Yesterday, I shared a blog post on generating data for fine-tuning ColPali using the Qwen/Qwen2-VL-7B-Instruct model.

To simplify testing this approach, I created a Space that lets you generate queries from an input document page image: davanstrien/ColPali-Query-Generator

I think there is much room for improvement, but I'm excited about the potential for relatively small VLMs to create synthetic data.

You can read the original blog post that goes into more detail here: https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html

Post

1813

ColPali is revolutionizing multimodal retrieval, but could it be even more effective with domain-specific fine-tuning?

Check out my latest blog post, where I guide you through creating a ColPali fine-tuning dataset using Qwen/Qwen2-VL-7B-Instruct to generate queries for a collection of UFO documents sourced from the Internet Archive.

The post covers:
- Introduction to data for ColPali models
- Using Qwen2-VL for retrieval query generation
- Tips for better query generation

Check out the post here:
https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html

The resulting Hugging Face dataset: davanstrien/ufo-ColPali

View all posts