Correlations: UI for debugging correlations of text and image embeddings

A simple UI for debugging correlations of text & image embeddings. At Jina AI we use this tool for vibe-checking embeddings, late chunking, multimodal retrieval etc.

Click the image below to watch a video demo on Youtube.

JINA_API_KEY is used for embedding and reading content from a URL when necessary, reading from local text file is of course supported. You can also bring your own embeddings and do npm run corr ... for visualization, in which case you don't need JINA_API_KEY.

You can get a Jina API Key with free 10M tokens from https://jina.ai/#apiform. Otherwise, just do npm run corr -- v3-blog.jsonl v3-arxiv.jsonl for a simple vibe-check with our pre-embedded data.

The correlation visualization is served on http://localhost:3000 by default. Here's what you can do:

• Switch Layout: Click buttons in top-right to toggle between single view, vertical split, or horizontal split
• Hover Cells: See full text and similarity score for any cell
• Draw Selection: Click and drag to select regions for analysis

Always visible in top-right corner, or move the control panel anywhere on screen.

• Adjust Thresholds: Use sliders to filter by cosine similarity score (0-1) and text length
• Change Colors: Switch between RdBu, Viridis, Plasma, or Inferno color schemes

Draw a selection rectangle on the heatmap to open the panel.

• Copy Text: Copy selected content to clipboard
• Close Panel: Click X to dismiss selection

Click "Vertical" or "Horizontal" layout button to open this table.

• Hover Rows: See full text in tooltip
• Scroll: Browse through 100 correlations at the borderline

You can do either npm run embed or npm run corr. Note the double dash -- is required for passing args, it tells npm that the following arguments should be passed to the script rather than being interpreted as npm options.

npm run embed uses Jina Embedding API, you can get a Jina API Key with free 10M tokens from https://jina.ai/#apiform. After that, make sure you do export JINA_API_KEY=your_jina_key_here.

Embed a text file with default settings (newline chunking):

Embed content from a URL:

Behind the scene, it calls Jina Reader to crawl the content behind the given URL.

Visualize self-correlations within a single file:

Visualize correlations between two files:

Use punctuation-based chunking:

Split by character count:

Use custom regex pattern for chunking:

Enable late chunking:

Specify task type for better embeddings:

Custom output file:

Embed images using jina-clip-v2, where each line in all_images.txt can be either a datauri string (without prefix) or an image URL:

Image will be also displayed in the hover tooltip and table when the input is recognized as image. Here's an example of self-correlations of image embeddings:

Correlations of text-image embeddings are also supported.

Change visualization server port:

Compare embeddings with different dimensions:

To use npm run corr, your data must be in a JSONL file where each line is a JSON object with:

• chunk: string The text segment, an image local file path or an image url
• embedding: Array<number> The embedding vector

Example line:

If you bring your own embeddings:

1. Ensure your JSONL file follows the above format.
2. Each line must have both chunk and embedding fields.
3. Place your file where you want and reference it when running npm run corr.

No extra metadata is needed; embeddings must be precomputed and stored as arrays of numbers.