In this section, we present how to connect Twelve Labs API to MindsDB.

Twelve Labs provides a powerful and seamless video search infrastructure for your application.

​
Prerequisites

Before proceeding, ensure the following prerequisites are met:

  1. Install MindsDB on your system or obtain access to cloud options.
  2. To use Twelve Labs with MindsDB, install the required dependencies following this instruction.
  3. Copy your Twelve Labs API Key by navigating to the Dashboard page.

​
AI Engine

The first step to use this handler is to create an ML Engine. The required argument to create the engine is:

  • api_key - The TweleveLabs api key.

Once you copy the api_key you can establish a connection by executing the following SQL command:

CREATE ML_ENGINE twelve_labs_engine
FROM twelve_labs
USING
  api_key = '<YOUR_API_KEY>';

​
AI Model

Now, you can use this ML Engine to create Models for the different tasks supported by the handler.

When executing the CREATE MODEL statement, the following parameters are supported in the USING clause of the query:

  • engine: The name of the ML Engine to use. This is a required parameter.
  • api_key: The Twelve Labs API key to use for authentication. This is a required parameter.
  • task: The task to perform. This is a required parameter and must be one of search or classification.
  • index_name: The name of the index to use; if it does not exist, it will be created. This is a required parameter. More information about indexes can be found here.
  • index_options: A list of the types of information within the video that will be processed by the video understanding engine. This is a required parameter and can be any combination of visual, conversation, text_in_video and logo. More information about index options can be found here.
  • video_urls: A list of URLs to the videos to be indexed. This is an optional parameter, but if not specified, one of video_files, video_urls_col or video_files_col must be specified instead.
  • video_files: A list of local paths to the videos to be indexed. This is an optional parameter, but if not specified, one of video_urls, video_urls_col or video_files_col must be specified instead.
  • video_urls_col: The name of the column containing the URLs to the videos to be indexed. This is an optional parameter, but if not specified, one of video_urls, video_files or video_files_col must be specified instead.
  • video_files_col: The name of the column containing the local paths to the videos to be indexed. This is an optional parameter, but if not specified, one of video_urls, video_files or video_urls_col must be specified instead.
  • search_options: A list of the sources of information to use when performing a search. This parameter is required if the task is search and it should be a subset of index_options. More information about search options can be found here.

Given below are examples of creating Models for each of the supported tasks.

CREATE MODEL mindsdb.twelve_labs_search
PREDICT search_results
USING
  engine = 'twelve_labs_engine',
  api_key = '<YOUR_API_KEY>',
  task = 'search',
  index_name = 'index_1',
  index_options = ['visual', 'conversation', 'text_in_video', 'logo'],
  video_urls = ['https://.../video_1.mp4', 'https://.../video_2.mp4'],
  search_options = ['visual', 'conversation', 'text_in_video', 'logo'],
  query_column = 'query'

As mentioned above, the search_options parameter is specific to the search task and should be a subset of index_options.

​
Making Predictions

Once you have created a Model, you can use it to make predictions.

Given below are examples of making predictions using Models created for each of the supported tasks.

​
Usage

Use the twelve_labs_search model to query the video as:

SELECT *
FROM mindsdb.twelve_labs_search
WHERE query = 'search query';

Here, the query column is a required parameter and should contain the search query.

Replicate img2textReplicate text2video