Search Records

The Search Records API enables powerful semantic search of your database records with optional filtering. Results are always returned with the most relevant records first.

tip

This API uses AI language model embeddings to understand the meaning behind your search queries, not just matching keywords. When you search for "innovative solutions," it can find records about "groundbreaking approaches" or "creative answers" because it understands these concepts are related. Think of it as searching by meaning rather than exact words, similar to how humans understand context and meaning, unlike traditional search engines that primarily rely on exact keyword matching.

`POST /v1/databases/{id}/search`

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The ID of your Lightfeed database

Request Body

The search API supports a simplified structure with separate fields for semantic search and filtering:

{
  "search": {
    "text": "innovative AI solutions",
    "threshold": 0.2
  },
  "filter": {
    "condition": "AND",
    "rules": [
      {
        "column": "industry",
        "operator": "equals",
        "value": "Finance"
      },
      {
        "condition": "OR",
        "rules": [
          {
            "column": "employees",
            "operator": "greater_than",
            "value": 100
          },
          {
            "column": "funding_amount",
            "operator": "greater_than",
            "value": 1000000
          }
        ]
      }
    ]
  },
  "time_range": {
    "start_time": "2024-01-01T00:00:00Z",
    "end_time": "2024-12-31T23:59:59Z"
  },
  "pagination": {
    "limit": 100,
    "cursor": "0.23_1234"
  }
}

Example Requests

Basic Semantic Search

This example shows how to perform a basic semantic search without any filters.

cURL
JavaScript
Python

curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/search" \
  -H "x-api-key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "search": {
      "text": "innovative AI solutions",
    },
  }'

import { LightfeedClient } from "lightfeed";

// Initialize client with your API key
const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

async function searchRecords() {
  try {
    const response = await client.searchRecords("your-database-id", {
      search: {
        text: "innovative AI solutions",
        threshold: 0.2,
      },
    });

    console.log(`Found ${response.results.length} matching records`);
    console.log(response.results);
  } catch (error) {
    console.error("Error searching records:", error);
  }
}

searchRecords();

from lightfeed import LightfeedClient

# Initialize client with your API key
client = LightfeedClient({
    "apiKey": "YOUR_API_KEY"
})

def search_records():
    try:
        response = client.search_records("your-database-id", {
            "search": {
                "text": "innovative AI solutions",
                "threshold": 0.2
            }
        })

        print(f"Found {len(response['results'])} matching records")
        print(response["results"])
    except Exception as e:
        print(f"Error searching records: {e}")

search_records()

Search with Filtering and Time Range

This example shows how to combine semantic search with filtering and time range constraints.

cURL
JavaScript
Python

curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/search" \
  -H "x-api-key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "search": {
      "text": "innovative AI solutions",
      "threshold": 0.2
    },
    "filter": {
      "condition": "AND",
      "rules": [
        {
          "column": "industry",
          "operator": "equals",
          "value": "Finance"
        },
        {
          "condition": "OR",
          "rules": [
            {
              "column": "employees",
              "operator": "greater_than",
              "value": 100
            },
            {
              "column": "funding_amount",
              "operator": "greater_than",
              "value": 1000000
            }
          ]
        }
      ]
    },
    "time_range": {
      "start_time": "2024-01-01T00:00:00Z",
      "end_time": "2024-12-31T23:59:59Z"
    },
    "pagination": {
      "limit": 100
    }
  }'

import { LightfeedClient } from "lightfeed";

// Initialize client with your API key
const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

async function searchRecordsWithFilters() {
  try {
    const response = await client.searchRecords("your-database-id", {
      search: {
        text: "innovative AI solutions",
        threshold: 0.2,
      },
      filter: {
        condition: "AND",
        rules: [
          {
            column: "industry",
            operator: "equals",
            value: "Finance",
          },
          {
            condition: "OR",
            rules: [
              {
                column: "employees",
                operator: "greater_than",
                value: 100,
              },
              {
                column: "funding_amount",
                operator: "greater_than",
                value: 1000000,
              },
            ],
          },
        ],
      },
      time_range: {
        start_time: "2024-01-01T00:00:00Z",
        end_time: "2024-12-31T23:59:59Z",
      },
      pagination: {
        limit: 100,
      },
    });

    console.log(`Found ${response.results.length} matching records`);
    console.log(response.results);
  } catch (error) {
    console.error("Error searching records:", error);
  }
}

searchRecordsWithFilters();

from lightfeed import LightfeedClient, Condition, Operator

# Initialize client with your API key
client = LightfeedClient({
    "apiKey": "YOUR_API_KEY"
})

def search_records_with_filters():
    try:
        response = client.search_records("your-database-id", {
            "search": {
                "text": "innovative AI solutions",
                "threshold": 0.2
            },
            "filter": {
                "condition": Condition.AND,
                "rules": [
                    {
                        "column": "industry",
                        "operator": Operator.EQUALS,
                        "value": "Finance"
                    },
                    {
                        "condition": Condition.OR,
                        "rules": [
                            {
                                "column": "employees",
                                "operator": Operator.GREATER_THAN,
                                "value": 100
                            },
                            {
                                "column": "funding_amount",
                                "operator": Operator.GREATER_THAN,
                                "value": 1000000
                            }
                        ]
                    }
                ]
            },
            "time_range": {
                "start_time": "2024-01-01T00:00:00Z",
                "end_time": "2024-12-31T23:59:59Z"
            },
            "pagination": {
                "limit": 100
            }
        })

        print(f"Found {len(response['results'])} matching records")
        print(response["results"])
    except Exception as e:
        print(f"Error searching records: {e}")

search_records_with_filters()

Request Parameters

Search Object (Required)

The semantic search parameters to find records similar to your query text using vector embeddings.

"search": {
  "text": "innovative AI solutions",
  "threshold": 0.2
}

Parameter	Type	Required	Description
`text`	string	Yes	The text to search for semantically
`threshold`	number	No	Minimum relevance score threshold (0-1), default: 0.2. Lower values return more results

How it works: Semantic search converts your query text into a vector embedding and compares it with the embeddings of records in your database. The relevance score ranges from 0 to 1, where values closer to 1 indicate higher relevance. For most use cases, a threshold of 0.2 provides a good balance between precision and recall.

Filter Object (Optional)

Defines filtering conditions to apply to the records after semantic search. Filter uses a logical group structure:

"filter": {
  "condition": "AND", // Optional, defaults to "AND" if not specified
  "rules": [
    // Array of field comparisons or nested logical groups
  ]
}

Field Comparison Rule

Parameter	Type	Required	Description
`column`	string	Yes	Database column name to filter on
`operator`	string	Yes	Comparison operator to use
`value`	any	Yes*	Value to compare against

*Not required for is_empty and is_not_empty operators.

Available Operators:

Operator	Description	Example Value Type
`equals`	Exact match	String, Number, Boolean
`not_equals`	Not an exact match	String, Number, Boolean
`greater_than`	Greater than value	Number
`less_than`	Less than value	Number
`greater_than_or_equals`	Greater than or equal to value	Number
`less_than_or_equals`	Less than or equal to value	Number
`contains`	String contains substring	String
`not_contains`	String does not contain substring	String
`starts_with`	String starts with substring	String
`ends_with`	String ends with substring	String
`is_empty`	Field is empty or null	No value needed
`is_not_empty`	Field is not empty or null	No value needed

Logical Group Rule

Parameter	Type	Required	Description
`condition`	string	No	Logical operator to apply: "AND" or "OR". Defaults to "AND"
`rules`	array	Yes	Array of field comparisons or nested logical groups

Time Range (Optional)

Filter results based on the last seen time of records. If not specified, all records are included regardless of timestamp.

"time_range": {
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-12-31T23:59:59Z"
}

Parameter	Type	Required	Description
`start_time`	string	No	ISO 8601 timestamp for the start of the range
`end_time`	string	No	ISO 8601 timestamp for the end of the range

Pagination (Optional)

Control the number of results returned and navigate through large result sets.

"pagination": {
  "limit": 100,
  "cursor": "0.23_1234"
}

Parameter	Type	Required	Description
`limit`	number	No	Maximum number of results to return (default: 50, max: 100)
`cursor`	string	No	Cursor for pagination (from previous response)

Example Response

{
  "results": [
    {
      "id": 1345,
      "relevance_score": 0.3,
      "data": {
        "company_name": "Acme Inc.",
        "industry": "Finance",
        "website": "https://acme.com",
        "employees": 250,
        "funding_amount": 5000000,
        "is_public": false
      },
      "timestamps": {
        "first_seen_time": "2022-03-15T14:22:13Z",
        "last_changed_time": "2024-07-02T09:45:30Z",
        "last_seen_time": "2024-10-01T18:30:45Z"
      }
    }
    // Additional results...
  ],
  "pagination": {
    "limit": 100,
    "next_cursor": "0.23_1234",
    "has_more": true
  }
}

Response Structure

The response will contain a list of records matching your search criteria, sorted by relevance score (most relevant first).

Field	Type	Description
`results`	array	List of matching records
`pagination`	object	Information for continued querying

Pagination Object

Field	Type	Description
`limit`	number	The requested limit parameter value from the original request
`next_cursor`	string	Token to use for fetching the next page
`has_more`	boolean	Indicates whether more results are available

Record Object

Each record in the results array has the following structure:

Field	Type	Description
`id`	number	Unique identifier for the record
`relevance_score`	number	Ranges from 0 to 1, where values closer to 1 indicate higher relevance (only present if semantic search is applied)
`data`	object	The record's structured data
`timestamps`	object	Timing information for the record

Tips and Best Practices

Combining Search and Filters

When using both semantic search and filters, the API simultaneously evaluates both criteria to find the most relevant results that also match your filtering conditions. This integrated approach ensures optimal results that satisfy both semantic relevance and your specific filtering requirements.

Optimizing Semantic Search

Use natural language phrases rather than single keywords for better results
Start with a lower threshold (e.g., 0.2) and increase it if you get too many irrelevant results
Be specific in your search text to improve precision

Use the Right API

Use this Search API when you need to find records based on semantic relevance to text queries
Use the Filter API when you only need to filter records by exact criteria without semantic search

POST /v1/databases/{id}/search​

Path Parameters​

Request Body​

Example Requests​

Basic Semantic Search​

Search with Filtering and Time Range​

Request Parameters​

Search Object (Required)​

Filter Object (Optional)​

Field Comparison Rule​

Logical Group Rule​

Time Range (Optional)​

Pagination (Optional)​

Example Response​

Response Structure​

Pagination Object​

Record Object​

Tips and Best Practices​

Combining Search and Filters​

Optimizing Semantic Search​

Use the Right API​