Filter Records

The Filter API enables you to retrieve records from your database that match specific filtering criteria. Results are always returned in descending order by last seen time (most recent first).

`POST /v1/databases/{id}/records/filter`

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The ID of your Lightfeed database

Request Body

{
  "filter": {
    "condition": "AND", // Optional, defaults to "AND" if not specified
    "rules": [
      {
        "column": "industry",
        "operator": "equals",
        "value": "Finance"
      },
      {
        "condition": "OR",
        "rules": [
          {
            "column": "employees",
            "operator": "greater_than",
            "value": 100
          },
          {
            "column": "funding_amount",
            "operator": "greater_than",
            "value": 1000000
          }
        ]
      }
    ]
  },
  "time_range": {
    "start_time": "2024-01-01T00:00:00Z",
    "end_time": "2024-12-31T23:59:59Z"
  },
  "pagination": {
    "limit": 100,
    "cursor": "2024-07-01T00:00:00Z_1234"
  }
}

Example Requests

Basic Filter

This example shows how to filter finance companies with more than 100 employees.

cURL
JavaScript
Python

curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/records/filter" \
  -H "x-api-key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "filter": {
      "rules": [
        {
          "column": "industry",
          "operator": "equals",
          "value": "Finance"
        },
        {
          "column": "employees",
          "operator": "greater_than",
          "value": 100
        }
      ]
    },
    "pagination": {
      "limit": 100
    }
  }'

import { LightfeedClient } from "lightfeed";

// Initialize client with your API key
const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

async function filterCompanies() {
  try {
    const response = await client.filterRecords("your-database-id", {
      filter: {
        rules: [
          {
            column: "industry",
            operator: "equals",
            value: "Finance",
          },
          {
            column: "employees",
            operator: "greater_than",
            value: 100,
          },
        ],
      },
      pagination: {
        limit: 100,
      },
    });

    console.log(
      `Retrieved ${response.results.length} finance companies with 100+ employees`
    );
    console.log(response.results);
  } catch (error) {
    console.error("Error filtering records:", error);
  }
}

from lightfeed import LightfeedClient, Operator

# Initialize client with your API key
client = LightfeedClient({
    "apiKey": "YOUR_API_KEY"
})

def filter_companies():
    try:
        response = client.filter_records("your-database-id", {
            "filter": {
                "rules": [
                    {
                        "column": "industry",
                        "operator": Operator.EQUALS,
                        "value": "Finance"
                    },
                    {
                        "column": "employees",
                        "operator": Operator.GREATER_THAN,
                        "value": 100
                    }
                ]
            },
            "pagination": {
                "limit": 100
            }
        })

        print(f"Retrieved {len(response['results'])} finance companies with 100+ employees")
        print(response["results"])
    except Exception as e:
        print(f"Error filtering records: {e}")

Complex Filter with Time Range

This example shows how to filter records with more complex conditions including a time range.

cURL
JavaScript
Python

curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/records/filter" \
  -H "x-api-key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "filter": {
      "condition": "AND",
      "rules": [
        {
          "column": "industry",
          "operator": "equals",
          "value": "Finance"
        },
        {
          "condition": "OR",
          "rules": [
            {
              "column": "employees",
              "operator": "greater_than",
              "value": 100
            },
            {
              "column": "funding_amount",
              "operator": "greater_than",
              "value": 1000000
            }
          ]
        }
      ]
    },
    "time_range": {
      "start_time": "2024-01-01T00:00:00Z",
      "end_time": "2024-12-31T23:59:59Z"
    },
    "pagination": {
      "limit": 100
    }
  }'

import { LightfeedClient } from "lightfeed";

// Initialize client with your API key
const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

async function filterCompaniesWithComplexCriteria() {
  try {
    const response = await client.filterRecords("your-database-id", {
      filter: {
        condition: "AND",
        rules: [
          {
            column: "industry",
            operator: "equals",
            value: "Finance",
          },
          {
            condition: "OR",
            rules: [
              {
                column: "employees",
                operator: "greater_than",
                value: 100,
              },
              {
                column: "funding_amount",
                operator: "greater_than",
                value: 1000000,
              },
            ],
          },
        ],
      },
      time_range: {
        start_time: "2024-01-01T00:00:00Z",
        end_time: "2024-12-31T23:59:59Z",
      },
      pagination: {
        limit: 100,
      },
    });

    console.log(`Found ${response.results.length} matching companies`);
    console.log(response.results);
  } catch (error) {
    console.error("Error filtering records:", error);
  }
}

from lightfeed import LightfeedClient, Condition, Operator

# Initialize client with your API key
client = LightfeedClient({
    "apiKey": "YOUR_API_KEY"
})

def filter_companies_with_complex_criteria():
    try:
        response = client.filter_records("your-database-id", {
            "filter": {
                "condition": Condition.AND,
                "rules": [
                    {
                        "column": "industry",
                        "operator": Operator.EQUALS,
                        "value": "Finance"
                    },
                    {
                        "condition": Condition.OR,
                        "rules": [
                            {
                                "column": "employees",
                                "operator": Operator.GREATER_THAN,
                                "value": 100
                            },
                            {
                                "column": "funding_amount",
                                "operator": Operator.GREATER_THAN,
                                "value": 1000000
                            }
                        ]
                    }
                ]
            },
            "time_range": {
                "start_time": "2024-01-01T00:00:00Z",
                "end_time": "2024-12-31T23:59:59Z"
            },
            "pagination": {
                "limit": 100
            }
        })

        print(f"Found {len(response['results'])} matching companies")
        print(response["results"])
    except Exception as e:
        print(f"Error filtering records: {e}")

Request Parameters

Filter Object (Required)

Defines filtering conditions to apply to the records. Filter uses a logical group structure:

"filter": {
  "condition": "AND", // Optional, defaults to "AND" if not specified
  "rules": [
    // Array of field comparisons or nested logical groups
  ]
}

Field Comparison Rule

Parameter	Type	Required	Description
`column`	string	Yes	Database column name to filter on
`operator`	string	Yes	Comparison operator to use
`value`	any	Yes*	Value to compare against

*Not required for is_empty and is_not_empty operators.

Available Operators:

Operator	Description	Example Value Type
`equals`	Exact match	String, Number, Boolean
`not_equals`	Not an exact match	String, Number, Boolean
`greater_than`	Greater than value	Number
`less_than`	Less than value	Number
`greater_than_or_equals`	Greater than or equal to value	Number
`less_than_or_equals`	Less than or equal to value	Number
`contains`	String contains substring	String
`not_contains`	String does not contain substring	String
`starts_with`	String starts with substring	String
`ends_with`	String ends with substring	String
`is_empty`	Field is empty or null	No value needed
`is_not_empty`	Field is not empty or null	No value needed

Logical Group Rule

Parameter	Type	Required	Description
`condition`	string	No	Logical operator to apply: "AND" or "OR". Defaults to "AND"
`rules`	array	Yes	Array of field comparisons or nested logical groups

Time Range (Optional)

Filter results based on the last seen time of records. If not specified, all records are included regardless of timestamp.

"time_range": {
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-12-31T23:59:59Z"
}

Parameter	Type	Required	Description
`start_time`	string	No	ISO 8601 timestamp for the start of the range
`end_time`	string	No	ISO 8601 timestamp for the end of the range

Pagination (Optional)

Control the number of results returned and navigate through large result sets.

"pagination": {
  "limit": 100,
  "cursor": "2024-07-01T00:00:00Z_1234"
}

Parameter	Type	Required	Description
`limit`	number	No	Maximum number of results to return (default: 50, max: 100)
`cursor`	string	No	Cursor for pagination (from previous response)

Example Response

{
  "results": [
    {
      "id": "1345",
      "data": {
        "company_name": "Acme Inc.",
        "industry": "Finance",
        "website": "https://acme.com",
        "employees": 250,
        "funding_amount": 5000000,
        "is_public": false
      },
      "timestamps": {
        "first_seen_time": "2022-03-15T14:22:13Z",
        "last_changed_time": "2024-07-02T09:45:30Z",
        "last_seen_time": "2024-10-01T18:30:45Z"
      }
    }
    // Additional results...
  ],
  "pagination": {
    "limit": 100,
    "next_cursor": "2024-07-01T00:00:00Z_1234",
    "has_more": true
  }
}

Response Structure

The response contains a list of records matching your filter criteria, sorted by last seen time (most recent first).

Field	Type	Description
`results`	array	List of matching records
`pagination`	object	Information for continued querying

Pagination Object

Field	Type	Description
`limit`	number	The requested limit parameter value from the original request
`next_cursor`	string	Token to use for fetching the next page
`has_more`	boolean	Indicates whether more results are available

Record Object

Each record in the results array has the following structure:

Field	Type	Description
`id`	string	Unique identifier for the record
`data`	object	The record's structured data
`timestamps`	object	Timing information for the record

Tips and Best Practices

Effective Filtering

Start with broad filters and progressively add constraints to narrow down results
Use nested conditions to create complex filtering logic
When filtering on text fields, consider using contains instead of equals for more flexible matching

Filter Examples

Simple single condition filter:

"filter": {
  "rules": [
    {
      "column": "industry",
      "operator": "equals",
      "value": "Finance"
    }
  ]
}

Multiple conditions with AND logic (default):

"filter": {
  "rules": [
    {
      "column": "industry",
      "operator": "equals",
      "value": "Finance"
    },
    {
      "column": "employees",
      "operator": "greater_than",
      "value": 100
    }
  ]
}

Multiple conditions with OR logic:

"filter": {
  "condition": "OR",
  "rules": [
    {
      "column": "industry",
      "operator": "equals",
      "value": "Finance"
    },
    {
      "column": "industry",
      "operator": "equals",
      "value": "Technology"
    }
  ]
}

Optimizing for Performance

Be specific with your filters to reduce the result set
Use time ranges to limit the data being searched
Combine multiple condition types effectively

POST /v1/databases/{id}/records/filter​

Path Parameters​

Request Body​

Example Requests​

Basic Filter​

Complex Filter with Time Range​

Request Parameters​

Filter Object (Required)​

Field Comparison Rule​

Logical Group Rule​

Time Range (Optional)​

Pagination (Optional)​

Example Response​

Response Structure​

Pagination Object​

Record Object​

Tips and Best Practices​

Effective Filtering​

Filter Examples​

Optimizing for Performance​