Skip to main content

One post tagged with "Data Integration"

View All Tags

Introducing Lightfeed API and SDK

· 4 min read
Lightfeed Team

Today we're launching the Lightfeed API along with official TypeScript and Python SDKs (GitHub). These tools provide direct access to structured web data that's been extracted through our platform, making it easy to bring this data into your applications.

Lightfeed API example code

Web Data Made Simple​

The Lightfeed API offers key advantages that transform how you work with web data:

  • Instant data retrieval through simple API calls, eliminating the wait times and costs associated with traditional crawl and scrape operations - your data is pre-extracted and ready for search
  • Intelligent extraction, no manual maintenance: our AI-powered extraction can search connected pages, reason and extract data from context. It adapts to website changes automatically, no need to manually maintain scraping scripts.
  • Clean, consistent, and up-to-date data from any list of websites

This approach shifts your focus from data collection infrastructure to actually using the insights from your web data in your applications.

Key Features​

Semantic search finds content based on meaning, not just keywords.

tip

This API uses AI language model embeddings to understand the meaning behind your search queries, not just matching keywords. When you search for "innovative AI solutions" it can find records about "generative AI solutions" or "large language models" because it understands these concepts are related. It can return more accurate matches than traditional search engines which primarily rely on exact keyword matching.

import { LightfeedClient } from "lightfeed";

// Initialize client with your API key
const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

const response = await client.searchRecords("your-database-id", {
  search: {
    text: "innovative AI solutions",
    threshold: 0.2,
  },
});

console.log(`Found ${response.results.length} matching records`);
console.log(response.results);

Flexible Filtering​

Find exactly what you need by applying specific conditions to narrow down results.

import { LightfeedClient } from "lightfeed";

const client = new LightfeedClient({
  apiKey: "YOUR_API_KEY",
});

const response = await client.filterRecords("your-database-id", {
  filter: {
    rules: [
      {
        column: "industry",
        operator: "equals",
        value: "Finance",
      },
      {
        column: "employees",
        operator: "greater_than",
        value: 100,
      },
    ],
  },
  pagination: {
    limit: 100,
  },
});

console.log(
  `Retrieved ${response.results.length} finance companies with 100+ employees`
);
console.log(response.results);

Time-Range Queries​

Access data from specific time periods to track changes over time.

import { LightfeedClient } from "lightfeed";

const client = new LightfeedClient({
  apiKey: "your-api-key",
});

const response = await client.getRecords("your-database-id", {
  start_time: "2024-12-01T00:00:00Z",
  end_time: "2024-12-31T23:59:59Z",
  limit: 100,
});

console.log(`Retrieved ${response.results.length} records`);
console.log(response);

Getting Started​

Ready to build with web data today? Getting started with Lightfeed API is simple:

  1. Generate your API key in the Lightfeed dashboard
  2. Install our client library: 
    npm install lightfeed   # JavaScript/TypeScript
    pip install lightfeed   # Python

More Resources​