Introducing Lightfeed API and SDK
Today we're launching the Lightfeed API along with official TypeScript and Python SDKs (GitHub). These tools provide direct access to structured web data that's been extracted through our platform, making it easy to bring this data into your applications.
Web Data Made Simple
The Lightfeed API offers key advantages that transform how you work with web data:
- Instant data retrieval through simple API calls, eliminating the wait times and costs associated with traditional crawl and scrape operations - your data is pre-extracted and ready for search
- Intelligent extraction, no manual maintenance: our AI-powered extraction can search connected pages, reason and extract data from context. It adapts to website changes automatically, no need to manually maintain scraping scripts.
- Clean, consistent, and up-to-date data from any list of websites
This approach shifts your focus from data collection infrastructure to actually using the insights from your web data in your applications.
Key Features
Semantic Search
Semantic search finds content based on meaning, not just keywords.
This API uses AI language model embeddings to understand the meaning behind your search queries, not just matching keywords. When you search for "innovative AI solutions" it can find records about "generative AI solutions" or "large language models" because it understands these concepts are related. It can return more accurate matches than traditional search engines which primarily rely on exact keyword matching.
- JavaScript
- Python
- cURL
import { LightfeedClient } from "lightfeed";
// Initialize client with your API key
const client = new LightfeedClient({
apiKey: "YOUR_API_KEY",
});
const response = await client.searchRecords("your-database-id", {
search: {
text: "innovative AI solutions",
threshold: 0.2,
},
});
console.log(`Found ${response.results.length} matching records`);
console.log(response.results);
from lightfeed import LightfeedClient
# Initialize client with your API key
client = LightfeedClient({
"apiKey": "YOUR_API_KEY"
})
response = client.search_records("your-database-id", {
"search": {
"text": "innovative AI solutions",
"threshold": 0.2
}
})
print(f"Found {len(response['results'])} matching records")
print(response["results"])
curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/records/search" \
-H "x-api-key: your-api-key" \
-H "Content-Type: application/json" \
-d '{
"search": {
"text": "innovative AI solutions",
"threshold": 0.2
}
}'
Flexible Filtering
Find exactly what you need by applying specific conditions to narrow down results.
- JavaScript
- Python
- cURL
import { LightfeedClient } from "lightfeed";
const client = new LightfeedClient({
apiKey: "YOUR_API_KEY",
});
const response = await client.filterRecords("your-database-id", {
filter: {
rules: [
{
column: "industry",
operator: "equals",
value: "Finance",
},
{
column: "employees",
operator: "greater_than",
value: 100,
},
],
},
pagination: {
limit: 100,
},
});
console.log(
`Retrieved ${response.results.length} finance companies with 100+ employees`
);
console.log(response.results);
from lightfeed import LightfeedClient, Operator
client = LightfeedClient({
"apiKey": "YOUR_API_KEY"
})
response = client.filter_records("your-database-id", {
"filter": {
"rules": [
{
"column": "industry",
"operator": Operator.EQUALS,
"value": "Finance"
},
{
"column": "employees",
"operator": Operator.GREATER_THAN,
"value": 100
}
]
},
"pagination": {
"limit": 100
}
})
print(f"Retrieved {len(response['results'])} finance companies with 100+ employees")
print(response["results"])
curl -X POST "https://api.lightfeed.ai/v1/databases/your-database-id/records/filter" \
-H "x-api-key: your-api-key" \
-H "Content-Type: application/json" \
-d '{
"filter": {
"rules": [
{
"column": "industry",
"operator": "equals",
"value": "Finance"
},
{
"column": "employees",
"operator": "greater_than",
"value": 100
}
]
},
"pagination": {
"limit": 100
}
}'
Time-Range Queries
Access data from specific time periods to track changes over time.
- JavaScript
- Python
- cURL
import { LightfeedClient } from "lightfeed";
const client = new LightfeedClient({
apiKey: "your-api-key",
});
const response = await client.getRecords("your-database-id", {
start_time: "2024-12-01T00:00:00Z",
end_time: "2024-12-31T23:59:59Z",
limit: 100,
});
console.log(`Retrieved ${response.results.length} records`);
console.log(response);
from lightfeed import LightfeedClient
client = LightfeedClient({
"apiKey": "your-api-key"
})
response = client.get_records("your-database-id", {
"start_time": "2024-12-01T00:00:00Z",
"end_time": "2024-12-31T23:59:59Z",
"limit": 100
})
print(f"Retrieved {len(response['results'])} records")
print(response)
curl "https://api.lightfeed.ai/v1/databases/your-database-id/records?\
start_time=2024-12-01T00:00:00Z&\
end_time=2024-12-31T23:59:59Z&\
limit=100" \
-H "x-api-key: your-api-key"
Getting Started
Ready to build with web data today? Getting started with Lightfeed API is simple:
- Generate your API key in the Lightfeed dashboard
- Install our client library:
npm install lightfeed # JavaScript/TypeScript
pip install lightfeed # Python
More Resources
- Check out our API documentation for detailed guides and advanced examples
- Explore our GitHub repository for the latest updates and sample code
- Book an intro call to learn more or see a live demo of Lightfeed in action
