Skip to main content

Extraction Schedule

Lightfeed's scheduling feature allows you to automate your data extraction process, ensuring your database stays up-to-date with no manual intervention.

Scheduling Options​

  • Daily: Run extractions every day at specified times
  • Weekly: Run extractions on specific days of the week
  • Multiple times per day: Schedule several extraction runs throughout the day
  • Any days and at any hours: Run extractions on specific days of the week and on specific hours of the day

Why Schedule Extractions?​

Scheduling regular extractions provides several key benefits:

  • Up-to-date data: Always have the most current data available
  • Consistency: Extract data at regular intervals for better trend analysis
  • Change tracking: Automatically detect and record changes to your data
  • Historical records: Build a comprehensive history of data over time

During each scheduled extraction, Lightfeed will extract data and follow the Deduplication Process to update the database.

Best Practices for Scheduling​

To get the most out of Lightfeed's scheduling capabilities:

Match Frequency to Data Volatility​

  • Schedule more frequent extractions for rapidly changing data
  • Use less frequent schedules for more stable information

Consider Time of Day​

  • Schedule extractions when source websites are likely to be updated
  • For international sources, consider time zone differences

Start Conservative​

  • Begin with less frequent extractions
  • Adjust schedules if needed based on observed data patterns

Pausing and Modifying Schedules​

You can pause or modify your extraction schedule at any time:

  • Pause: Temporarily stop scheduled extractions
  • Resume: Restart scheduled extractions
  • Modify: Change days or times for extractions

This flexibility allows you to adjust your data collection strategy as your needs evolve.

Last updated on